Beacon messages and emergency messages in vehicular ad hoc networks (VANETs) require a lower delay and higher reliability. The optimal MAC protocol can effectively reduce data collision in VANETs communication, thus minimizing delay and improving reliability. In this paper, we propose a Q-learning MAC protocol based on detecting the number of two-hop neighbors. The number of two-hop neighbors in highway scenarios is calculated with very little overhead using the beacon messages and neighbor locations to reduce the impact of hidden nodes. Vehicle nodes are regarded as agents, using Q-learning and beacon messages to train the near-optimal contention window value of the MAC layer under different vehicle densities to reduce the collision probability of beacon messages. Furthermore, based on the contention window value after training, a multi-hop broadcast protocol combined with contention window adjustment for emergency messages in highway scenarios is proposed to reduce forwarding delay and improve forwarding reliability. We use the trained contention window value and the state information of neighboring vehicles to assign an appropriate forwarding waiting time to the forwarding node. Simulation experiments are conducted to evaluate the proposed MAC protocol and multi-hop broadcast protocol and compare them with other related protocols. The results show that our proposed protocols outperform the other related protocols on several different evaluation metrics.