V2V Routing in VANET Based on Heuristic Q-Learning
Keywords:
V2V communication, Q-learning, VANETs, ITS, Heuristic Q-learningAbstract
Designing efficient routing algorithms in vehicular ad hoc networks (VANETs) plays an important role in the emerging intelligent transportation systems. In this paper, a routing algorithm based on the improved Q-learning is proposed for vehicle-to-vehicle (V2V) communications in VANETs. Firstly, a link maintenance time model is established, and the maintenance time is taken as an important parameter in the design of routing algorithm to ensure the reliability of each hop link. Aiming at the low efficiency and slow convergence of Q-learning, heuristic function and evaluation function are introduced to accelerate the update of Q-value of current optimal action, reduce unnecessary exploration, accelerate the convergence speed of Q-learning process and improve learning efficiency. The learning task is dispersed in each vehicle node in the new routing algorithm and it maintains the reliable routing path by periodically exchanging beacon information with surrounding nodes, guides the node’s forwarding action by combining the delay information between nodes to improve the efficiency of data forwarding. The performance of the algorithm is evaluated by NS2 simulator. The results show that the algorithm has a good effect on the package delivery rate and end-to-end delay.
References
[2] Agrawal, S.; Tyagi, N.; Iqbal, A.; Rao, R.S. (2020). An intelligent greedy position-based multi-hop routing algorithm for next-hop node selection in VANETs, Proceedings of the National Academy of Sciences, India Section A: Physical Sciences, 90(1), 39-47, 2020. https://doi.org/10.1007/s40010-018-0556-9
[3] Cao, W.J. (2016). Optimization of AODV protocol for vehicular network, Nanjing University of Aeronautics and Astronautics, 2016.
[4] Ghazzai, H.; Ghorbel, M.B.; Kadri, A.; Hossain, M.J.; Menouar, H. (2017). Energy-efficient management of unmanned aerial vehicles for underlay cognitive radio systems, IEEE Transactions on Green Communications and Networking, 1(4), 434-443, 2017. https://doi.org/10.1109/TGCN.2017.2750721
[5] Karp, B.; Kung, H.T. (2000). GPSR: Greedy perimeter stateless routing for wireless networks, In Proceedings of the 6th annual international conference on Mobile computing and networking, 243-254, 2020. https://doi.org/10.1145/345910.345953
[6] Kim, S. (2019). Effective crowdsensing and routing algorithms for next generation vehicular networks, Wireless Networks, 25(4), 1815-1827, 2019. https://doi.org/10.1007/s11276-017-1632-9
[7] Kumar, I.; Sachan, V.; Shankar, R.; Mishra, R.K. (2018). An investigation of wireless S-DF hybrid satellite terrestrial relaying network over time selective fading channel, Traitement Du Signal, 35(2), 103-120, 2018. https://doi.org/10.3166/ts.35.103-120
[8] Li, C.; Han, J.H.; Wei, Z.C. (2015). GPSR-R routing algorithm in VANET Scenario, Journal of Hefei University of Technology, 38(2), 181-185, 2015.
[9] Li, F.; Song, X.; Chen, H.; Li, X.; Wang, Y. (2018). Hierarchical routing for vehicular ad hoc networks via reinforcement learning, IEEE Transactions on Vehicular Technology, 68(2), 1852- 1865, 2018. https://doi.org/10.1109/TVT.2018.2887282
[10] Li, Y.; Shi, D.; Bu, F. (2019). Automatic recognition of rock images based on convolutional neural network and discrete cosine transform, Traitement du Signal, 36(5), 463-469, 2019. https://doi.org/10.18280/ts.360512
[11] Li, G.; Sun, Q.; Boukhatem, L., Wu, J.S.; Yang, J. (2019). Intelligent vehicle-to-vehicle charging navigation for mobile electric vehicles via VANET-based communication, IEEE Access, 7, 170888- 170906, 2019. https://doi.org/10.1109/ACCESS.2019.2955927
[12] Lin, D.; Kang, J.; Squicciarini, A.; Wu, Y.; Gurung, S.; Tonguz, O. (2016). MoZo: A moving zone based routing protocol using pure V2V communication in VANETs, IEEE Transactions on Mobile Computing, 16(5), 1357-1370, 2016. https://doi.org/10.1109/TMC.2016.2592915
[13] Perkins, C.; Belding-Royer, E.; Das, S. (2003). RFC3561: Ad hoc on-demand distance vector (AODV) routing, IEEE Personal Communication, 36-45, 1997. https://doi.org/10.17487/rfc3561
[14] Safiulina, A.M.; Ivanets, D.V.; Kudryavtsev, E.M.; Baulin, D.V.; Baulin, V.E.; Tsivadze, A.Y. (2019). Liquid- and solid-phase extraction of uranium(VI), thorium(IV), and rare earth elements( III) from nitric acid solutions using acid-type phosphoryl-containing podands, Russian Journal of Inorganic Chemistry, 64(4), 536-542, 2019. https://doi.org/10.1134/S0036023619040181
[15] Schoeneich, R.O.; Prus, P. (2018). Improving DTNs performance by reduction of bundles redundancy using clustering algorithm, International Journal of Computers Communications & Control (IJCCC), 13(4), 550-565, 2018. https://doi.org/10.15837/ijccc.2018.4.3180
[16] Senthilkumar, R.; Tamilselvan, G.M.; Kanithan, S.; Arun Vignesh, N. (2019). Routing in WSNs powered by a hybrid energy storage system through a CEAR protocol based on cost welfare and route score metric, International Journal of Computers Communications & Control (IJCCC), 14(2), 233-252, 2019. https://doi.org/10.15837/ijccc.2019.2.3184
[17] Venkatramana, D.K.N.; Srikantaiah S.B.; Moodabidri, J. (2018). CISRP: Connectivity-aware intersection-based shortest path routing protocol for VANETs in urban environments, Iet Networks, 7(3), 152-161, 2018. https://doi.org/10.1049/iet-net.2017.0012
[18] Wu, C.; Kumekawa, K.; Kato, T. (2010). Distributed reinforcement learning approach for vehicular ad hoc networks, IEICE transactions on communications, 93(6), 1431-1442, 2010. https://doi.org/10.1587/transcom.E93.B.1431
[19] Wu, Z.; Chen, J.; Sun, X.Y.; Qu, L.D. (2017). AODV routing method based on prediction of nodes' moving direction, Computer Engineering and Design, 38(9), 2296-2301, 2017.
[20] Xiao, D.G.; Peng, L.X.; Song, D. (2012). Improved GPSR routing algorithm in hybrid VANET environment, Journal of Software, 23(S1), 100-107, 2012.
[21] Xiao, D.G.; Peng, L.X.; Song, D. (2012). Improved GPSR routing algorithm in hybrid VANET environment, Journal of Software, 23(S1), 100-107, 2012.
[22] Xiao, L.; Lu, X.; Xu, D.; Tang, Y.; Wang, L.; Zhuang, W. (2018). UAV relay in VANETs against smart jamming with reinforcement learning, IEEE Transactions on Vehicular Technology, 67(5), 4087-4097, 2018. https://doi.org/10.1109/TVT.2018.2789466
[23] Yao, L.; Wang, J.; Wang, X.; Chen, A.L.; Wang, Y.Q. (2017). V2X routing in a VANET based on the hidden Markov model, IEEE Transactions on Intelligent Transportation Systems, 1-11, 2017.
[24] Yuan, M. (2017). Research on VANET routing algorithm based on reinforcement learning, Xi'an: Xi'an University of Electronic Science and technology, 2017.
[25] Zhang, X.L.; Zhao, Q.; Zhang, T. (2016). Improving GPSR routing protocol in vehicular Ad Hoc network, Journal of Highway & Transportation Research & Development, 11(4), 98-103, 2016. https://doi.org/10.1061/JHTRCQ.0000601
[26] Zhang, D.G.; Ge, H.; Liu, X.H.; Zhang, X.D.; Li, W.B. (2018). A new adaptive routing algorithm based on Q-Learning strategy, Journal of Electronics, 46(10), 2325-2332, 2018.
Additional Files
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
