Exploiting Deep Reinforcement Learning for Coverage Maximization and Cost Minimization in Air Quality Monitoring Systems
Main Article Content
Abstract
Air-quality monitoring is highly desired in modern life where an environmental problem has become more and more serious. Such a task requires continuous surveillance over large urban areas, which is costly both in infrastructure and sensor resources. Hence gives rise to the Vehicular Mobile Networks (VMNs), in which mobile vehicles play the role of sensor devices and constantly monitor the area. However, with extensive constraints, the optimization of both maximizing the coverage and minimizing the sensory costs is vastly challenging. In this research, we resolve the problem in terms of a learning process. Applying deep reinforcement learning, we outperform more than 1.65% in terms of coverage, compared to common setups while remaining considerably small sensory costs in terms of sensor activation rate. We conduct extensive experiments for a better understanding of the behavior of the deep reinforcement learning model.
Keywords
Reinforcement Learning, VMNs, Energy optimization
Article Details

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
[1] P. Zhu, J. Xu, J. Li, D. Wang, X. You, Learningempowered privacy preservation in beyond 5G edge
intelligence networks. IEEE Wireless
Communications, vol. 28, no. 2, pp. 12-18, April 2021,
https://doi.org/10.1109/MWC.001.2000331.
[2] J. Fu, J. Hua, J. Wen, K. Zhou, J. Li, B. Sheng,
Optimization of achievable rate in the multi-user
satellite IoT system with SWIPT and MEC. IEEE
Trans. Ind. Inform, vol 17, no. 3, pp. 2072-2080,
March 2021,
https://doi.org/10.1109/TII.2020.2985157.
[3] L. Yang, H. Zhang, M. Li, J. Guo, and H. Ji, Mobile
edge computing empowered energy efficient task
offloading in 5G, IEEE Transactions on Vehicular
Technology, vol. 67, no. 7, pp. 6398-6409, July 2018,
https://doi.org/10.1109/TVT.2018.2799620.
[4] T. Q. Dinh, J. Tang, Q. D. La, and T. Q. S. Quek,
Offloading in mobile edge computing: Task allocation
and computational frequency scaling, IEEE
Transactions on Communications, vol. 65, no. 8, Aug.
2017, pp. 3571-3584,
https://doi.org/10.1109/TCOMM.2017.2699660.
[5] F. Wang, J. Xu, X. Wang, and S. Cui, Joint offloading
and computing optimization in wireless powered
mobile-edge computing systems, IEEE Transactions
on Wireless Communications, vol. 17, no. 3, March
2018, pp. 1784-1797,
https://doi.org/10.1109/TWC.2017.2785305.
[6] M. Chen and Y. Hao, Task offloading for mobile edge
computing in software defined ultra-dense network,
IEEE Journal on Selected Areas in Communications,
vol. 36, no. 3, March 2018, pp. 587-597,
https://doi.org/10.1109/JSAC.2018.2815360.
[7] M. Emara, M. C. Filippou, and D. Sabella, MECassisted end-to-end latency evaluations for C-V2X
communications, in 2018 European Conference on
Networks and Communications (EuCNC), Jun. 2018,
pp. 1-9,
https://doi.org/10.1109/EuCNC.2018.8442825.
[8] P. Dong, X. Wang, J. Rodrigues, Deep reinforcement
learning for vehicular edge computing: An intelligent
offloading system, ACM Transactions on Intelligent
Systems and Technology, vol. 10, no. 6, Nov. 2019,
pp. 1-24,
https://doi.org/10.1145/3317572.
[9] H. Zhang, Z. Wang, K. Liu, V2X offloading and
resource allocation in SDN-assisted mec-based
vehicular networks, China Communications, vol. 17,
no. 05, May 2020, pp. 274-291,
https://doi.org/10.23919/JCC.2020.05.020.
[10] Y. Cui, Y. Liang, R. Wang, Resource allocation
algorithm with multi-platform intelligent offloading in
D2D-enabled vehicular networks, IEEE Access, vol. 7,
2019, pp. 21246-21253,
https://doi.org/10.1109/ACCESS.2018.2882000.
[11] X. Xu, Y. Xue, X. Li, L. Qi, S. Wan, A computation
offloading method for edge computing with vehicleto-everything, IEEE Access, vol. 7, 2019, pp. 131068-
131077,
https://doi.org/10.1109/ACCESS.2019.2940295.
[12] Z. Xiao, X. Dai, H. Jiang, D. Wang, H. Chen, L. Yang,
F. Zeng, Vehicular task offloading via heat-aware
MEC cooperation using game-theoretic method, IEEE
Internet of Things Journal, vol. 7, no. 3, March 2020,
pp. 2038-2052,
https://doi.org/10.1109/JIOT.2019.2960631.
intelligence networks. IEEE Wireless
Communications, vol. 28, no. 2, pp. 12-18, April 2021,
https://doi.org/10.1109/MWC.001.2000331.
[2] J. Fu, J. Hua, J. Wen, K. Zhou, J. Li, B. Sheng,
Optimization of achievable rate in the multi-user
satellite IoT system with SWIPT and MEC. IEEE
Trans. Ind. Inform, vol 17, no. 3, pp. 2072-2080,
March 2021,
https://doi.org/10.1109/TII.2020.2985157.
[3] L. Yang, H. Zhang, M. Li, J. Guo, and H. Ji, Mobile
edge computing empowered energy efficient task
offloading in 5G, IEEE Transactions on Vehicular
Technology, vol. 67, no. 7, pp. 6398-6409, July 2018,
https://doi.org/10.1109/TVT.2018.2799620.
[4] T. Q. Dinh, J. Tang, Q. D. La, and T. Q. S. Quek,
Offloading in mobile edge computing: Task allocation
and computational frequency scaling, IEEE
Transactions on Communications, vol. 65, no. 8, Aug.
2017, pp. 3571-3584,
https://doi.org/10.1109/TCOMM.2017.2699660.
[5] F. Wang, J. Xu, X. Wang, and S. Cui, Joint offloading
and computing optimization in wireless powered
mobile-edge computing systems, IEEE Transactions
on Wireless Communications, vol. 17, no. 3, March
2018, pp. 1784-1797,
https://doi.org/10.1109/TWC.2017.2785305.
[6] M. Chen and Y. Hao, Task offloading for mobile edge
computing in software defined ultra-dense network,
IEEE Journal on Selected Areas in Communications,
vol. 36, no. 3, March 2018, pp. 587-597,
https://doi.org/10.1109/JSAC.2018.2815360.
[7] M. Emara, M. C. Filippou, and D. Sabella, MECassisted end-to-end latency evaluations for C-V2X
communications, in 2018 European Conference on
Networks and Communications (EuCNC), Jun. 2018,
pp. 1-9,
https://doi.org/10.1109/EuCNC.2018.8442825.
[8] P. Dong, X. Wang, J. Rodrigues, Deep reinforcement
learning for vehicular edge computing: An intelligent
offloading system, ACM Transactions on Intelligent
Systems and Technology, vol. 10, no. 6, Nov. 2019,
pp. 1-24,
https://doi.org/10.1145/3317572.
[9] H. Zhang, Z. Wang, K. Liu, V2X offloading and
resource allocation in SDN-assisted mec-based
vehicular networks, China Communications, vol. 17,
no. 05, May 2020, pp. 274-291,
https://doi.org/10.23919/JCC.2020.05.020.
[10] Y. Cui, Y. Liang, R. Wang, Resource allocation
algorithm with multi-platform intelligent offloading in
D2D-enabled vehicular networks, IEEE Access, vol. 7,
2019, pp. 21246-21253,
https://doi.org/10.1109/ACCESS.2018.2882000.
[11] X. Xu, Y. Xue, X. Li, L. Qi, S. Wan, A computation
offloading method for edge computing with vehicleto-everything, IEEE Access, vol. 7, 2019, pp. 131068-
131077,
https://doi.org/10.1109/ACCESS.2019.2940295.
[12] Z. Xiao, X. Dai, H. Jiang, D. Wang, H. Chen, L. Yang,
F. Zeng, Vehicular task offloading via heat-aware
MEC cooperation using game-theoretic method, IEEE
Internet of Things Journal, vol. 7, no. 3, March 2020,
pp. 2038-2052,
https://doi.org/10.1109/JIOT.2019.2960631.