Real-Time Vehicle Monitoring: A Unified Framework for Detection, Tracking, and Behavioural Classification
DOI:
https://doi.org/10.26437/mqagpa72Keywords:
Behaviour classification. DeepSORT. real-time monitoring. vehicle detection.YOLOv8Abstract
Purpose: This paper proposes a unified framework that integrates YOLOv8s for accurate object detection and classification, DeepSORT for robust multi-object tracking, and an attention-based LSTM model for analysing temporal vehicle behaviours in urban environments.
Design/Methodology/Approach: The proposed framework was evaluated using the UAVDT dataset through a structured methodology. Initially, YOLOv8s was trained to detect and classify vehicles using appropriate preprocessing and training configurations. Subsequently, DeepSORT was employed to associate detected objects across frames and maintain consistent tracking identities. Temporal features extracted from object trajectories were then fed into the LSTM-Attention model to recognise vehicle behaviour patterns.
Research Limitations: The system’s performance may be affected by class imbalance in the dataset and challenges in recognising transitional or ambiguous behaviours in highly complex traffic scenarios. Additionally, deployment on resource-constrained UAV platforms requires further optimisation.
Findings: Experimental results demonstrate strong performance, achieving an overall detection precision of 89.8%, a recall of 75.3%, and an mAP@50 of 82.2%. The DeepSORT tracker achieved robust identity preservation with an IDF1 score of 87.9%, even in dense urban environments. Furthermore, the behaviour recognition module achieved an overall F1-score of 0.93, confirming the effectiveness of the proposed system across various behavioural scenarios.
Practical Implications: The proposed framework can be effectively deployed in intelligent transportation systems and UAV-based monitoring platforms to enhance traffic management, improve surveillance efficiency, and support real-time decision-making.
Social Implications: The system helps reduce traffic accidents by enabling early detection of risky driving behaviours and supporting smart city surveillance systems, thereby improving public safety.
Originality/Value: The novelty of this work lies in integrating detection, tracking, and temporal behaviour analysis within a single unified framework, along with the use of an attention-based LSTM for improved behaviour recognition in real-world urban traffic scenarios.
References
REFERENCES
Al Mudawi, N., Qureshi, A. M., Abdelhaq, M., Alshahrani, A., Alazeb, A., Alonazi, M., & Algarni, A. (2023). Vehicle detection and classification via YOLOv8 and deep belief network over aerial image sequences. Sustainability, 15(19), 14597. DOI: https://doi.org/10.3390/su151914597
Alrayes, F. S., Ahmad, N., Alshuhail, A., Alshammeri, M., Alqazzaz, A., Alkhiri, H., Alqurni, J. S., & Said, Y. (2025a). Convolutional transform learning based fusion framework for scale invariant long term target detection and tracking in unmanned aerial vehicles. Scientific Reports, 15(1), 28248.
Alrayes, F. S., Ahmad, N., Alshuhail, A., Alshammeri, M., Alqazzaz, A., Alkhiri, H., Alqurni, J. S., & Said, Y. (2025b). Convolutional transform learning based fusion framework for scale invariant long term target detection and tracking in unmanned aerial vehicles. Scientific Reports, 15(1), 28248. DOI: https://doi.org/10.1038/s41598-025-09652-1
Alsaihati, F., Aldossary, H., Alzamil, R., Almadan, R., Al Mousa, Z., & Alahmadi, A. (2025). Waste Classification and Detection Model Using YOLOv8 for Waste Management. In Integrating Big Data and IoT for Enhanced Decision-Making Systems in Business: Volume 2 (pp. 473–484). Springer. DOI: https://doi.org/10.1007/978-3-031-97613-1_39
Al-Selwi, S. M., Hassan, M. F., Abdulkadir, S. J., Muneer, A., Sumiea, E. H., Alqushaibi, A., & Ragab, M. G. (2024). RNN-LSTM: From applications to modeling techniques and beyond—Systematic review. Journal of King Saud University-Computer and Information Sciences, 36(5), 102068. DOI: https://doi.org/10.1016/j.jksuci.2024.102068
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv Preprint arXiv:1409.0473.
Bartlett, B., Santos, M., Dorian, T., Moreno, M., Trslic, P., & Dooly, G. (2025). Real-time UAV surveys with the modular detection and targeting system: Balancing wide-area coverage and high-resolution precision in wildlife monitoring. Remote Sensing, 17(5), 879. DOI: https://doi.org/10.3390/rs17050879
Bernardin, K., & Stiefelhagen, R. (2008). Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 2008(1), 246309. DOI: https://doi.org/10.1155/2008/246309
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv Preprint arXiv:2004.10934.
Bouguettaya, A., Zarzour, H., Kechida, A., & Taberkit, A. M. (2021). Vehicle detection from UAV imagery with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, 33(11), 6047–6067. DOI: https://doi.org/10.1109/TNNLS.2021.3080276
Chang, H., & Wang, Z. (2024). UAV vehicle detection system based on YOLOv8. 2872(1), 012019. DOI: https://doi.org/10.1088/1742-6596/2872/1/012019
Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., & Tian, Q. (2018). The unmanned aerial vehicle benchmark: Object detection and tracking. 370–386. DOI: https://doi.org/10.1007/978-3-030-01249-6_23
Du, Y., Zhao, Z., Song, Y., Zhao, Y., Su, F., Gong, T., & Meng, H. (2023). Strongsort: Make deepsort great again. IEEE Transactions on Multimedia, 25, 8725–8737. DOI: https://doi.org/10.1109/TMM.2023.3240881
Estrada-Solano, F., Caicedo, O. M., & S da Fonseca, N. L. (n.d.). An Approach Based on Incremental Deep Learning and Traffic-Flow Characteristics for Scheduling Elephant Flows in Software-Defined Data Center Networks. Nelson L., An Approach Based on Incremental Deep Learning and Traffic-Flow Characteristics for Scheduling Elephant Flows in Software-Defined Data Center Networks.
Financial Times, F. T. (2025). China made millions of drones. Now it has to find uses for them. https://www.ft.com/content/65e4bb30-b04c-4f0e-8686-effb96398999
Ghosh, K., Bellinger, C., Corizzo, R., Branco, P., Krawczyk, B., & Japkowicz, N. (2024). The class imbalance problem in deep learning. Machine Learning, 113(7), 4845–4901. DOI: https://doi.org/10.1007/s10994-022-06268-8
Hanzla, M., Yusuf, M. O., Al Mudawi, N., Sadiq, T., Almujally, N. A., Rahman, H., Alazeb, A., & Algarni, A. (2024). Vehicle recognition pipeline via DeepSort on aerial image datasets. Frontiers in Neurorobotics, 18, 1430155. DOI: https://doi.org/10.3389/fnbot.2024.1430155
Hermens, F. (2024). Automatic object detection for behavioural research using YOLOv8. Behavior Research Methods, 56(7), 7307–7330. DOI: https://doi.org/10.3758/s13428-024-02420-5
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. DOI: https://doi.org/10.1162/neco.1997.9.8.1735
Hung, I.-K., Unger, D., Kulhavy, D., & Zhang, Y. (2019). Positional precision analysis of orthomosaics derived from drone captured aerial imagery. Drones, 3(2), 46. DOI: https://doi.org/10.3390/drones3020046
IMARC Group, I. G. (2025). Drones market size, share, trends and forecast by type, component, payload, point of sale, end-use industry, and region 2025–2033. https://www.imarcgroup.com
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. DOI: https://doi.org/10.1115/1.3662552
Krichen, M., & Mihoub, A. (2025). Long short-term memory networks: A comprehensive survey. AI, 6(9), 215. DOI: https://doi.org/10.3390/ai6090215
Kumar, N., Acharya, D., & Lohani, D. (2020). An IoT-based vehicle accident detection and classification system using sensor fusion. IEEE Internet of Things Journal, 8(2), 869–880. DOI: https://doi.org/10.1109/JIOT.2020.3008896
Kumar, S., Jain, A., Rani, S., Alshazly, H., Idris, S. A., & Bourouis, S. (2022). Deep Neural Network Based Vehicle Detection and Classification of Aerial Images. Intelligent Automation & Soft Computing, 34(1). DOI: https://doi.org/10.32604/iasc.2022.024812
Li, C., Zhao, R., Wang, Z., Xu, H., & Zhu, X. (2025). Remdet: Rethinking efficient model design for uav object detection. 39(5), 4643–4651. DOI: https://doi.org/10.1609/aaai.v39i5.32490
Li, X., Li, X., Li, Z., Xiong, X., Khyam, M. O., & Sun, C. (2021). Robust vehicle detection in high-resolution aerial images with imbalanced data. IEEE Transactions on Artificial Intelligence, 2(3), 238–250. DOI: https://doi.org/10.1109/TAI.2021.3081057
Liu, S., Li, X., Lu, H., & He, Y. (2022). Multi-object tracking meets moving UAV. 8876–8885. DOI: https://doi.org/10.1109/CVPR52688.2022.00867
Liu, S., Shen, X., Xiao, S., Li, H., & Tao, H. (2025a). A Multi-Scale Feature-Fusion Multi-Object Tracking Algorithm for Scale-Variant Vehicle Tracking in UAV Videos. Remote Sensing, 17(6), 1014.
Liu, S., Shen, X., Xiao, S., Li, H., & Tao, H. (2025b). A Multi-Scale Feature-Fusion Multi-Object Tracking Algorithm for Scale-Variant Vehicle Tracking in UAV Videos. Remote Sensing, 17(6), 1014. DOI: https://doi.org/10.3390/rs17061014
Loukinas, P. (2022). Drones for border surveillance: Multipurpose use, uncertainty and challenges at EU borders. Geopolitics, 27(1), 89–112. DOI: https://doi.org/10.1080/14650045.2021.1929182
Ma, J., Liu, D., Qin, S., Jia, G., Zhang, J., & Xu, Z. (2023). An Asymmetric Feature Enhancement Network for Multiple Object Tracking of Unmanned Aerial Vehicle. Remote Sensing, 16(1), 70. DOI: https://doi.org/10.3390/rs16010070
Meimetis, D., Daramouskas, I., Perikos, I., & Hatzilygeroudis, I. (2023). Real-time multiple object tracking using deep learning methods. Neural Computing and Applications, 35(1), 89–118. DOI: https://doi.org/10.1007/s00521-021-06391-y
Pemila, M., Pongiannan, R., Narayanamoorthi, R., Sweelem, E. A., Hendawi, E., & El-Sebah, M. I. A. (2024). Real-time classification of vehicles using machine learning algorithm on the extensive dataset. IEEE Access, 12, 98338–98351. DOI: https://doi.org/10.1109/ACCESS.2024.3417436
Rishika, A. L., Aishwarya, C., Sahithi, A., & Premchender, M. (2023). Real-time vehicle detection and tracking using yolo-based deep sort model: A computer vision application for traffic surveillance. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 14(1), 255–264.
Sekhar, G. B., Srilatha, M., Srinivasulu, J., & Chowdary, M. B. (2022). Vehicle Tracking and Speed Estimation Using Deep Sort. 146–151. DOI: https://doi.org/10.1109/ICPS55917.2022.00035
Sommer, L. W., Schuchert, T., & Beyerer, J. (2017). Fast deep vehicle detection in aerial images. 311–319. DOI: https://doi.org/10.1109/WACV.2017.41
Srivastava, A., & Prakash, J. (2023). Techniques, answers, and real-world UAV implementations for precision farming. Wireless Personal Communications, 131(4), 2715–2746. DOI: https://doi.org/10.1007/s11277-023-10577-z
Teo, T.-A., Chang, M.-J., & Wen, T.-H. (2024). Automatic vehicle trajectory behavior classification based on unmanned aerial vehicle-derived trajectories using machine learning techniques. ISPRS International Journal of Geo-Information, 13(8), 264. DOI: https://doi.org/10.3390/ijgi13080264
Wang, J., Simeonova, S., & Shahbazi, M. (2019). Orientation-and scale-invariant multi-vehicle detection and tracking from unmanned aerial videos. Remote Sensing, 11(18), 2155. DOI: https://doi.org/10.3390/rs11182155
Wojke, N., Bewley, A., & Paulus, D. (2017a). Simple online and realtime tracking with a deep association metric. 3645–3649.
Wojke, N., Bewley, A., & Paulus, D. (2017b). Simple online and realtime tracking with a deep association metric. 3645–3649.
Wojke, N., Bewley, A., & Paulus, D. (2017c). Simple online and realtime tracking with a deep association metric. 3645–3649. DOI: https://doi.org/10.1109/ICIP.2017.8296962
Wu, X., Li, W., Hong, D., Tao, R., & Du, Q. (2021). Deep learning for unmanned aerial vehicle-based object detection and tracking: A survey. IEEE Geoscience and Remote Sensing Magazine, 10(1), 91–124. DOI: https://doi.org/10.1109/MGRS.2021.3115137
Wu, Z., Suresh, K., Narayanan, P., Xu, H., Kwon, H., & Wang, Z. (2019). Delving into robust object detection from unmanned aerial vehicles: A deep nuisance disentanglement approach. 1201–1210. DOI: https://doi.org/10.1109/ICCV.2019.00129
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018). DOTA: A large-scale dataset for object detection in aerial images. 3974–3983. DOI: https://doi.org/10.1109/CVPR.2018.00418
Xu, Z., Zhao, H., Liu, P., Wang, L., Zhang, G., & Chai, Y. (2025). SRTSOD-YOLO: stronger real-time small object detection algorithm based on improved YOLO11 for UAV imageries. Remote Sensing, 17(20), 3414. DOI: https://doi.org/10.3390/rs17203414
Yang, W.-J., Liow, W.-J., Chen, S.-F., Yang, J.-F., Chung, P.-C., & Mao, S. (2022a). Improved vehicle detection systems with double-layer LSTM modules. EURASIP Journal on Advances in Signal Processing, 2022(1), 7.
Yang, W.-J., Liow, W.-J., Chen, S.-F., Yang, J.-F., Chung, P.-C., & Mao, S. (2022b). Improved vehicle detection systems with double-layer LSTM modules. EURASIP Journal on Advances in Signal Processing, 2022(1), 7. DOI: https://doi.org/10.1186/s13634-022-00839-6
You, L., Chen, Y., Xiao, C., Sun, C., & Li, R. (2024). Multi-object vehicle detection and tracking algorithm based on improved YOLOv8 and ByteTrack. Electronics, 13(15), 3033. DOI: https://doi.org/10.3390/electronics13153033
Yu, H., Li, G., Zhang, W., Huang, Q., Du, D., Tian, Q., & Sebe, N. (2020). The unmanned aerial vehicle benchmark: Object detection, tracking and baseline. International Journal of Computer Vision, 128(5), 1141–1159. DOI: https://doi.org/10.1007/s11263-019-01266-1
Yu, Y., Si, X., Hu, C., & Zhang, J. (2019). A review of recurrent neural networks: LSTM cells and network architectures. Neural Computation, 31(7), 1235–1270. DOI: https://doi.org/10.1162/neco_a_01199
Yusuf, M. O., Hanzla, M., Al Mudawi, N., Sadiq, T., Alabdullah, B., Rahman, H., & Algarni, A. (2024). Target detection and classification via EfficientDet and CNN over unmanned aerial vehicles. Frontiers in Neurorobotics, 18, 1448538. DOI: https://doi.org/10.3389/fnbot.2024.1448538
Zhao, X., Xia, Y., Zhang, W., Zheng, C., & Zhang, Z. (2023). YOLO-ViT-based method for unmanned aerial vehicle infrared vehicle target detection. Remote Sensing, 15(15), 3778. DOI: https://doi.org/10.3390/rs15153778
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2020). Distance-IoU loss: Faster and better learning for bounding box regression. 34(07), 12993–13000. DOI: https://doi.org/10.1609/aaai.v34i07.6999
Zhu, P., Wen, L., Bian, X., Ling, H., & Hu, Q. (2018). Vision meets drones: A challenge. arXiv Preprint arXiv:1804.07437.
Zhu, P., Wen, L., Du, D., Bian, X., Ling, H., Hu, Q., Nie, Q., Cheng, H., Liu, C., & Liu, X. (2018). Visdrone-det2018: The vision meets drone object detection in image challenge results. 0–0.
Zhu, Y., Wang, Y., An, Y., Yang, H., & Pan, Y. (2024). Real-time vehicle detection and urban traffic behavior analysis based on uav traffic videos on mobile devices. arXiv Preprint arXiv:2402.16246. DOI: https://doi.org/10.2139/ssrn.4976574
Zou, C., Jeon, W.-S., & Rhee, S.-Y. (2024). Research on the multiple small target detection methodology in remote sensing. Sensors, 24(10), 3211. DOI: https://doi.org/10.3390/s24103211
Downloads
Published
Issue
Section
License
Copyright (c) 2026 AFRICAN JOURNAL OF APPLIED RESEARCH
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
By submitting and publishing your articles in the African Journal of Applied Research, you agree to transfer the copyright of the Article from the authors to the Journal ( African Journal of Applied Research).
