Scalability and Efficiency of Deep Learning Models on High-Performance Computing Clusters: Bibliometric Analysis
DOI:
https://doi.org/10.26437/ajar.v10i2.807Keywords:
Anomaly detection. autoencoder. bibliometric. deep learning. efficiencyAbstract
Purpose: This systematic review identifies the main advancements, core papers, voids, and outlooks regarding utilising high-performance computing (HPC) clusters for large-scale deep learning models. Moreover, it aims to assess research trends, significant approaches, and methods to improve the effectiveness and adaptability of these models.
Design/Methodology/Approach: This paper involves a systematic literature review and bibliometric analysis of published articles from 2018-2023. To limit the results, some specific keywords have been introduced: Web of Science, deep learning efficiency, scalability, and HPC clusters. Studies were screened according to the PRISMA flowchart, covering 364 articles, 19 of which were included in the systematic review based on further criteria.
Findings: The bibliometric analysis revealed that the most globally cited articles were from IEEE Transactions on Emerging Topics in Computing and Joule. However, the most relevant sources identified were the Journal of Supercomputing, Concurrency and Computation: Practice and Experience, and IEEE Access. Researchers from the USA, China, Korea, and the UK authored the most significant contributions.
Research Limitation: The study examined only works from the Web of Science database from 2018 to 2023.
Practical Implication: The proposed research results contribute crucial information to enhance the effectiveness of deep predictive models in large-scale HPC environments, which are essential for enterprises adopting artificial intelligence (AI) and machine learning (ML) methodologies in colossal data analysis applications.
Social Implication: Propelling deep learning models with the help of HPC clusters can create more vital AI solutions that can respond to society's needs.
Originality/ Value: The novelty of the research stems from the bibliometric assessment and the question of which sources and authors in this field are the most important.
References
Ahmad, A., Paul, A., Din, S., Rathore, M. M., Choi, G. S., & Jeon, G. (2018). Multilevel Data Processing Using Parallel Algorithms for Analyzing Big Data in High-Performance Computing. INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING, 46(3), 508–527. https://doi.org/10.1007/s10766-017-0498-x
Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975.
Batistič, S., & Van Der Laken, P. (2019). History, Evolution and Future of Big Data and Analytics: A Bibliometric Analysis of Its Relationship to Performance in Organizations. British Journal of Management, 30(2), 229–251. https://doi.org/10.1111/1467-8551.12340
Benchara, F. Z., & Youssfi, M. (2021). A new scalable distributed k-means algorithm based on Cloud micro-services for High-performance computing. PARALLEL COMPUTING, 101. https://doi.org/10.1016/j.parco.2020.102736
Borghesi, A., Bartolini, A., Lombardi, M., Milano, M., & Benini, L. (2019). A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems. ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 85, 634–644. https://doi.org/10.1016/j.engappai.2019.07.008
Chen, Q., Rankine, A., Peng, Y., Aghaarabi, E., & Lu, Z. (2021). Benchmarking Effectiveness and Efficiency of Deep Learning Models for Semantic Textual Similarity in the Clinical Domain: Validation Study. JMIR Medical Informatics, 9(12), e27386–e27386. https://doi.org/10.2196/27386
Chen, S., He, Z., Han, X., He, X., Li, R., Zhu, H., Zhao, D., Dai, C., Zhang, Y., Lu, Z., Chi, X., & Niu, B. (2019). How Big Data and High-performance Computing Drive Brain Science. GENOMICS PROTEOMICS & BIOINFORMATICS, 17(4), 381–392. https://doi.org/10.1016/j.gpb.2019.09.003
Cliff, A., Romero, J., Kainer, D., Walker, A., Furches, A., & Jacobson, D. (2019). A High-Performance Computing Implementation of Iterative Random Forest for the Creation of Predictive Expression Networks. GENES, 10(12). https://doi.org/10.3390/genes10120996
Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology, 62(7), 1382–1402. https://doi.org/10.1002/asi.21525
Correa-Baena, J.-P., Hippalgaonkar, K., van Duren, J., Jaffer, S., Chandrasekhar, V. R., Stevanovic, V., Wadia, C., Guha, S., & Buonassisi, T. (2018). Accelerating Materials Development via Automation, Machine Learning, and High-Performance Computing. JOULE, 2(8), 1410–1420. https://doi.org/10.1016/j.joule.2018.05.009
Dakudjie, J. K., Braye, A., & Otchere, A. A. (2018). An Investigation into System Components That Supports Biometric Frame in Ghana. African Journal of Applied Research, 4(2), 171-179.
Dutta, M., & Gupta, D. (2023). Bibliometric Analysis on Herbaceous Plants using Smart Precision Farming. 2023 IEEE Renewable Energy and Sustainable E-Mobility Conference (RESEM), 1–6. https://doi.org/10.1109/RESEM57584.2023.10236141
Fan, C., Hu, K., Yuan, Y., & Li, Y. (2023). A data-driven analysis of global research trends in medical image: A survey. Neurocomputing, 518, 308–320. https://doi.org/10.1016/j.neucom.2022.10.047
Graziani, M., Eggel, I., Deligand, F., Bobak, M., Andrearczyk, V., & Mueller, H. (2020). BREAST HISTOPATHOLOGY WITH HIGH-PERFORMANCE COMPUTING AND DEEP LEARNING. COMPUTING AND INFORMATICS, 39(4), 780–807. https://doi.org/10.31577/cai_2020_4_780
Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews, 18(2), e1230. https://doi.org/10.1002/cl2.1230
Kounta, C. A. K. A., Arnaud, L., Kamsu-Foguem, B., & Tangara, F. (2022). Review of AI-based methods for chatter detection in machining based on bibliometric analysis. The International Journal of Advanced Manufacturing Technology, 122(5–6), 2161–2186. https://doi.org/10.1007/s00170-022-10059-9
Lages, C. R., Perez-Vega, R., Kadić-Maglajlić, S., & Borghei-Razavi, N. (2023). A systematic review and bibliometric analysis of the dark side of customer behavior: An integrative customer incivility framework. Journal of Business Research, 161, 113779. https://doi.org/10.1016/j.jbusres.2023.113779
Lee, K. L. K., & Kumar, N. (2023). Artificial Intelligence for Scientific Discovery at High-Performance Computing Scales. COMPUTER, 56(4), 116–122. https://doi.org/10.1109/MC.2023.3241692
Lei, Y., Yao, H., Jiang, B., Tian, T., & Xing, P. (2022). Anti-UAV High-Performance Computing Early Warning Neural Network Based on PSO Algorithm. SCIENTIFIC PROGRAMMING, 2022. https://doi.org/10.1155/2022/7150128
Li, J.-M., Wu, T.-J., Wu, Y. J., & Goh, M. (2023). Systematic literature review of human–machine collaboration in organizations using bibliometric analysis. Management Decision. https://doi.org/10.1108/MD-09-2022-1183
Lopez-Martinez, M., Diaz-Florez, G., Villagrana-Barraza, S., Solis-Sanchez, L. O., Guerrero-Osuna, H. A., Soto-Zarazua, G. M., & Olvera-Olvera, C. A. (2023). A High-Performance Computing Cluster for Distributed Deep Learning: A Practical Case of Weed Classification Using Convolutional Neural Network Models. APPLIED SCIENCES-BASEL, 13(10). https://doi.org/10.3390/app13106007
Pyzer-Knapp, E. O., Pitera, J. W., Staar, P. W. J., Takeda, S., Laino, T., Sanders, D. P., Sexton, J., Smith, J. R., & Curioni, A. (2022). Accelerating materials discovery using artificial intelligence, high performance computing and robotics. NPJ COMPUTATIONAL MATERIALS, 8(1). https://doi.org/10.1038/s41524-022-00765-z
Sidharta, S., Warnars, H. L. H. S., Gaol, F. L., & Soewito, B. (2022). Building Damage Assessment Using Deep Learning: Bibliometric Analysis. 2022 IEEE 7th International Conference on Information Technology and Digital Applications (ICITDA), 1–6. https://doi.org/10.1109/ICITDA55840.2022.9971269
Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3
Wang, G.-G., Cai, X., Cui, Z., Min, G., & Chen, J. (2017). High Performance Computing for Cyber Physical Social Systems by Using Evolutionary Multi-Objective Optimization Algorithm. IEEE Transactions on Emerging Topics in Computing, 1–1. https://doi.org/10.1109/TETC.2017.2703784
Xie, H., Zhang, Y., Wu, Z., & Lv, T. (2020). A Bibliometric Analysis on Land Degradation: Current Status, Development, and Future Directions. Land, 9(1), 28.
Zhang, F., Petersen, M., Johnson, L., Hall, J., & O’Bryant, S. E. (2021). Accelerating Hyperparameter Tuning in Machine Learning for Alzheimer’s Disease With High Performance Computing. Frontiers in Artificial Intelligence, 4, 798962–798962. https://doi.org/10.3389/frai.2021.798962
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2024 AFRICAN JOURNAL OF APPLIED RESEARCH
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
By submitting and publishing your articles in the African Journal of Applied Research, you agree to transfer the copyright of the Article from the authors to the Journal ( African Journal of Applied Research).
