Development of an equation-free reduced-order model based on different feature extraction patterns on the two-dimensional steady-state heat transfer dataset

Document Type : Original Article

Authors

1 Department of Computer Engineering and Information Technology, University of Qom, Qom, Iran.

2 Department of Mechanical Engineering, University of Qom, Qom, Iran.

Abstract

Due to the high computational cost and time of the direct or numerical methods of the partial differential equations governing phenomena, this paper proposes an equation-free model based on deep learning algorithms using dimension reduction methods. Two methods of the principal component analysis (linear) and the autoencoder (nonlinear) were used to simulate the steady-state heat transfer phenomenon using the two-dimensional steady-state heat transfer dataset of size 64×64 and 128×128. These methods were implemented using the tools and libraries available in the Python environment. According to the results, autoencoder has higher accuracy in reduction of order to very low dimensions, while principal component analysis has a higher accuracy in reduction of order to very high dimensions. Additionally, the outputs of the proposed reduced-order model were compared with simulation results obtained from a model based on convolutional neural network with a large number of layers and filters. The simulation results of the temperature distribution of the steady-state heat transfer phenomenon in terms of mean square error (MSE) using principal component analysis, autoencoder, and convolutional neural network-based models at dimensions of 64×64 were equal to 1.617×10-4, 2.528×10-6, and 0.015 °C per-pixel, respectively. At dimensions of 128×128, the results were equal to 2.046×10-4, 7.253×10-6, and 0.0058 °C per-pixel, respectively. Therefore, the proposed reduced-order models, especially the autoencoder-based model, have significantly higher accuracy than the convolutional neural network-based model.

Keywords

References

[1] Krizhevsky A., Sutskever I., and Hinton G.E., "in Advances in neural information processing systems", pp. 1097–1105, 2012.
[2] Johnson M., Schuster M., Le Q., Krikun M., Wu Y., Chen Z., Thorat N., Viégas F., Wattenberg M., Corrado G., Hughes M., and Dean J., "Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation", Transactions of the Association for Computational Linguistics, 5:339-351, 2017.
[3] Goodfellow I., Bengio Y., and Courville A., Deep learning, Cambridge, MIT Press, 2016.
[4] Zhao, H., "Advances and Prospects in Machine Vision: a Critical Review Based on CiteSpace", Journal of the Frontiers of Society, Science and Technology, 2(16): 86-92, 2020.
[5] Joshi K., and Patil B., Evaluation of Surface Roughness by Machine Vision Using Neural Networks Approach, Recent Advances in Mechanical Infrastructure. Lecture Notes in Intelligent Transportation and Infrastructure, Springer, Singapore, 2020.
[6] Torfi A., Shirvani R.A., Keneshloo Y., Tavaf N., and Edward A. F., "Natural Language Processing Advancements By Deep Learning: A Survey", Journal of Computer Science, 2020.
[7] Otter D. W., Medina J. R., and Kalita J. K., "A Survey of the Usages of Deep Learning for Natural Language Processing" , IEEE Transactions on Neural Networks and Learning Systems, 32(2): 604-624, Feb 2021.
[8] Vedantam V. K., "The Survey: Advances in Natural Language Processing using Deep Learning", Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(4), 2021.
[9] Leskovec J., Rajaraman A., and Ullman, J., Mining of Massive Datasets, 2nd edition, Stanford University, California, Cambridge University Press, 2014.
[10] Samet H., Foundations of Multidimensional and Metric Data Structures, Morgan Kaufmann Publishers, 2006.
[11] Andrienko G., Andrienko N., Drucker S., Fekete J.-D., and Fisher D., "Big Data Visualization and Analytics: Future Research Challenges and Emerging Applications", 3rd International Workshop on Big Data Visual Exploration and Analytics, Copenhagen, Denmark, Mar 2020.
[12] Brunsdon C. and Comber A., "Big Issues for Big Data: challenges for critical spatial data analytics", Journal of Computer Science, 2020.
[13] Erum M. and Anees T., "Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature Review", Journal of IEEE Access, 8, 2020. 
[14] Yang X.-S., Lee S., Lee, S., and Theera-Umpon, N., "Information Analysis of High-Dimensional Data and Applications", mathematical problems in engineering journal, 2015.
[15] Guillemard M., Iske A., and Krause-Solberg S., "Dimensionality Reduction Methodsin Independent Subspace Analysisfor Signal Detection", Sampling Theory and Applications (SampTA’11), 2011.
[16] Sharma R., Sircar P., and Pachori R. B., "Automated focal EEG signal detection based on third order cumulant function", Journal of Biomedical Signal Processing and Control, 58, 2020.
[17] Fewzee P. and Karray F., "Dimensionality Reduction for Emotional Speech Recognition", International Conference on Privacy, Security, Risk and Trust and International Confernece on Social Computing, pp. 532-537, Amsterdam, 2012.
[18] Kumar K.A. and Mazher Iqbal J. L., "Handling high dimensional features by ensemble learning for emotion identification from speech signal", International Journal of Speech Technology, 2021. 
[19] Smietanka L. and Maka T., "Audio Feature Space Analysis for Emotion Recognition from Spoken Sentences", Journal of Archives of Acoustics, 46(2): 271-277, 2021. 
[20] Fernandes Sinead V. and Ullah M. S., "Use of Machine Learning for Deception Detection From Spectral and Cepstral Features of Speech Signals", Journal of  IEEE Access, 9: 78925-78935, 2021. 
[21] Ratnovsky A., Malayev S., Ratnovsky S., Naftali S., and Rabin N., "EMG-based speech recognition using dimensionality reduction methods", Journal of Ambient Intelligence and Humanized Computing, 2021. 
[22] Agrawal A. and Jain A., "Emotion Recognition of Speech in Hindi Using Dimensionality Reduction and Machine Learning Techniques", International Conference on Information and Communication Technology and Applications, pp. 119-129, 2020.
[23] Mwangi B., Tian S., and Soares J. C, "A review of feature reduction techniques in neuroimaging". Neuroinformatics 12, pp 229–244. 2014.
[24] Fang C., Li, C., Forouzannezhad P., Cabrerizo M. E., Curiel R., Loewenstein D., Duara R., and Adjouadi M., "Gaussian discriminative component analysis for early detection of Alzheimer’s disease: A supervised dimensionality reduction algorithm", Journal of Neuroscience Methods, 344, 2020. 
[25] Yew A. Y. L. and Rahim M. S. M., "Dimensionality Reduction Methods for Alzheimer's Disease Classification", Journal of Computing and Digital Systems, 2021.
[26] Shinde K. and Thakare A., "Significance of Dimensionality Reduction Techniques for Fetal Brain MRI Analysis", Proceedings of the 3rd International Conference on Communication, 2021.
[27] Messina D., Borrelli P., Russo P., Salvatore M., and Aiello M., "Voxel-wise feature selection method for CNN binary classification of neuroimaging data", Journal of  Frontiers in Neuroscience, 15, 2021.
[28] Calesella F., Testolin A., De Grazia M., and Zorzi M., "A comparison of feature extraction methods for prediction of neuropsychological scores from functional connectivity data of stroke patients", Journal of Brain Informatics, 8, 2021.
[29] Miah A. S. M., Rashid M., Rahman R., Hossain T., Sujon S., Nawal N., Hasan M., and Shin J., "Alzheimer’s Disease Detection Using CNN Based on Effective Dimensionality Reduction Approach", International Conference on Intelligent Computing and Optimization, pp. 801-811, 2021.
[30] Filipovic V., "Optimization, classification and dimensionality reduction in biomedicine and bioinformatics", Biologia Serbica 39(1): 83 – 98, 2017.
[31] Van Der Maaten L., Postma E., and Van Den Herik J., "Dimensionality Reduction: A Comparative Review", J Mach Learn Res. 10:66–71, 2009.
[32] Rico-Sulayes A., "Reducing Vector Space Dimensionality in Automatic Classification for Authorship Attribution", Revista Ingenieria Electronica, Automatica y Comunicaciones. 38(3):26–35, 2017.
[33] مویدی م.ک.، صباغ زادگان ف.، «توسعه مدل رتبه کاسته برای معادله نفوذ جابجایی بر مبنای روش تجزیه متعامد بهینه»، هجدهمین کنفرانس دینامیک شاره‌ها، مشهد، 1398.
[34] Pudil P. and Novovicova J., "Novel Methods for Feature Subset Selection with Respect to Problem Knowledge", In Liu, Huan; Motoda, Hiroshi (eds.), Feature Extraction, Construction and Selection, pp. 101, 1998.
[35] Bolon-Canedo V., Sanchez-Marono N., and Alonso-Betanzos A., "Feature Selection for High-Dimentional Data", pp. 65, Springer, 2015.
[36] نجفی ا.، «فشرده سازی دیتا بیس چهره با تجزیه مؤلفه‌های اصلی غیرخطی با کمک تقسیم بندی»، پایان نامه کارشناسی ارشد مهندسی برق، دانشکده مهندسی برق و کامپیوتر دانشگاه شهید بهشتی، تهران، ص12، 1394.
[37] Jolliffe I. T., Principal Component Analysis, Series: Springer Series in Statistics, 2nd ed., pp.28, Springer, 2002. 
[38] همایون‌پور م.م.، شجاع مودب ح.، «ارزیابی و مقایسه چهار روش کاهش بعد ویژگی‌‌ها برای سیستم تشخیص نفوذ مبتنی بر ماشین بردار»، چهارمین کنفرانس انجمن رمز ایران، 1386.
[39] Kramer M. A., "Nonlinear principal component analysis using autoassociative neural networks". AIChE Journal, 37 (2):233–243, 1991.
[40] Wang J., He H., and Prokhorov D. V., "A Folded Neural Network Autoencoder for Dimensionality Reduction", In proceeding of the International Neural Network Society Winter Conference (INNS-WC 2012) , pp.120 – 127, procedia computer science 13, 2012. 
[41] Nugroho H., Susanty M., Irawan A., Koyimatu M., and Yunita A., "Fully convolutional variational autoencoder to features extraction of the fire detection system", Journal of Computer Science and Information, 13/1, pp.9-15, 2020.
[42] Jolliffe I. T. and Cadima J., "Principal component analysis: a review and recent developments", Phil. Trans. R. Soc. A, 374(2065), 2016.
Soft Computing Journal
Volume 10, Issue 1 - Serial Number 19
September 2021
Pages 16-31

Files

History

  • Receive Date: 22 May 2021
  • Revise Date: 13 October 2021
  • Accept Date: 22 November 2021

Share

How to cite

Statistics