sapiСистемный анализ и прикладная информатика«System analysis and applied information science»2309-49232414-0481Belarusian National Technical University10.21122/2309-4923-2023-1-35-41sapi-605Research ArticleОбработка информации и принятие решенийData processing and decision–makingМетоды и средства обработки звуков кашля на базе нейронной сетиMethodics and tools of cough sound processing on basic of neural netВишняковВ. А.VishniakouU. A.

Вишняков Владимир Анатольевич – профессор БГУИР, каф. ИКТ.

Минск

Vishniakou Uladzimir - doctor of technical science, professor of ICT department of Belarusian State University of Informatics and Radioelectronics. 

Minsk, Republic of Belarus

vish2002@list.ruСайяБахааShayaBahaa

Сайя Бахаа – магистр технических наук, аспирант кафедры ИКТ БГУИР.

 Минск

Shaya Bahaa - master of technical science, PhD-student of ICT department of Belarusian State University of Informatics and Radioelectronics.

Minsk, Republic of Belarus

Белорусский государственный университет информатики и радиоэлектроникиБеларусьBelarusian state University of Informatics and RadioelectronicsBelarus202301082023013541Copyright © Вишняков В.А., Сайя Б., 20232023Вишняков В.А., Сайя Б.Vishniakou U.A., Shaya B.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://sapi.bntu.by/jour/article/view/605

Целью статьи является анализ методики и средств обработки звуков кашля для выявления легочных заболевания, а также описание разработанной системы для классификации и обнаружения звуков кашля на базе глубинной нейронной сети. Рассмотрены четыре типа машинного обучения, использование сверточной нейронной сети (СНС). Приведены гипермаркеты СНС. Обсуждены разновидности машинного обучения на базе СНС. Выполнен анализ работ по методике и средствам обработки звуков кашля на базе СНС с приведением используемых средств и точности распознавания. Обсуждены детали машинного обучения с использованием набора данных классификации звуков (environmental sound classification 50 – ESC-50). Для распознования кашля COVID-19 проанализирован классификатор, используя CNN в качестве модели машинного обучения. Предлагаемая система СНН предназначена для классификации и обнаружения звуков кашля на базе ESC-50. После выбора набора данных классификации звуков описаны четыре этапа: извлечение признаков из аудиофайлов, маркировки, обучение, тестирование. ESC-50, использованный для исследования, был загружен с веб-сайта Kaggle. Для реализации проекта были использованы библиотеки и модули Python, которые связаны с глубоким обучением и наукой о данных: NumPy, Librosa, Matplotlib, Hickle, Sci-Kit Learn, Keras. Реализованная сеть использовала алгоритм стохастического градиента. Несколько добровольцев записали свои голоса во время кашля с помощью своих смартфонов, было гарантировано, что они будут записывать свои голоса в общественных местах, чтобы внести шум в звуки, в дополнение к некоторым аудиофайлам, которые были загружены онлайн. Результаты показали среднюю точность 85,37 %, точность 78,8 % и рекорд отзыва 91,9 %.

The purpose of the article is to analyze the methods and means of processing cough sounds to detect lung diseases, as well as to describe the developed system for classifying and detecting cough sounds based on a deep neural network. Four types of machine learning and the use of convolutional neural network (CNN) are considered. Hypermarkets of CNN are given. Varieties of machine learning based on the CNN are discussed. The analysis of works on the methodology and means of processing cough sounds based on the CNN with the reduction of the means used and the accuracy of recognition is carried out. Details of machine learning using the environmental sound classification 50 (ESC-50) dataset are discussed. To recognize COVID-19 cough, a classifier was analyzed using CNN as a machine learning model. The proposed CNN system is designed to classify and detect cough sounds based on ESC-50. After selecting a set of sound classification data, four stages are described: extraction of features from audio files, labeling, training, testing. The ESC-50 used for the study was downloaded from the Kaggle website. Python libraries and modules related to deep learning and data science were used to implement the project: NumPy, Librosa, Matplotlib, Hickle, Sci-Kit Learn, Keras. The implemented network used a stochastic gradient algorithm. Several volunteers recorded their voices while coughing using their smartphones and it was assured to record their voices in a public environment to introduce noise to the sounds, in addition to some audio files that were downloaded online. The results showed an average accuracy of 85.37 %, precision of 78.8 % and a recall record of 91.9 %.

машинное обучениенейронные сетиобработка звуков кашлясистема классификации кашляmachine learningneural networkcough sound processingcough classification systemReferencesRadhakrishnan P. Towards Data Science. 9 Aug 2017. [Electronic resource]. Avialable : https://towardsdatascience.com/what-are-hyperparameters-and-how-to-tune-the-hyperparameters-in-a-deep-neural-network-d0604917584a. ‒ Data of accsses: Aug 2022.Radhakrishnan P. Towards Data Science. 9 Aug 2017. [Electronic resource]. Avialable : https://towardsdatascience.com/what-are-hyperparameters-and-how-to-tune-the-hyperparameters-in-a-deep-neural-network-d0604917584a. Date of access: Aug 2022.LeCun J., Boser B., Denker J., Henderson D., Howard R., Hubbard W., Jackel L. Backpropagation Applied to Handwritten Zip Code. Neural Computation, vol. 1, No. 4, 1989. ‒ Pp. 541-551.LeCun J., Boser B., Denker J., Henderson D., Howard R., Hubbard W., Jackel L. Backpropagation Applied to Handwritten Zip Code. Neural Computation, 1989. ‒ Vol. 1, No. 4. ‒ Pp. 541-551.Dhillon A., Verma G.K. Convolutional neural network: A review of models, methodologies and applications to object detection. Progress in Artificial Intelligence, vol. 9, No. 2, 2019. ‒ Pp. 85-112.Dhillon A., Verma G.K. Convolutional neural network: A review of models, methodologies and applications to object detection. Progress in Artificial Intelligence, 2019. ‒ Vol. 9, No. 2. ‒ Pp. 85-112.Kriszhevsky G.K., Sutskever G.K., Hinton G.E. ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural, vol. 60, No. 6, 2012. ‒ Pp. 1097-1105.Kriszhevsky G.K., Sutskever G.K., Hinton G.E. ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural, vol. 60, No. 6, 2012. – Pp. 1097-1105.Chu, S. Unstructured Audio Classification for Environment Recognition // Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 2008. – Pp. 1845-1846.Chu, S. Unstructured Audio Classification for Environment Recognition.// Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 2008. – Pp. 1845-1846.Banchhor K.S., Khan A. Musical Instrument Recognition using Zero Crossing Rate and Short-time Energy. International Journal of Applied Information Systems. vol. 1, N 3, 2012. ‒ Pp. 85-97.Banchhor K.S., Khan A. Musical Instrument Recognition using Zero Crossing Rate and Short-time Energy. International Journal of Applied Information Systems. vol. 1, No 3, 2012. – Pp. 85-97.Feroze K., Maud A.R. Sound Event Detection in Real Life Audio using Perceptual Linear Predictive Feature with Neural Network // Proceeding of 15th International Bhurban Conferance on Applied Sceinces & Technology, Islamabad, Pakistan, 2018.Feroze K., Maud A.R. Sound Event Detection in Real Life Audio using Perceptual Linear Predictive Feature with Neural Network // Proceeding of 15th International Bhurban Conferance on Applied Sceinces & Technology, Islamabad, Pakistan, 2018.Wang H.-H., Liu J.-M., You M., Li G.-Z. Audio Signals Encoding for Cough Classification Using Convolutional Neural Networks: A Comparative Study // IEEE International Conference on Bioinformatics and Biomedicine, Shanghai, 2015.Wang H.-H., Liu J.-M., You M., Li G.-Z. Audio Signals Encoding for Cough Classification Using Convolutional Neural Networks: A Comparative Study // IEEE International Conference on Bioinformatics and Biomedicine, Shanghai, 2015.Chowdhury A., Ross A. Fusing MFCC and LPC Features using 1D Triplet CNN for Speaker Recognition in Severely Degraded Audio Signals. IEEE Transactions on Information Forensics and Security, vol. 15, 2019. ‒ Pp. 1616-629.Chowdhury A., Ross A. Fusing MFCC and LPC Features using 1D Triplet CNN for Speaker Recognition in Severely Degraded Audio Signals. IEEE Transactions on Information Forensics and Security, vol. 15, 2019. – Pp. 1616-1629.Piczak K.J. Environmental sound classification with convolutional neural networks // IEEE International workshop on machine learning for signal processing, Boston, 2015.Piczak K.J. Environmental sound classification with convolutional neural networks // IEEE International workshop on machine learning for signal processing, Boston, 2015.Bales C., Nabeel M., John N.C., Masood U., Qureshi H., Farooq H. Can Machine Learning Be Used to Recognize and Diagnose Coughs? // The 8th IEEE International Conference on E-Health and Bioengineering, Romania, 2020.Bales C., Nabeel M., John N.C., Masood U., Qureshi H., Farooq H. Can Machine Learning Be Used to Recognize and Diagnose Coughs? // The 8th IEEE International Conference on E-Health and Bioengineering, Romania, 2020.Amoh A., Odame K. Deep Neural Networks for Identifying Cough Sounds. IEEE Transactions on Biomedical Circuits and Systems, vol. 10, No. 5, 2016. ‒ Pp. 1003-1011.Amoh A., Odame K. Deep Neural Networks for Identifying Cough Sounds. IEEE Transactions on Biomedical Circuits and Systems, vol. 10, No. 5, 2016. – Pp. 1003-1011.Kosasih K., Abeyratne U., Swarnkar V., Triasih R. Wavelet Augmented Cough Analysis for Rapid Childhood Pneumonia Diagnosis. IEEE Transactions on biomedical engineering, vol. 62, No. 4, 2015. ‒ Pp. 1185-1194.Kosasih K., Abeyratne U., Swarnkar V., Triasih R. Wavelet Augmented Cough Analysis for Rapid Childhood Pneumonia Diagnosis. IEEE Transactions on biomedical engineering, vol. 62, No. 4, 2015. – Pp. 1185-1194.Barata F., Kipfer K., Weber M., Tinschert P., Fleisch E., Kowatsch T. Towards Device-Agnostic Mobile Cough Detection with Convolutional Neural Networks // IEEE International Conference on Healthcare Informatics, Xi’an, 2019.Barata F., Kipfer K., Weber M., Tinschert P., Fleisch E., Kowatsch T. Towards Device-Agnostic Mobile Cough Detection with Convolutional Neural Networks // IEEE International Conference on Healthcare Informatics, Xi’an, 2019.Khomsay S., Vanijjirattikhan R., Suwatthikul J. Cough detection using PCA and Deep Learning // Intern. Conf. on Information and Communication Technology Convergence (ICTC), Jeju, 2019.Khomsay S., Vanijjirattikhan R., Suwatthikul J. Cough detection using PCA and Deep Learning // Intern. Conf. on Information and Communication Technology Convergence (ICTC), Jeju, 2019.Chen X., Hu X. Zhai G. Cough Detection Using Selected Informative Features from Audio Signals. Cirnell University, August, 2021.Chen X., Hu X. Zhai G. Cough Detection Using Selected Informative Features from Audio Signals. Cirnell University, August, 2021.Rashid H.-A., Mazumder A.N., Pan U., Niyogi K., Mohsenin T. CoughNet: A Flexible Low Power CNN-LSTM Processor for Cough Sound Detection // IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), Washington DC, 2021.Rashid H.-A., Mazumder A.N., Pan U., Niyogi K., Mohsenin T. CoughNet: A Flexible Low Power CNN-LSTM Processor for Cough Sound Detection // IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), Washington DC, 2021.Bansal V., Pahwa G., Kannan N. Cough Classification for COVID-19 based on audio mfcc features using Convolutional Neural Networks // IEEE International Conference on Computing, Power and Communication Technologies (GUCON), Greater Noida, 2020.Bansal V., Pahwa G., Kannan N. Cough Classification for COVID-19 based on audio mfcc features using Convolutional Neural Networks // IEEE International Conference on Computing, Power and Communication Technologies (GUCON), Greater Noida, 2020.Piczak K.J. ESC: Dataset for Environmental Sound Classification // Proceedings of the 23rd Annual ACM Conference on Multimedia, Brisbane, 2015.Piczak K.J. ESC: Dataset for Environmental Sound Classification // Proceedings of the 23rd Annual ACM Conference on Multimedia, Brisbane, 2015.Abadi M., Barham P., Chen P., Chen Z, Davis A. TensorFlow: A System for Large-Scale Machine Learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16). November, 2016 Savannah, GA, USA.Abadi M., Barham P., Chen P., Chen Z, Davis A. TensorFlow: A System for Large-Scale Machine Learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16). November, 2016 Savannah, GA, USA.

The authors declare that there are no conflicts of interest present.