Skip to main content

Improving Bert-Based Model for Medical Text Classification with an Optimization Algorithm

Author name : Karim Gasmi
Publication Date : 2022-09-21
Journal Name : Conference on Computational Collective Intelligence Technologies and Applications

Abstract

In the field of Natural Language Processing (NLP), automatic text classification is a classic topic that involves classifying textual material into predetermined categories based on its content. These models have been effectively applied to data containing a large number of dimensional features, some of which are inherently sparse. Machine learning and other statistical approaches, such as those used in medical text categorization, appear to be extremely successful for these tasks. However, much human work is still required to classify a large collection of training data. Recent research has shown the usefulness of pre-trained language models such as Bidirectional Encoder Representations from Transformers (BERT), all of which have demonstrated their ability to reduce the amount of work required for feature engineering. However, directly using the pre-trained BERT model in the classification task does not result in a statistically significant increase in performance. To improve the result of the BERT model, we propose an optimal deep learning model based on a BERT model and hyperparameter selection. The model consists of three steps: (1) processing medical text; (2) extracting medical text features using a BERT architecture; and (3) selecting hyperparameters for the Deep Learning model based on a Particle Swarm Optimization (PSO) algorithm. Finally, our approach uses a k-Nearest Neighbors algorithm (KNN) model to predict the matching response. Experiments conducted on the Hallmarks dataset have shown that the proposed method significantly increases the accuracy of the results.

Keywords

-

Publication Link

https://doi.org/10.1007/978-3-031-16210-7_8

Block_researches_list_suggestions

Suggestions to read

HIDS-IoMT: A Deep Learning-Based Intelligent Intrusion Detection System for the Internet of Medical Things
Ahlem . Harchy Ep Berguiga
Generalized first approximation Matsumoto metric
AMR SOLIMAN MAHMOUD HASSAN
Structure–Performance Relationship of Novel Azo-Salicylaldehyde Disperse Dyes: Dyeing Optimization and Theoretical Insights
EBTSAM KHALEFAH H ALENEZY
“Synthesis and Characterization of SnO₂/α-Fe₂O₃, In₂O₃/α-Fe₂O₃, and ZnO/α-Fe₂O₃ Thin Films: Photocatalytic and Antibacterial Applications”
Asma Arfaoui
Contact