A Hybrid Transformer-Based Model for Optimizing Fake News Detection

Fake news, colloquially referred to as false news, exerts a profound influence on fundamental
facets of our societal framework. Manual fact verification is a frequently utilized strategy for mitigating
the harmful impacts of false news transmission. However, when assessing the massive volume of freshly
produced material, manual fact verification is insufficient. Furthermore, the quantity of labeled datasets is
limited, people are unreliable annotators, resources are largely in the English language, and they mostly
concentrate on articles related to news. Cutting-edge deep learning algorithms are employed to deal with
this challenge automatically and address these difficulties. Nevertheless, the large number of models and
variability of characteristics utilized in the literature frequently constitute a barrier for researchers attempting
to enhance the effectiveness of models. This paper introduces a model designed to address the issue of false
news, with a focus on analyzing news headlines. The approach is rooted in a hybrid classification model
where our model integrates a BERT architecture, and the outputs are seamlessly linked to bi-directional
deep learning layers, including a bi-LSTM layer and a bi-GRU layer. The model’s training and evaluation
were conducted using the WELFake dataset, comprising four prevalent news databases. A comparative
analysis was carried out using the proposed model and standard classification models. Additionally, standard
machine learning and deep learning models along with vanilla BERT were trained on the same dataset with
comparable constraints to assess the impact of f integrating a bi-directional deep learning layer with BERT.
The results revealed an improvement in accuracy, as our proposed model achieved an accuracy of 98.1% and
an F1 score of 0.982.