Truth be told: a multimodal ensemble approach for enhanced fake news detection in textual and visual media

Fake news detection (FND) has emerged as a pivotal domain within the natural language processing field. The primary goal of FND is to discern and evaluate the veracity of pivotal claims in news articles, thereby determining the credibility of the presented news. Employing FND acts as a safeguard against the adverse ramifications of propagating misrepresented information, potentially wreaking socio-political and national havoc on targeted societal segments. Furthermore, with the accelerated spread of false information across social media platforms, spanning both textual and visual formats, an imperative arises to enhance the efficacy and accuracy of FND methodologies tailored to these diverse data sources. This paper presents (Verifiable Fake News Detection), a framework tailored to detect fake news in articles that incorporate both textual and visual content. employs a multi-modal ensemble approach, an integration technique that combines various models and data sources for a holistic analysis, to aggregate feature vectors from different media sources within a news article and effectively classify its credibility. Specifically, uses the SBERT and DeBERT models, both widely-used and pre-trained language representation models, to convert the textual news information into word vector representations. Similarly, uses the ResNet model, a deep convolutional neural network known for its efficacy in image feature extraction and recognition, to derive a feature vector from the image(s) in the new article. then combines these generated vectors using a weighted fusion strategy to obtain a unified feature representation capturing nuances from both textual and visual data. For classification of the resulting vector(s), a deep neural network classifier with dropout regularization is deployed to determine the authenticity of the news article. In analyzing the textual and the image data, a reduction in the number of parameters is performed to optimize the performance of the detection model, all while maintaining high accuracy. Evaluation results performed on the Twitter MediaEval Dataset and the Weibo Corpus indicate that achieves accuracies of 87.40% and 88.34% on the Twitter and Weibo datasets, respectively, a better performance compared to other recent FND studies.