FakeCLIP: Multimodal Fake Caption Detection with Mixed Languages for Explainable Visualization

Christian Nathaniel Purwanto, Joan Santoso, Po Ruey Lei, Hui Kuo Yang, Wen Chih Peng

研究成果: Conference contribution同行評審

1 引文 斯高帕斯(Scopus)

摘要

Existing fake news research relies on news propagation or news metadata. Waiting for propagation structure to be enough is a waste of time. Hoping for reliable metadata information is also a waste because all data can be forged. The most natural way for human when verifying a news is through the content itself. In social media, most of the circulating news are in minimal content which consist of image and its text caption. We propose FakeCLIP to examine whether a caption truly describes the corresponding image or not. As far as we know, we are the first one to tackle fake news using fake caption approach. We found mixed languages problem where one single text can consist of many different languages mixed together. We provide explainable visualization for intuitive reasoning of which part contains fake information. Moreover, we also consider alignment of what happens in the image that being discussed in the text caption while showing the fake signal over them. Our proposed method performs better than the current state-of-the-art on Twitter datasets by 11.1%.

原文English
主出版物標題Proceedings - 2021 International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021
發行者Institute of Electrical and Electronics Engineers Inc.
頁面1-6
頁數6
ISBN(電子)9781665408257
DOIs
出版狀態Published - 2021
事件26th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021 - Taichung, Taiwan
持續時間: 18 11月 202120 11月 2021

出版系列

名字Proceedings - 2021 International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021

Conference

Conference26th International Conference on Technologies and Applications of Artificial Intelligence, TAAI 2021
國家/地區Taiwan
城市Taichung
期間18/11/2120/11/21

指紋

深入研究「FakeCLIP: Multimodal Fake Caption Detection with Mixed Languages for Explainable Visualization」主題。共同形成了獨特的指紋。

引用此