Multimodal Deep Learning for Online Meme Classification
Date Issued
2024-12-15
Author(s)
Han, Stephanie
Leal-Arenas, Sebastian
Cavalcante, Charles C
Boukouvalas, Zois
Corizzo, Roberto
Abstract
Memes possess a humorous intent, yet they can
also be used for malicious purposes. Analysing meme data has
the potential to enhance content monitoring, identify emerging
topics, and support content moderation in online platforms.
Memes also represent an interesting use case for multimodal
machine learning, as they combine text and image data. In this
study, we explored the linguistic characteristics and analysed
the convergent themes of five meme classes through common
word extraction. Moreover, we compared the effectiveness of
various machine learning models, i.e., unimodal (text or image)
and multimodal (early fusion, late fusion) in binary and multiclass meme classification tasks. Our results on a large meme
dataset showed that memes heavily adhered to current affairs,
demonstrated by the high frequency of topical words across meme
classes. Regarding model accuracy, early fusion achieved superior
accuracy over late fusion in meme classification. Binary models
outperformed multi-class classification methods. However, fusion
models did not consistently surpass the accuracy of independent
text or image-based models.
also be used for malicious purposes. Analysing meme data has
the potential to enhance content monitoring, identify emerging
topics, and support content moderation in online platforms.
Memes also represent an interesting use case for multimodal
machine learning, as they combine text and image data. In this
study, we explored the linguistic characteristics and analysed
the convergent themes of five meme classes through common
word extraction. Moreover, we compared the effectiveness of
various machine learning models, i.e., unimodal (text or image)
and multimodal (early fusion, late fusion) in binary and multiclass meme classification tasks. Our results on a large meme
dataset showed that memes heavily adhered to current affairs,
demonstrated by the high frequency of topical words across meme
classes. Regarding model accuracy, early fusion achieved superior
accuracy over late fusion in meme classification. Binary models
outperformed multi-class classification methods. However, fusion
models did not consistently surpass the accuracy of independent
text or image-based models.
Subjects
File(s)![Thumbnail Image]()
Loading...
Name
Multimodal_Deep_Learning_of_Online_Memes__Copy_-8.pdf
Size
1.13 MB
Format
Adobe PDF
Checksum
(MD5):94380a129f724cc47efe7c1a83ce93b1