Comparing the performance of Text Classification Models for climate change-related texts

Abstract

This study aims to evaluate and compare the performance of two text classification models specifically tailored for classifying climate change-related texts. The models under investigation are ClimateBert Environmental Claims and ClimateBert Fact Checking, both of which are based on the ClimateBert model and available in the HuggingFace Hub. Our analysis focuses on the impact of fine-tuning these models using specific climate change-related datasets, as well as their performance without fine-tuning. We assess the models using various metrics, including accuracy, precision, recall, and F1 score, and identify the areas where they predominantly make classification errors. Through our findings, we highlight the significance of using these methodologies for the evaluation and comparison of climate change-related text classification models and to appropriately fine-tune the models with context-specific data to achieve optimal classification results.