Small Prompts, Big Energy and CO2 Impact: Benchmarking Ollama LLMs on CPU and GPU

Date Issued
2025-11-25
Author(s)
Kolovska, Ana
Gusev, Marjan
Mileski, Dimitar
DOI
10.1109/telfor67910.2025.11314365
Abstract
Energy efficiency is a crucial challenge when deploying Large Language Models (LLMs). Electricity usage and related CO2 emissions can differ greatly depending on model architecture, parameter size, prompt length, and inference hardware. In this study, we evaluate 31 popular Ollama models across CPU and GPU inference, resulting in 60 testing scenarios. Energy and carbon metrics were gathered using the NVML and CodeCarbon libraries, providing insights into the environmental impact of LLM inference in data center settings.
Subjects

LLM , Ollama , Energy...

File(s)
Loading...
Thumbnail Image
Name

Small Prompts, Big Energy and CO2 Impact Benchmarking Ollama LLMs on CPU and GPU - accepted version.pdf

Description
Accepted version
Size

227.26 KB

Format

Adobe PDF

Checksum

(MD5):10972e85f04ae5d3cb05af9f671f9a5b