Deep Dive into Invoice Intelligence: A Benchmark Study of Leading Models

Abstract

Invoice data extraction is crucial in contemporary business environments, streamlining the processing and organization of voluminous invoice data for effi-cient management and decision-making. This paper presents a thorough benchmark study, evaluating the performance of five state-of-the-art models-LayoutLM, LiLT, Donut, Yolov8x, and Yolov5x—in the automated extraction of data from invoices. Leveraging transfer learning, each model is fine-tuned and assessed based on a private dataset comprising diverse invoice types. The evaluation metrics encompass precision, recall, F1-score, and accuracy, providing a comprehensive performance analysis. Results indicate that Yolov8x outperforms its counterparts with a com-mendable accuracy of 0.913 and an F1-score of 0.928, whereas LiLT trails with an accuracy of 0.537. Furthermore, the paper sheds light on the real-world deployment performance of these models, considering factors such as GPU usage, inference time on GPU and CPU, loading time, and loading GPU usage. LayoutLM demonstrates balanced performance with moderate resource consumption, while other models exhibit varied efficiencies and resource utilizations. Through this investigation, the paper furnishes invaluable insights into the strengths and limitations of each model, offering guidance for practitioners in selecting the optimal model tailored to specific invoice data extraction tasks and operational constraints.