Benchmarking Framework for Multimodal Large Language Models in Image Understanding and Processing

Costin-Emanuel Vasile, Sophia-Anca Minișan, Călin Bîră

MDPI Journal of Imaging

JournalQ2IF 3.8To be published

New Programming Approaches for Hardware Accelerators

This work introduces a configurable benchmarking framework for evaluating MLLMs in image understanding and enhancement. The framework supports quality assessment, no-reference enhancement, and reference-guided enhancement, while recording both image-quality metrics and agent-specific characteristics such as execution time, token usage, model requests, and tool interactions. We further propose an MLLM-driven image enhancement approach in which external image analysis and image processing operations are exposed as callable tools, allowing the model to iteratively select suitable operations and their parameters. The framework is demonstrated using a locally deployed general-purpose MLLM and blur and contrast distortions from KADID-10k. The results show that the effectiveness of enhancement depends heavily on the type of degradation, tool availability, and reference information, while unrestricted access to tools does not necessarily improve performance. The proposed framework provides a reproducible basis for studying and comparing MLLM-based image understanding and processing systems.