Ciência_Iscte
Publications
Publication Detailed Description
Benchmarking Small Language Models for Medical Question Answering in Resource-Constrained Environments
Proceedings of the 28th International Conference on Enterprise Information Systems
Year (definitive publication)
2026
Language
English
Country
Spain
More Information
Web of Science®
This publication is not indexed in Web of Science®
Scopus
This publication is not indexed in Scopus
Google Scholar
This publication is not indexed in Google Scholar
This publication is not indexed in Overton
Abstract
While Large Language Models (LLMs) demonstrate significant potential in identifying complex patterns, their deployment in the medical domain is restricted by critical barriers: the misalignment between standardized benchmarks and real-world patient consultations, the data privacy risks inherent in commercial APIs, and the prohibitive computational costs of local deployment. This study investigates the feasibility of Small Language Models (SLMs)—defined here as architectures with fewer than 10 billion parameters—as a viable solution to these constraints. We conduct a comparative evaluation of domain-specific and general-purpose models from the Hugging Face ecosystem, analyzing their effectiveness in medical question-answering tasks specifically tailored for privacy-conscious and resource-limited settings. Our results reveal a distinct trade-off between information density and factual reliability; while general-purpose models achieved higher comprehensiveness at the cost of increased hallucinations, clinically conservative models minimized errors but lacked coverage. Ultimately, we identify that balanced architectures, such as Medgemma-1.5-4b, offer the most promising path for deploying safe, accessible, and efficient AI decision support in resource-constrained medical environments.
Acknowledgements
--
Keywords
Medical Question Answering,Language Models,Hugging Face,Resource-Constrained Environments
Português