Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models (code)

This repository provides reproducibility code for the paper "Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models".

Errors in medical text can cause delays or even result in incorrect treatment for patients. We explore the importance of prompt optimisation for small and large language models applied to the task of error detection in clinical notes, performing rigorous experiments across frontier models (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, Grok 4) and open-source models (Qwen3 0.6B-32B). We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection accuracy from 0.669 to 0.785 with GPT-5 and from 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art on the MEDEC benchmark.