Towards representative GenAI in health professions education: detecting and correcting demographic bias in AI-powered simulation education

Generative AI (GenAI) is increasingly deployed in health professions education, particularly for simulated patients and instructional imagery. However, concerns have emerged regarding demographic bias in AI-generated outputs, with potential consequences for equity, realism, and global applicability. This study presents a multi-method analysis of demographic representation across simulated ‘patient’ cohorts (GPT-3.5, GPT-4-mini) and AI-generated ‘clinical’ images (DALL·E 3, Midjourney). Quantitative comparisons against national census and survey benchmarks revealed significant overrepresentation of lighter skin tones, males, and middle-aged adults, alongside the near-complete absence of certain ethnic and age groups. However, prompt-based interventions incorporating demographic data achieved marked improvements in representativeness.

These findings raise important questions about the readiness of current GenAI models for use in inclusive medical training environments. Inaccurate or stereotyped representations may undermine educational authenticity, reinforce existing disparities, and skew students’ expectations about the patient populations they will encounter in practice.

Building on this analysis, we propose a framework for systematically auditing AI tools in medical education. Central to this is the development of an “AI report card” to evaluate models on key dimensions of demographic safety, regional appropriateness, and educational validity. The report card is designed to support educators and institutions in selecting GenAI tools that align with their curricular and equity goals.

This work contributes to ongoing international efforts to ensure that the globalisation of health professions education is underpinned by principles of fairness, inclusivity, and contextual relevance. Future work will validate the framework across diverse educational settings and explore model fine-tuning and prompt engineering strategies to ensure safer, more representative AI-assisted simulation.