Generative artificial intelligence (AI) has emerged as one of the most transformative technologies shaping contemporary society, specifically in education and healthcare. Although AI has been discussed for decades, its widespread adoption in the 2020s has intensified both enthusiasm and concern regarding its ethical, educational, and professional implications. Generative AI is already influencing clinical practice, research, and administrative workflows in health professions education and creating an urgent need for academic programs to prepare students for AI-enabled workplaces. Employers increasingly expect graduates to possess AI-related competencies, in some cases prioritizing these skills over traditional experiences. Health informatics, positioned at the intersection of technology and healthcare, is uniquely positioned to lead educational innovation by integrating AI literacy, defined as the ability to critically, effectively, and ethically understand and use AI in graduate curricula.
The aim of this study was to evaluate whether integrating generative AI-based assignments into graduate health informatics education could improve students’ foundational knowledge of generative AI and support the development of AI-related skills and professional attitudes. It also determined whether students showed measurable knowledge gains after completion of generative AI-based assignments and whether students perceived growth in their ability to responsibly and effectively use generative AI in future health informatics careers.
This multisite, mixed-methods design included Master’s-level health informatics students at the University of Illinois Chicago (UIC) and the University of San Francisco (USF) during the Fall 2024 semester. Seventeen students participated (10 from UIC and seven from USF), with institutional review board approval obtained at both institutions under exempt research determinations. The study was embedded within an online capstone course in the Master of Science in Health Informatics program at UIC. Students completed individualized generative AI projects focused on real-world clinical, organizational, or ethical challenges. The study tool was placed in a hybrid course titled Exploring Generative AI Ethics at USF, in which students designed ethically grounded generative AI implementations for healthcare or operational problems.
Data collection involved a 23-item open-book pretest and post-test to assess foundational generative AI knowledge, along with structured student reflections collected at the beginning and end of the semester. Quantitative data were used to assess changes in knowledge. Qualitative reflection data were analyzed using four AI competency domains: essentials of AI, applications to health informatics, AI transformations of information and knowledge, and organizational change and adoption. Reflections were categorized by faculty reviewers and reconciled by consensus.
Students at both institutions showed improvements in generative AI knowledge from pretest to post-test. UIC students’ average scores increased from 81% to 93%, while USF students’ scores rose from 77% to 80%. Although the open-book format limited causal claims, these gains indicated strong engagement with AI concepts. Qualitative reflections revealed substantial development in AI-related skills and professional attitudes. Students reported increased confidence, creativity, and efficiency, as well as reduced anxiety when using generative AI, particularly for brainstorming, research support, and focusing on higher-level analysis. Ethical awareness emerged as a dominant theme, with students expressing concerns about bias, data privacy, accuracy, patient safety, and overreliance on AI. Differences between cohorts reflected programmatic orientations: UIC students emphasized clinical applications and workflow integration, whereas USF students focused more on technical foundations and governance. Across both programs, ethics and knowledge transformation were the most prominent areas of interest, followed by organizational efficiency and technology selection.
Overall, this study demonstrates that rapidly integrating generative AI assignments into graduate health informatics curricula can meaningfully increase AI knowledge, skills, and professional attitudes of students. A modular curriculum design allowed both programs to address immediate workforce demands without extensive curricular overhaul, offering a practical and scalable model for health professions education. Study limitations include a small sample size and differences in course structure, which may limit generalizability.
Reference: Seba F, Isola M, Mills L, Zalake M, Krive J. Incorporating generative AI into a health informatics curriculum to build 21st century competencies: multisite pre-post study. JMIR Med Inform. 2025;13:e76507. doi:10.2196/76507
