GSE doctoral student Marcos Rojas
A flexible teaching tool
Rojas began his PhD program at the GSE in 2022, intending to study the use of virtual and augmented reality in medical education. But two months after he arrived, the launch of the AI chatbot ChatGPT took the world by storm, prompting him to shift direction.
“I didn’t know anything about AI, but I was here at Stanford, which is a very rich environment for studying it, and I didn’t know anyone else who was using it in medical education,” he said. “I decided to jump into it.”
He signed up for a computer science course on AI and its potential applications in health care, where he created a rough prototype for his idea. He developed it further through the Learning Design Challenge, a two-quarter program of the Stanford Accelerator for Learning that leads students through the process of investigating a problem in education and designing a technology solution.
Rojas had identified several problems with existing ways of teaching clinical reasoning. Given the cost of hiring actors for in-person medical simulations, he said, instructors often present students with written summaries of a clinical case, a far cry from the experience of talking with a patient to collect the appropriate information to develop a working diagnosis.
Tech firms have developed more interactive approaches, but those carry their own drawbacks, he pointed out. For one thing, they come with preset case studies reflecting certain demographics and epidemiological scenarios that might not be culturally relevant, with limited options for language. What’s more, they can’t account for the different theoretical frameworks of clinical reasoning, making it harder to assess students on a particular approach.
“Different instructors understand clinical reasoning and learning outcomes in different ways,” he said. “These platforms are just too rigid in language, in content, and in understanding of the cognitive task that is being carried out.”
With generative AI, Rojas saw a way to create a more flexible platform that could adapt to any context. Instead of being equipped with pre-programmed case studies, the platform prompts educators to input their preferred content and demographic profiles, based on their own setting and what they want to teach. And the chatbot, as the patient, can converse in whatever language the student uses to start the interaction.
Educators can also choose the type of feedback they want the chatbot to provide, after the student has gathered enough information to make a diagnosis and explained their assessment. Insights highlight areas of improvement for students, such as the relevance of their questions, the use of straightforward language instead of medical jargon, and opportunities to give a more thorough explanation for a recommended course of action.
A convergence of disciplines
The project embodies a rare convergence of education, medicine, and computer science at Stanford, said Salehi, who herself pursued a career in electrical engineering before changing her focus to the science of learning. “At the IDEAL Research Lab, we want everyone to have equal access to fields that can be engines for social mobility, and that includes equity in science education and medical education,” said Salehi. “The beauty of Marcos’ project is that the solution is applicable across culture and language, and it doesn’t require much in the way of resources for the institutions that implement it.”
Caruso, a physician and pediatric anesthesiologist who earned his doctorate in education from the GSE in 2023, attributes the success of the collaboration in part to keeping the focus on the outcome, with scholars contributing expertise from different fields.
“In interdisciplinary research, people sometimes lose sight of the mission, which in this case is providing better medical education and, ultimately, better care for patients,” he said. “It’s easy to get hung up on who paid for what, who gets the credit, what’s in it for me. But this collaboration worked because we’re all mission-focused, we all have the same motivation, we all want to create a good product and get it out to where it can affect the most number of people.”
Rojas first piloted the platform with a focus group of med students, instructors, and attending physicians, whose input informed decisions about making the chatbot’s dialogue more realistic and the type of feedback the platform could provide about students’ performance. He is currently testing it with 100 medical instructors across the United States and plans to pilot it internationally in 2025.
As an education scholar, Rojas emphasized his commitment to research to assess the tool’s effectiveness, something he said distinguishes it from many edtech products on the market.
“We’re in an era where whatever you can imagine with technology, we can make it happen,” he said. “It’s not a question of whether it’s possible. The important question is about the impact. Are people learning? Is it better than the traditional way of doing something? Is it more efficient? We need evidence, and that’s the direction we’re taking with this – to make sure it serves an educational purpose.”
The IDEAL Research Lab and the Stanford Chariot Program have both contributed resources for the platform’s development, and the project has also received funding from Stanford HAI and Stanford Impact Labs. In addition, Rojas won two student awards through the Learning Design Challenge at Stanford, as well as another student award from the Stanford Accelerator for Learning to explore ways to connect the platform with a virtual reality simulator.
Going forward, the team plans to expand the platform to include the ability to conduct physical exams and medical tests, and to incorporate more metrics for assessing students’ skills. They hope to eventually provide the tool at no cost to institutions with limited resources around the world.