Code in Place instructors come from all sorts of backgrounds, from undergrads who’ve recently taken the course themselves to professional computer programmers working in the industry. Enthusiastic as they are to introduce beginners to the world of coding, many instructors approach the opportunity with little or no prior teaching experience.
The volunteer instructors received basic training, clear lesson goals, and session outlines to prepare for their role, and many welcomed the chance to receive automated input on their sessions, said study co-author Chris Piech, an assistant professor of computer science education at Stanford and co-founder of Code in Place.
“We make such a big deal in education about the importance of timely feedback for students, but when do teachers get that kind of feedback?” he said. “Maybe the principal will come in and sit in on your class, which seems terrifying. It’s much more comfortable to engage with feedback that’s not coming from your principal, and you can get it not just after years of practice but from your first day on the job.”
Instructors received their feedback from the tool through an app within a few days after each class, so they could reflect on it before the next session. Presented in a colorful, easy-to-read format, the feedback used positive, nonjudgmental language and included specific examples of dialogue from their class to illustrate supportive conversational patterns.
The researchers found that, on average, instructors who reviewed their feedback subsequently increased their use of uptake and questioning, with the most significant changes taking place in the third week of the course. Student learning and satisfaction with the course also increased among those whose instructors received feedback, compared with the control group. Code in Place doesn’t administer an end-of-course exam, so the researchers used the completion rates of optional assignments and course surveys to measure student learning and satisfaction.
Testing in other settings
Subsequent research by Demszky with one of the study’s coauthors, Jing Liu, PhD ’18, studied the use of the tool among instructors who worked one-on-one with high school students in an online mentoring program called Polygence. The researchers, who will present their findings in July at the 2023 Learning at Scale conference, found that on average the tool improved mentors’ uptake of student contributions by 10%, reduced their talk time by 5%, and improved students’ experience with the program as well as their relative optimism about their academic future.
Demszky is currently conducting a study of the tool’s use for in-person, K-12 school classrooms, and she noted the challenge of generating the high-quality transcription she was able to obtain from a virtual setting. “The audio quality from the classroom is not great, and separating voices is not easy,” she said. “Natural language processing can do so much once you have the transcripts – but you need good transcripts.”
She stressed that the tool was not designed for surveillance or evaluation purposes, but to support teachers’ professional development by giving them an opportunity to reflect on their practices. She likened it to a fitness tracker, providing information for its users’ own benefit.
The tool also was not designed to replace human feedback but to complement other professional development resources, she said.
Along with Dora Demszky, Jing Liu, and Chris Piech, the study was co-authored by Dan Jurafsky, a professor of linguistics and of computer science at Stanford, and Heather C. Hill, a professor at the Harvard Graduate School of Education.