When AI took over my teaching videos, students enjoyed them less but learned the same

by Torbjørn Netland, ETH Zurich, Switzerland.

Generative AI has the potential to change education forever. Since the public launch of OpenAI’s generative AI tool ChatGPT in November 2022, students and teachers have eagerly experimented with this promising technology. An important question for teachers is whether AI can fully assume some of our core job tasks.

We checked if AI could take over the resource-intensive job of making teaching videos and how it would impact students’ learning effects. At the outset, we thought the students would learn more from the human-made videos, but, as it turns out, that isn’t necessarily the case. Here’s what we found.

The experiment

We used generative AI tools to generate teaching videos on four different production management concepts and compared their effectiveness versus human-made videos on the same topics. While the human-made videos took several days to make, the analogous AI videos were completed in a few hours. Evidently, generative AI tools can speed up video production by an order of magnitude.

The AI videos were created by prompting ChatGPT for a video script, Midjourney for illustrations, and HeyGen for teacher avatars that spoke the script. To make the videos visually similar to the human-made videos, we uploaded a photo of the human teachers to HeyGen to make human-like avatars. A teacher read the AI-generated scripts to check that the content was correct, and a video curator assembled the generative AI content together to videos in Adobe Premier.

The human-made videos were made in a traditional manner: Teachers wrote a script and recorded themselves in front of a green screen before the video producer selected illustrations and created the educational video in Adobe Premier.

In an online experiment, 447 online participants were split into two experimental groups. Both groups watched four short teaching videos. One group watched two human-made and two AI-generated videos, and the other group watched the same but with AI-generated and human-made videos flipped. After each video, participants rated their enjoyment, and upon completion, they took a short exam to assess their learning outcomes.

Fig. Human-made vs. AI-generated teaching video on the concept of Economic Order Quantity

Same same but different

Contrary to our hypothesis, the results showed no statistically significant differences in exam results for the AI-generated versus human-made videos. Hence, ultimately, the students learned the same in our experiment. This result may serve as a wake-up call for teachers who see themselves as irreplaceable in learning settings.

However, in agreement with our hypothesis, the students reported that they enjoyed the human-made videos more than the AI-generated equivalents. The reduced enjoyment of AI-generated videos may stem from the absence of a personal connection and the nuanced communication styles that human educators naturally incorporate. Such interpersonal elements may not directly impact test scores but contribute to student engagement and motivation, which are quintessential foundations for continued studying and learning.

While our study has shown the potential for AI-generated educational videos, we note that more experimentation and research are needed before broad-brush conclusions can be drawn about generative AI’s potential in teaching. We only tested learning effects among an online panel of gig workers incentivized with a small payment for participating in our study. Further, we tested the effect of short AI-generated educational videos on topics on which consensual information is available on the internet.

Nonetheless, considering that AI is rapidly developing, our study provides an early indication that educational videos can soon be outsourced partly or entirely to generative AI technology. For example, teachers might consider using AI-generated videos for knowledge transfer of basic concepts, thereby freeing up time to focus on more personalized instruction and classroom time. Our results hint that teachers should integrate AI tools in a manner that seeks to complement, rather than substitute, the human touch in teaching. One thing is sure: With the rise of generative AI, it’s time to rethink many teaching tasks besides video generation.

Editor’s note: this article is based on the research published in Netland, T., von Dzengelevski, O., Tesch, K., & Kwasnitschka, D. (2025). Comparing human-made and AI-generated teaching videos: An experimental study on learning effects. Computers & Education, 224, 105164, https://doi.org/10.1016/j.compedu.2024.105164

Author