I completed my Interdisciplinary Project (IDP) at the Technical University of Munich (TUM) during my master’s studies. In this program, informatics students like me collaborated with other departments to contribute our technical expertise to cross-disciplinary research projects. My project explored how Artificial Intelligence (AI) and Virtual Reality (VR) can be used together to teach cultural heritage.
🕹️ Quiz Game Overview
Download the IDP Report:
📄 View on Google Drive
Author: Ricardo Kraft & Yunus Emre Topal
Supervisor: Prof. Dr. Enkelejda Kasneci
Advisor: M. Sc. Carrie Lau
Institution: Technical University of Munich
Chair: Chair for Human-Centered Technologies for Learning
📘 Abstract
In this project, we integrate Artificial Intelligence (AI) into a Virtual Reality (VR) quiz game setting for Intangible Cultural Heritage (ICH) education, emphasizing the potential of using AI in a VR as a collaborative game master to foster multicultural education. Centered around the Chinese New Year celebration, a multiplayer VR game has been developed to enhance multicultural understanding through immersive, competitive and collaborative experiences. The novelty lies in utilizing an adaptive AI chatbot, powered by Large Language Models (LLM), to serve as a dynamic cultural quiz master. This chatbot provides information on cultural context, facilitates in-game communication, and adapts to user interactions. This project focuses on the technical implementation of adaptive AI in VR, detailing the development challenges, AI integration strategies, and the anticipated impact on cultural education. Highlighting the synergy between VR and AI, it aims to offer new insights for AI-assisted learning within virtual collaborative spaces.
🎮 QuizCraft — The Quiz & Decoration Game
The game combines a quiz and decoration mechanic. It runs in VR and is multiplayer, requiring two players to collaborate. At several points in the game loop, the NPC uses Text-to-Speech (TTS) and ChatGPT + Speech-to-Text (STT) for dynamic, AI-driven dialogue.
The objective is to decorate a living room with cultural elements related to the Chinese New Year. Below is an example of a room before and after decoration:
Before
After
To get a better feeling of the game, here’s the NPC’s scripted dialogue where it explains the game:
🔁 Game Loop
The final game loop of quizzing, talking & decorating is visualized below:
Each round, players split up and enter separate quiz rooms, where they learn about a cultural object and then answer a question about it.
Afterward, players meet in the summary hall to explain what they learned to each other. The NPC listens using STT, then uses ChatGPT to evaluate and provide feedback using TTS on their explanations. Once the discussion is complete, a totem representing the object appears in the room.
When the explanation step is done, the objects are unlocked as decoration elements for the living room. Players can therefore use these to decorate the scene.
Repeating this loop several times results in a fully decorated living room. The goal is to make learning about cultural objects engaging by:
- Reading & quizzing about them
- Explaining them to another player(and receiving AI feedback)
- Decorating a room with the new items
This blend of quizzing, dialogue, and creativity helps players explore cultural heritage in a fun, interactive way with the emphasis on repetition and stimulation through reading, speaking and placing.