Adaptive learning systems based on Intelligent Tutoring System (ITS) principles have long been established as effective tools for improving learning gains, particularly in algebra and statistics. Despite their proven tutoring capabilities, no ITS, nor any other type of adaptive tutor, has yet been open-sourced. The absence of such a system has meant a high barrier to entry for those interested in replicating adaptive learning experiments or extending and fielding components of their design.
We introduce Open Adaptive Tutor (OATutor), the first fully open-source adaptive tutoring system designed to accelerate research in Intelligent Tutoring Systems (ITS).
Built on the principles of ITS, OATutors offers a Creative Commons-licensed problem library based on popular open-license algebra textbooks. It uses Bayesian Knowledge Tracing for skill mastery estimation and is implemented entirely in React JS with optional logging using Firebase. This platform allows researchers to easily deploy the system on GitHub, run A/B experiments, and integrate OATutor content to learning management system with LTI. OATutor is Section 508 accessibility compliant.
OATutor also features built-in A/B testing functionality through random assignment. This allows researchers to easily assign students to different experimental conditions directly within the platform.
In addition to static pre-set responses, OATutor enables researchers to interact directly with the ChatGPT API for dynamic hint generation. This flexibility allows for testing and experimentation within the platform, enabling studies on AI-driven learning interventions.
OATutor’s flexible meta tags allow researchers to control various aspects of the tutoring experience, enabling fine-tuned experiments. Some examples of meta tags include displaying or not displaying correctness feedback, providing hints, and tracking mastery.
OATutor's Test Mode feature allows the platform to be used in assessment settings by disabling immediate feedback, hints, and mastery updates. In this mode, students can attempt problems independently, without receiving guidance from the system, allowing OATutor to be used in an assessment setting.
Data logs are created as students submit answers to problem steps and scaffolds and unlock hints. Every student interaction with the platform is logged as its own entry to enable researchers and teachers to reconstruct the user’s interaction history. This allows for hint usage analytics and efficacy of A/B testing to be evaluated. The data logs are stored in NoSQL JSON, accessible through the Firebase web interface and can be easily exported to a CSV.
Researchers can seamlessly integrate OATutor with platforms like Qualtrics to conduct controlled A/B tests. This integration allows for precise tracking of student interactions and outcomes across different learning scenarios.
OATutor can be deployed without the need for a backend server, making it easy for researchers to implement in diverse study settings and streamline experimentation.
OATutor was employed in a study to compare the effectiveness of its human-authored hints with those generated by Large Language Models (LLMs), specifically ChatGPT. The experiment involved two conditions: one where participants received hints from OATutor, and another where they received hints generated by ChatGPT. The study aimed to assess the quality and impact of AI-generated hints on learning outcomes.The steps of the experiment were as follows:
You can access all our research paper here:
Explore the full OATutor codebase on GitHub. Our open-source repository provides all the tools you need to customize, deploy, and contribute to the development of this adaptive tutoring system.