Predicting Student Mastery: How Bayesian Knowledge Tracing and Item Response Theory Drive Adaptive Learning

When school district curriculum directors and university IT administrators evaluate adaptive learning platforms during their annual software procurement cycles, they are ultimately buying an algorithm. They decide which mathematical model will govern their students' daily practice—whether a system advances a student after three correct answers or holds them back for remediation. That choice hinges on how the software calculates human memory and mastery, a divide currently dominated by two competing statistical frameworks: Bayesian Knowledge Tracing (BKT) and Item Response Theory (IRT).

Intelligent Tutoring Systems (ITS) are not new, but their efficacy depends entirely on the underlying math. A 2016 meta-analysis published in the Review of Educational Research examined 50 controlled evaluations of these systems. The researchers found that, on average, intelligent tutoring raised test scores by 0.66 standard deviations over conventional classroom instruction. "The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 0.66 standard deviations over conventional levels, or from the 50th to the 75th percentile," the authors concluded. But how the system decides a student has reached that 75th percentile dictates how efficiently they get there.[1]

Item Response Theory (IRT) was originally developed by psychometricians for standardized testing, such as the SAT and GRE. It models the probability of a correct answer based on two primary factors: the student's latent ability and the item's inherent difficulty. It also includes a guessing parameter and an item discrimination parameter, which measures how well a question separates high-ability students from low-ability ones. IRT is excellent at placing students on a continuous scale of proficiency, but it assumes that a student's underlying ability is static during the assessment.[3]

Bayesian Knowledge Tracing, developed in the 1990s for the Cognitive Tutor software, takes a fundamentally different approach. It models learning as a dynamic, ongoing process rather than a static measurement. BKT assumes a student is in one of two unobserved states: they either know the target skill or they do not. It uses a Hidden Markov Model to update the probability of mastery after every single interaction the student has with the software.[3]

The math of BKT relies on four explicit probabilities. First is the probability the student already knew the skill before starting. Second is the probability they will learn it after an opportunity to practice. Third is the probability they will guess correctly without actually knowing the skill. Finally, there is the probability they will slip—getting the answer wrong despite actually possessing the knowledge.[3]

The inclusion of the slip parameter is what makes BKT fundamentally different from standard IRT. If a student with a high probability of mastery suddenly misses an easy algebra question, BKT can mathematically attribute the failure to a careless mechanical error rather than a sudden drop in actual knowledge. IRT, conversely, would simply lower the student's latent ability score. This makes BKT inherently more forgiving for procedural tasks where typos are common.[3][4]

While BKT and IRT dominate procedural subjects like mathematics, declarative memory tasks—such as learning a new language—require different algorithms. For vocabulary acquisition, systems rely on spaced repetition. In 2016, researchers at Duolingo developed Half-Life Regression (HLR) to optimize their platform's review schedules. By analyzing 13 million student learning traces, HLR combined the psycholinguistic theory of the "forgetting curve" with machine learning.[2]

"The model marries psycholinguistic theory with modern machine learning techniques, indirectly estimating the 'half-life' of words and concepts in a student's long-term memory," the Duolingo researchers wrote in their 2016 paper published by the Association for Computational Linguistics. Implementing HLR allowed the software to predict exactly when a student was about to forget a word, scheduling a review just in time. The algorithmic shift increased Duolingo's daily student engagement by 12 percent.[2]

In recent years, the EdTech industry has begun moving toward neural networks. Deep Knowledge Tracing (DKT) uses Long Short-Term Memory networks to predict student performance without manually defining parameters like slipping or guessing. A 2020 paper published on arXiv demonstrated that combining BKT with neural networks (BKT-LSTM) outperforms traditional models. "Bayesian Knowledge Tracing (BKT) is a model to capture mastery level of each skill with psychologically meaningful parameters and widely used in successful tutoring systems," the author noted, explaining that the hybrid model captures learning transfer across different skills—something original BKT struggled to do.[3]

The algorithm a district chooses ultimately dictates the student experience. A system running pure IRT functions like a continuous exam, constantly measuring and ranking ability. A system running BKT functions like a patient human tutor, explicitly modeling the moment of learning and forgiving the occasional slip. As neural networks become the commercial standard, the challenge for educators will be maintaining the psychological interpretability of the older models while harnessing the raw predictive power of the new.[4]