Overcoming the 2-Sigma Problem: A Personal Journey in Intelligent Tutoring Systems

It has always been obvious to me that education is the most powerful force on earth and the single greatest accomplishment of humanity.  Becoming learned makes a life worth living.

This is why I have dedicated my life to helping to improve the quality of education for every pupil and have spent my career grappling with one of the most profound challenges in educational research: Benjamin Bloom’s 2-Sigma Problem. Bloom, one of the most influential figures in educational psychology, discovered that pupils who received one-on-one tutoring performed two standard deviations better than those who learned through conventional classroom methods. This means that the average tutored pupil performed better than 98% of pupils who learned in a traditional classroom setting. For decades, educators and researchers have sought ways to replicate these outcomes at scale, but the challenge remains daunting.

For decades I have been exploring how we can use technology to replicate the effectiveness of one-on-one tutoring. Central to this quest has been the development of Intelligent Tutoring Systems (ITS), which promise to bring personalised, high-quality education to every child, regardless of their circumstances. However, as I have grappled with this work, I have found myself balancing the excitement of what ITS can achieve with the realism of its limitations, particularly in comparison to the gold standard of expert human teaching.

A Lifelong Commitment to Educational Excellence

My journey in education began long before I became involved with technology. As a mathematics teacher, I was always struck by the vast differences in how pupils learn and understand complex concepts. This experience led me to embrace the concept of mastery learning, a pedagogical approach that insists every pupil can achieve a high level of understanding, provided they are given the time and support they need. Mastery learning isn’t just about teaching; it’s about ensuring that every pupil has the opportunity to succeed.

Over time, my work in mastery learning has grown into a broader mission to improve educational outcomes for all pupils. This mission led me to explore how technology, particularly ITS, might help solve some of the most persistent problems in education.

The Promise and Pragmatism of Intelligent Tutoring Systems

Intelligent Tutoring Systems have been heralded as a major breakthrough in educational technology. By leveraging advances in artificial intelligence and cognitive science, ITS can adapt to the needs of individual pupils, offering personalised instruction that responds to their unique learning patterns. Studies, including meta-analyses from organisations like the What Works Clearing House, have shown that ITS can significantly improve learning outcomes, particularly in mathematics and science.

However, the reality of ITS is more nuanced. While these systems can provide tailored instruction and immediate feedback, they often fall short of replicating the full depth of the human teacher-pupil interaction. This is particularly true when it comes to the subtleties of pedagogical fidelity—the nuanced, responsive, and empathetic engagement that an expert teacher provides.

The Loss of Fidelity in Machine Learning

One of my biggest concerns with ITS is the loss of fidelity that occurs when pedagogy shifts from teacher to machine. While ITS can simulate many aspects of personalised instruction, they lack the ability to fully replicate the subtle, human elements of teaching that are crucial for deep learning.

I find Michael Tomasello’s work on shared intentionality particularly illuminating here. Tomasello’s research suggests that humans are evolutionarily predisposed to learn from other humans, particularly through shared experiences and joint attention. This idea resonates deeply with my own observations as a teacher. Learning is not just about the transmission of knowledge; it is about the shared process of weaving into existence new  knowledge and understanding in a pupil’s mind – this is negotiation of sorts between the teacher and the pupil, where both understand and appreciate pedagogy for what it is.  Which is to say, the pupil appreciates there is a deliberate act occurring, in which knowledge that is currently known to the teacher will become known to them if they attend carefully. When we replace this human negotiation with a machine, we risk losing the richness of that interaction.  Furthermore, we do not yet fully appreciate the reasons, in human otogeny, that humans are programmed to learn from other humans and what the implications are for attempting to leapfrog millennia of evolution by demanding this innate ability becomes present when one human is replaced by a machine.

The question then becomes: Can we truly achieve the same level of joint intentionality between a pupil and an ITS? The answer, I believe, is still unclear. While ITS can be powerful tools for reinforcing and extending learning, I remain sceptical that they can fully replace the role of a human teacher in fostering deep, meaningful and durable knowledge and understanding.

Pragmatism in a World Without Enough Teachers

Yet, this scepticism must be tempered by a pragmatic recognition of the global educational landscape. The harsh reality is that millions of children worldwide do not have access to any teacher, let alone an expert one. In many regions, classrooms are overcrowded, and pupils are left with minimal support to guide their learning. For these pupils, the alternative to ITS is not an expert teacher but no teacher at all.

In such contexts, the introduction of ITS represents a significant step forward. While an ITS may not provide the same level of personalised interaction as a skilled human tutor, it can offer consistent, accessible, and effective instruction to pupils who would otherwise receive little or no guidance. The system-wide impact of ITS in these situations is profound. By providing educational support at scale, ITS can help close the gap for millions of children, offering them a chance at a quality education that would otherwise be out of reach.

Building a System with Balanced Optimism

Despite my concerns, I remain deeply committed to the development of ITS as part of a broader strategy to improve global education. The system I have been working on in recent years represents an effort to balance the promises of technology with the realities of human learning. This system aims to replicate some of the most effective aspects of one-on-one tutoring while maintaining a deep respect for the pedagogical fidelity that can only be achieved through human interaction.

In building this system, I have had to confront many of the limitations of existing ITS. One key challenge has been developing algorithms that can adapt in real-time to the needs of individual pupils, much like an expert tutor would. Another challenge has been finding ways to incorporate elements of shared intentionality into the system, ensuring that the learning experience remains as close as possible to the kind of joint attention that is so critical in human learning.

This work has been deeply rewarding, but it has also reinforced my belief that while ITS can play a significant role in education, they should not be seen as a replacement for human teachers. Instead, they should be viewed as powerful tools that can support and enhance the work that teachers do, particularly in providing timely personalised feedback and reinforcing learning.

The Future of ITS: Balancing Optimism with Realism

Looking ahead, it is clear to me that ITS will play an important role in the future of education, but one that must be approached with caution and pragmatism. The potential of these systems to provide personalised education at scale is enormous, particularly in contexts where access to quality education is limited. However, as we continue to develop and deploy ITS, we must remain mindful of their limitations.

The work of understanding how joint intentionality can be replicated (or if it even can be) in machine learning is still in its infancy. Until we have a clearer picture, it is crucial that we do not overestimate the capabilities of ITS. These systems should be used to complement, not replace, the irreplaceable human elements of teaching.

A Call for Thoughtful Innovation

Now that I am retired and I reflect on my journey, both as an educator and as an inventor of ITS, I am filled with a sense of cautious optimism. There is no doubt that technology will continue to play a vital role in the future of education, but we must be thoughtful about how we integrate it into our classrooms.

My desire to see Bloom’s 2-Sigma Problem solved remains as strong as ever, but it is tempered by a deep respect for the complexities of human learning. I invite educators, technologists, and policymakers to join me in this conversation, to explore how we can use technology to enhance education while preserving the human connections that are at the heart of all meaningful learning.

As we move forward, I urge everyone to strive to balance innovation with wisdom, ensuring that every pupil has the opportunity to achieve mastery, not just through machines, but through the enduring guidance of their teachers.