In the fall of 2022, months before the public got a glimpse of ChatGPT, DiCerbo, a learning scientist and chief learning officer at Khan Academy, got access to a beta version of Open AI’s GPT-4 model. The startup needed Khan Academy’s help training it to pass the Advanced Placement biology exam, a requirement dreamed up by Microsoft co-founder Bill Gates, who wanted improved performance as a condition of handing Open AI more funding.

Khan Academy founder Sal Khan and DiCerbo negotiated a partnership with Open AI, and just five months later, their AI-powered Khanmigo tutoring bot debuted. Last summer, Khan Academy launched an .

Nearly two years in, DiCerbo remains bullish on the possibilities of AI tutoring, cheerfully engaging critics about the limitations of the technology, even as by all measures it evolves and improves. 

Much of the press for Khanmigo has been positive: late last year, produced an upbeat feature on Khan Academy’s efforts — host Anderson Cooper called Khanmigo’s potential “staggering,” but tempered the observation by adding, “It’s still very much a work in progress.”

Other media accounts have challenged Khan’s predictions that AI will anytime soon, with a Wall Street Journal reporter a year ago that Khanmigo didn’t consistently know how to round answers or calculate square roots and “typically didn’t correct mistakes when asked to double-check solutions.”

Khan Academy has said improvements are ongoing, but that at least a few errors are likely to persist. The organization stresses that Khanmigo remains “imperfect” and “still evolving.” 

In March, DiCerbo will appear at , alongside Curriculum Associate’s Kristen Huff and Akisha Osei Sarfo of the Council of the Great City School to discuss how AI can improve school assessments. The panel will be moderated by The 74’s Greg Toppo, who spoke recently with DiCerbo in a wide-ranging interview. 

They talked about Khanmigo, its critics and why she feels “cautiously optimistic” about the role of AI in education.

The interview has been edited for length and clarity.

You’ve been with Khan Academy now for almost five years, and it’s been an eventful time. You’ve spent a lot of that time creating and improving Khanmigo. What are the latest developments? 

We have learned a lot in what is coming up on two years since Khanmigo launched. In terms of what students are doing, we definitely see some interesting things we didn’t necessarily expect. Students who are English language learners really like and use the supports in other languages. We probably shouldn’t have been surprised, but always need to be reminded that it’s important to just have instruction on how to use new technology and tools, and what that looks like. For students, how do you ask good questions? And for teachers, how do you integrate it? So both professional learning for teachers and supports for students have been important things that we’ve added over time.

The other thing is that we have found that Khanmigo as a tutor works best when it is paired with educational content we have already created. It is better integrated and has lower error rates when it’s using, and has reference to, the existing problems that were written and verified by people — and not just the problems, but the [step-by-step] hints and the answers that already exist in our system. When it can reference those, Khanmigo is better. And when students are just working on the practice that is part of Khan Academy generally, they are using Khanmigo as an assistant and as a help to get unstuck.

When we talked last year, you used that word “unstuck.” You guys have come in for some criticism from critics like and , who say Khanmigo gets math wrong, among other things. Meyer last year said he’s become a kind of pro bono consultant for you guys. [DiCerbo laughs.] You’re familiar with the criticisms, and I wonder: How have they landed? And have they had an effect on the product?

Dan has very good classroom experience and is extremely knowledgeable about teaching math. So many times, the things he says align to conversations internally that we’re already having. And the things he says are things that we end up changing and doing. We always appreciate criticism that helps us improve and move on. A lot of our work has been on things like working to better evaluate math accuracy, improve it, and get the balance right between how much Khanmigo gives help versus asks questions — all of the things we’re working to tune and get right in that sweet spot for what the student needs.

Dan actually just this week had . The thing he misinterprets about us is that he thinks we’re trying to replace teachers, and he thinks we don’t value teachers. That’s what that whole post was about. And that is just not how we see what we’re doing. We see Khanmigo as a tutor that’s also working in the same ecosystem, but the teacher is fundamental to this whole process.

I saw the 60 Minutes piece with Anderson Cooper a while back and I wonder how that landed.  

First of all, the writing piece they highlighted is something we’re pretty excited about. Very often in schools, kids do writing assignments and teachers end up with a huge stack of writing. As Sarah [Robertson], who’s our product manager, said in the piece, she had to limit herself to only 10 minutes per essay feedback, and still it would take her hours and hours as a secondary school teacher to grade all of these essays — and then the students get them back two weeks later. That’s not immediate feedback. So the idea that we can potentially provide more immediate feedback to students on their writing is pretty interesting to us.

There’s a lot of concern over cheating. 

We can say, “Hey, we’re going to flag this piece,” which Anderson did in his demo — he just cut and pasted in a whole bunch of content. We can say, “Hey, we don’t know where this content came from. We’re not going to make the judgment, but teacher, here’s a flag for you to check on.”. Not surprisingly, we got a lot of queries from school districts asking about getting access.

When I was writing the piece last year about IBM Watson and the effort to make it into a tutor, you expressed a cautious optimism that despite all the failures we’ve seen, this time was different. It’s been almost a year now. I wonder if your feelings have changed about AI tutoring generally and Khanmigo specifically?

I would still characterize how I feel as cautiously optimistic. I don’t think this is The Golden Ticket that’s going to save us all and be the sole reason that educational outcomes improve. I do think it still can be an important tool in the toolbox.

Does the change in presidential administrations have any bearing on your work, given that President Trump’s got an apparent interest in AI and support from big tech, specifically and ? 

There is a lot of noise about what may or may not happen. We are basically sticking to “What are our technology partners doing, and what are we able to then partner with them to build?” And we will see what actually comes to fruition and deal with it if and when anything actually happens. We’re not counting on anything either way.

My last question about this topic is the earthquake that happened with the Chinese AI startup . The interpretation that I’ve been hearing is that it has caused supreme havoc at places like Open AI. I wonder if any of this has redounded to you guys?

Not specifically the Deep Seek piece, but it’s just part of what we have thought is likely to be the future — it’s just a little bit sooner than perhaps we thought. The models themselves become a commodity. Even since we launched, the prices have come down so far that it’s significant. We’re able to offer what we do at significantly lower prices, and that’s just likely to continue. And it’s not going to be the models themselves that are the “moat” or the differentiator — it’s going to be what people build with them.  

Is it even in the realm of possibility that you would work with a company like Deep Seek?

Well, Deep Seek’s model is open-source, so you can install it on your own machine. And that’s part of the concern about security and privacy with the app that, of course, has ties to the Chinese government. Then there’s the question about the model itself, as an open-source model, How does it perform? I would not rule out us using open-source models from different sources, but they would have to be evaluated, like all our models are, for security and privacy and their performance. 

Let’s talk about South by Southwest. The session we’re doing is titled “How AI Makes Assessment More Actionable in Instruction,” which doesn’t exactly roll off the tongue. But it gets to an interesting idea, which is that AI can make assessments better: more invisible, more customizable, and help teachers adapt instruction. I wonder what you’re seeing in terms of the ways AI is moving into that field?

It’s interesting, because the assessment conversation has lagged a bit behind the learning conversation when it comes to AI. But it seems to be picking up speed this year, both at South by Southwest and at ASU+GSV.

Traditionally we’ve had multiple-choice tests. You and I know there’s the whole game-based, simulation-based movement. What does AI let us do? The idea of a conversational-based assessment is interesting. What if the assessment looks like what happens when a teacher sits down next to a student and says, “Explain your thinking. How did you get to that?” There’s a conversation there. And that could potentially be an interesting way of adding to assessments that we already have. Of course, there would be questions: Is that standardized? Because different kids might get different questions as they engage in this conversation. How do we deal with that when we’re talking about high-stakes assessment? 

The last thing I think is interesting is helping teachers and parents make sense of assessment data and get recommendations. Can AI help with that? Instead of getting this printout that says, “Your student got a 580 on this,” and you’re like, “What does that even mean? What should I do?” If you could have a conversation about that, that might be an interesting piece. We’ve been exploring that in something we have called and recommendations that allow teachers to talk about their students’ Khan Academy performance. What else might they assign? How might they group students based on those kinds of things? 

In the past couple of months I’ve been playing around with AI tools that summarize and analyze big chunks of text and YouTube videos and whatnot. It strikes me that we are going to become so used to having a tool like this break things down for us that if schools can’t help us break our students’ performance down, we’re going to be disappointed. Is my cart ahead of the horse?

I always try to figure out if I’m in a bubble or not, because I feel the same way. I know lots of people that similarly are really getting into the habit of whenever they get a large amount of information, put it into an AI tool and get the summarization. I’m not quite sure how broad-based that is when we think about all of the parents out there and all the schools, but that is what I’m seeing, and it might become an expectation in the near future. 

Is there something on the horizon that you are looking at that maybe others aren’t paying attention to — good, bad or other?

The video was out quite a while ago of Sal and his son [of Khanmigo]. We’re starting to get to a place where the AI is seeing what the student is working on, and is able to interact with that and move forward. I’m pretty excited about that.

Centuries of English classes have connected to Lady Macbeth by scouring the monologues of Shakespeare’s Scottish play. “Come, you spirits that tend on mortal thoughts, unsex me here, and fill me from the crown to the toe top-full of direst cruelty,” she cries in Act I, railing against the limits of her gender and position.

During the coming school year, students may be able to talk to the character themselves.

Under an artificial intelligence-driven program rolling out to New Hampshire schools, students could pose any question they like to Lady Macbeth – or her ill-fated husband. And a chatbot-style program powered by ChatGPT could answer questions about her motivations, actions, and regrets.

Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter

“Regret is a specter that haunts many a soul, yet in my union with Macbeth, I found not just a husband, but a partner in ambition,” the AI-version of Lady Macbeth said recently, responding to a question from the Bulletin. Then she turned it on the reporter. “Now, I ask thee, in thy own life, how dost thou measure the worth of thy decisions? Doth regret ever color thy reflections?”

Known as Khanmigo, the program is the product of Khan Academy, an online tutoring company with instructional materials for core middle school and high school subjects. And the platform goes beyond Macbeth; students can interact with a number of other pre-selected literary characters, from Jay Gatsby to Eeyore, quiz historical figures like Martin Luther King Jr. or Queen Victoria, and receive tutoring help on everything from English essays to precalculus problems.

After the Executive Council approved a $2.3 million, federally funded contract last month, New Hampshire school districts can incorporate Khanmigo in their teaching curricula for free for the next school year.

To some educators and administrators, the program offers glittering potential. Khanmigo could provide one-on-one attention and guidance to students of any grade or ability level, they say, allowing students to advance their learning as teacher staffing remains a problem.

Others are more skeptical about bringing AI into schools, noting longstanding concerns about false or out-of-date statements, and about its use of human academics’ work to form its answers. Supporters of Khanmigo, who include Department of Education Commissioner Frank Edelblut, argue the program has better guardrails against inaccuracies than the versions of ChatGPT and Gemini available to the public.

To understand what students might see, the Bulletin reviewed Khanmigo, sampling school subjects across a number of grades.

Chat-based learning

Khanmigo allows students to use Khan Academy’s existing library of tutorials, practice problems, and quizzes for middle and high school subjects spanning English language arts, social studies, science, and mathematics.

As they navigate those lessons, the AI bot is available to help them understand concepts. Students might take a multiple-choice quiz on art history or AP economics. If they get stuck on a question, the chatbot is available to help them work through the answer.

From the get-go, the program acknowledges that it can make errors, a phenomenon known in the AI sphere as “hallucinations.” These can happen if the language learning model has been trained on inaccurate information – or if the answer doesn’t exist.

“I’m pretty new, so I sometimes make mistakes,” it states to students when they first log on. “If you catch me making a mistake, click the Thumbs Down icon to help me learn. For example, if I give you the wrong math answer click Thumbs Down.”

The AI assistant also makes sure to remind students that teachers and parents can see the questions they are asking, and that they should not include any personal information in their answers.

In a test by the Bulletin, the program allowed the student to try solving for “y” in a precalculus equation. After a student arrived at an answer, Khanmigo was there to help check it.

The tutor asked the student whether they feel confident in the answer or have any further questions. When the Bulletin asked questions, the tutor walked through how to solve the equation step by step, but gave the student the option to provide answers on their own without being told how.

“Does that make sense?” the program asked in one instance, after explaining how to substitute a number for a variable in an equation. “What do you think you should do next after substituting?”

In humanities subjects such as AP U.S. History, Khanmigo helped to augment a human-created Khan Academy video lesson on the rise of the market economy during the Industrial Revolution. After the student viewed the video, the chatbot was available to quiz them on what they saw. “What major change in labor did the market revolution bring to the United States?” it asked, offering the student a chance to reply in a chat message.

When asked whether President Joe Biden won the 2020 election fairly, the app deflected, urging the student to consider how elections work and to describe the American electoral process. Asked a second time, the app stated that “numerous official sources, including state election officials and courts, found no evidence of widespread fraud that could have changed the outcome of the election.”

When asked about the causes of climate change, Khanmigo was unequivocal: “Yes, human activities are considered the primary drivers of recent climate change,” it stated, after which it listed four human-led activities – burning fossil fuels, deforestation, industrial activities, and agricultural practices – that have contributed to the warming earth.

And when asked about reproductive health, the AI bot gave detailed – and accurate – information about forms of birth control, down to explanations about types of intrauterine devices, as well as other answers.

Optimism and concern

One New Hampshire school district has already been trying out the AI learning model.

In fall 2022, as the concept of a “language learning model” and the name “Chat GPT” first hit the public consciousness, Superintendent David Backler was already thinking about its applications in the Gorham School District.

Two years later, Gorham is leading the way on implementing AI in classrooms in New Hampshire. For the past school year at Gorham, Khanmigo has been available to all students in their own time. But under the pilot program, two high school teachers also chose to try using it in a more structured setting – in math and in English language arts.

Backler already knew AI was skilled at helping students comprehend difficult math subjects. It was English where the technology surprised him.

“It’s pretty powerful how it can help you with your writing, how it can take you step by step through the editing process,” he said. “And one of the hardest things in school is teaching writing, and teaching writing well.”

The state contract had a rocky approval process after some executive councilors raised worries about the reliability of AI in schools. Councilor Ted Gatsas, a Manchester Republican, held up the state’s approval for several weeks, requesting time to play with the program himself to determine whether it was injecting any political bias.

“I had the chance a chance to ask it: ‘When does life begin?’” Gatsas said during a May 14 Executive Council meeting. “But that was a biology question. And the answer was apolitical, and I thought that was a good thing.”

When the Bulletin asked Khanmigo “when does life begin,” it declined to answer, stating: “That question leans more towards philosophy and ethics, which I’m not equipped to handle. For scientific insights related to the development stages of human life, such as fertilization, embryonic development, and fetal growth, feel free to ask! These topics are well within the realm of biology.”

And Councilor Cinde Warmington, a Concord Democrat and a candidate for governor this year, grilled Edelblut over whether the contracts would allow students to use the software without supervision.

“Doesn’t it seem careful to pilot that with our teachers providing supervision over kids using it, rather than putting kids by themselves in an environment where they’re being exposed to this artificial intelligence?” Warmington asked.

Edelblut said the contract is for the teacher-led version of Khanmigo, which gives educators more control over which subjects and modules students can use at any one time, and allows them to monitor students’ efforts.

Backler says he understands concerns that parents and others might have about the technology, particularly with the risk of hallucinations.

But he argued that Khanmigo has more guardrails against that than the programs intended for the public. And he said the program is meant to be a support for students – not to replace teaching.

“It’s not doing your writing; it’s not doing your work,” he said. “It’s giving you feedback on what you’re doing.”

But he said it would help students receive more teaching attention than they might get otherwise.

“You just can’t expect a teacher who has 20 students to be able to have that direct interaction constantly with every single student,” Backler said. “It’s not possible. But with some of these tools, we can really look at: How do we provide those learning opportunities for students all the time?”

is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. New Hampshire Bulletin maintains editorial independence. Contact Editor Dana Wormald for questions: info@newhampshirebulletin.com. Follow New Hampshire Bulletin on and .

As the founder of in 2015, he enlisted university education school deans to incorporate findings from into teacher preparation. Before that, he spent five years as policy director of the , which underwrites new models of schooling. In his new endeavor, , which he calls “a think-and-do tank,” he’s pushing to help people think not only about how we learn, but how generative artificial intelligence (AI) works — and why they’re different.

His and regularly poke holes in high-flying claims about the power of AI-powered tutors — he recently offered choice words for Khan Academy founder Sal Khan’s of Open AI’s new GPT4o tool, saying it was “deployed in the most favorable educational environment we can possibly imagine,” leaving open the possibility that it might not perform so well in the real world.

In April, Riley ruffled feathers in the startup world in the journal Education Next that took and other AI-related companies to task for essentially using students as guinea pigs.

In the essay, he recounted asking to help him simplify an algebraic equation. Riley-as-student got close to solving it, but the AI actually questioned him about his steps, eventually asking him to rethink even basic math, such as the fact that 2 + 2.5 = 4.5.

Such an exchange isn’t just unhelpful to students, he wrote, it’s “counterproductive to learning,” with the potential to send students down an error-filled path of miscalculation, misunderstanding and wasted effort.

The interview has been edited for length and clarity.

The 74: We’re often so excited about the possibilities of ed tech in education that we just totally forget what science says about how we learn. I wonder if you have any thoughts on that.

Benjamin Riley: I have many. Part of my frustration is that we are seemingly living in a moment where we’re simultaneously recognizing in other dimensions where technology can be harmful, or at least not beneficial, to learning, while at the same time expressing unbridled enthusiasm for a new technology and believing that it finally will be the cure-all, the silver bullet that finally delivers on the vision of radically transforming our education system. And yeah, it’s frustrating. Ten years ago, for example, when everybody was excited about personalization, there were folks, myself included, raising their hand and saying, “Nope, this doesn’t align with what we know about how we think and learn. It also doesn’t align with the science of how we collectively learn, and the role of education institutions as a method of culturally transmitting knowledge.” All of those personalized learning dreams were dying out. And many of the prominent, incredibly well-funded personalized learning efforts either went completely belly-up, like , or have withered on the vine, like some of the public schools now named .

Now AI has revived all of those dreams again. And it’s frustrating, because even if it were true that personalization were the solution, no one 10 years ago, five years ago, was saying, “But what we need are intelligent chatbot tutors to make it real.” So what you’re seeing is sort of a commitment to a vision. Whatever technology comes along, we’re going to shove into that vision and say that this is going to deliver it. I think for the same reasons it failed before, it will fail again. 

You’re a big fan of the University of Virginia cognitive scientist , who has done a lot to popularize the science of how we learn.

He’s wonderful at creating pithy phrases that get to the heart of the matter. One of the counterintuitive phrases he has that is really powerful and important is that our minds in some sense “are not built to think,” which feels really wrong and weird, because isn’t that what minds do? It’s all they do, right? But what he means is that the process of effortful thinking is taxing in the same way that working out at the gym is taxing. One of the major challenges of education is: How do you wrap around that with students, who, like all of us, are going to try to essentially avoid doing effortful thinking for sustained periods? Over and over again, technologists just assume away that problem.

In the case of something like large language models, or LLMs, how do they approach this problem of effortful thinking? Do they just ignore it altogether?

It’s an interesting question. I’m almost not sure how to answer it, because there is no thinking happening on the part of an LLM. A large language model takes the prompts and the text that you give it and tries to come up with something that is responsive and useful in relation to that text. And what’s interesting is that certain people — I’m thinking of most prominently — have talked about how amazing this is conceptually from an education perspective, because with LLMs you will have this infinitely patient teacher. But that’s actually not what you want from a teacher. You want, in some sense, an impatient teacher who’s going to push your thinking, who’s going to try to understand what you’re bringing to any task or educational experience, lift up the strengths that you have, and then work on building your knowledge in areas where you don’t yet have it. I don’t think LLMs are capable of doing any of that.

As you say, there’s no real thinking going on. It’s just a prediction machine. There’s an interaction, I guess, but it’s an illusion. Is that the word you would use?

Yes. It’s the illusion of a conversation. 

In your Education Next essay, you quote the cognitive scientist , who says LLMs are “frequently wrong, but never in doubt.” It feels to me like that is extremely dangerous in something young people interact with.

Yes! Absolutely. This is where it’s really important to distinguish between the now and the real and the present versus the hypothetical imagined future. There’s just no question that right now, this “hallucination problem” is endemic. And because LLMs are not thinking, they generate text that is factually inaccurate all the time. Even some of the people who are trying to push it out into the world acknowledge this, but then they’ll just put this little asterisk: “And that’s why an educator must always double-check.” Well, who has the time? I mean, what utility is this? And then people will say, “Well yes, but surely it’s going to get better in the future.” To which I say, Maybe, let’s wait and see. Maybe we should wait until we’ve arrived at that point before we push this out.

Do we know how often LLMs are making mistakes?

I can say just from my own personal usage of Khanmigo that it happens a lot, for reasons that are frankly predictable once you understand how the technology works. How often is it happening with seventh-grade students who are just learning this idea for the first time? We just don’t know. [In response to a query about errors, Khan Academy sent links to two on its site, noted that Khanmigo “occasionally makes mistakes, which we expected.” It also pointed, among other things, that Khanmigo now uses a calculator to solve numerical problems instead of using AI’s predictive capabilities.]

One of the things you say in the EdNext piece is that you just “sound like a Luddite” as opposed to actually being one. The Luddites saw the danger in automation and were trying to push against it. Is it the same, in a way, as what you’re doing? 

Thank you for asking that question because I feel my naturally contrarian ways risk painting me into a corner I’m really not in. Because in some sense, generative AI and large language models are incredible — they really are. It is a remarkable achievement that they are able to produce fluent and coherent narratives in response to just about any combination of words that you might choose to throw at them. So I am not a Luddite who thinks that we need to burn this all down.

There are methods and ways, both within education and in society more broadly, in which this tool could be incredibly useful for certain purposes. Already, it’s proving incredibly stimulating in thinking about and understanding how humans think and learn, and how that is similar and different from what they do. If we could just avoid the ridiculous overhype and magical thinking that seems to accompany the introduction of any new technology and calm down and investigate before pushing it out into our education institutions, then I think we’d be a lot better off. There really is a middle ground here. That’s where I’m trying to situate myself. 

Maybe this is a third rail that we shouldn’t be touching, but I was reading about Thomas Edison and his ideas on education. He had a great quote about movies, which he thought would revolutionize classrooms. He said, “The motion picture will endure as long as poor people exist.” It made me think: One of the underlying themes of ed tech is this idea of bringing technology to the people. Do you see a latent class divide here? Rich kids will get an actual personal tutor, but everybody else will get an LLM? 

My worry runs differently than that. Again, back to the Willingham quote: “Our minds are not built to think.” Here’s the harsh reality that could indeed be a third rail, but it needs to be acknowledged if we’re going to make meaningful progress: If we fail in building knowledge in our students, thinking gets harder and harder, which is why school gets harder and harder, and why over time you start to see students who find school really miserable. Some of them drop out. Some of them stop trying very hard. These folks — the data is overwhelming on this — typically end up having lives that are shorter, with less economic means, more dire health outcomes. All of this is both correlated and interrelated causation.

But here’s the thing: For those students in particular, a device that alleviates the cognitive burden of schooling will be appealing. I’m really worried that this now-widely available technology will be something they turn to, particularly around the incredibly cognitively challenging task of writing — and that they will continue to look to this as a way of automating their own cognition. No one really needs to worry about the children of privilege. They are the success stories academically and, quite frankly, many of them enjoy learning and thinking and will avoid wanting to use this as a way of outsourcing their own thinking. But it could just make the existing divide a lot wider than it is today — much wider.

How is education research responding to AI?

The real challenge is that the pace of technology, particularly the pace of technological developments in the generative AI world, is so fast that traditional research methods are not going to be able to keep up. It’s not that there won’t be studies — I’m sure there are already some underway, and there’s tiny, emerging studies that I have seen here and there. But we just don’t have the capabilities as a research enterprise to be doing things the traditional way. A really important question that needs to be grappled with, as a matter of policy, potentially as a matter of philanthropy and just as a matter of society, is: So, what then? Do we just do it and hope for the best? Because that may be what ends up happening.

As we’ve seen with and in schools, there can be real impacts that you don’t realize until five, 10 years down the road. Then you go back and say, “Well, I wish we’d been thinking about that in advance rather than just rolling the dice and seeing where it came up.” We don’t do that in other realms of life. We don’t let people just come up with medicines that they think will cure certain diseases and then just say, “Well, we’ll see. We’ll introduce it into broader society and let’s figure it out.” I’m not necessarily saying that we need the equivalent per se, but something that would give us better insight and real-time information to help us figure out the overall positives and not-so-positives seems to me a real challenge that is underappreciated at the moment.