Is the Oral Exam the Answer to AI?

The question 0:01

If a student can explain their work out loud, under follow-up questions, does that prove they did the thinking, and can a university afford to find out for every student?

This is WAC Signal, from the World Assessment Council. Evidence and ideas for the changing world of learning.

Episode eight. Is the oral exam the answer to AI?

The signal 0:32

In March 2026 the Associated Press reported that a growing number of US college professors were turning to oral exams. Panos Ipeirotis, a professor at New York University's Stern School of Business, gave the reason, quote. I don't trust written assignments anymore to be the result of actual thinking.

Supervised exams are back too. In May 2026 Princeton's faculty voted to proctor all in-person exams, altering a one hundred and thirty-three year old honor system.

And one class shows why. In July, Ars Technica reported that a Brown University economist's class had averaged ninety-six out of one hundred on a take-home midterm, and forty-eight, among those who sat it, on the in-person final that followed.

So the live conversation is back. The question is whether it works as assessment, what it costs, and whether it can be run for hundreds of students.

The evidence 1:43

Does an oral measure something real? The best-designed comparison is small and predates generative AI. Mark Huxham and colleagues randomly divided ninety-nine biology students into two groups that answered the same questions, either orally or in writing. The oral group scored significantly higher. But higher marks are not proof of better measurement. An examiner can rephrase a question or offer a prompt, so oral marks cannot be read on the same scale as written ones.

There is a more encouraging signal from the University of California San Diego. Across ten engineering classes with nine hundred and twenty-five students, scores on low-stakes midterm orals predicted final written exam scores over and above the written midterm. That is an observational result from one university.

Then consistency. The strongest measurement evidence comes from medicine. Researchers analyzed the structured oral component of the membership exam of the UK Royal College of General Practitioners, covering eight hundred and ninety-six candidates, each of whom took two twenty-minute orals with two different pairs of examiners. Only forty-one percent of the variation in scores reflected differences between candidates. Thirty-two percent came from the content of the orals, and twenty-seven percent from examiners and general error.

Reliability from one case to the next was zero point six five, below the authors' benchmark of zero point eight. Their analysis suggested that four twenty-minute orals with paired examiners, or five with a single examiner each, would lift it to zero point seven eight. Our reading is that the number of separate questions and examiners a student meets matters more than seating two examiners in one room.

Then fairness. An oral cannot be marked blind. In a study of a French entrance exam, Thomas Breda and Son Thierry Ly compared candidates' anonymous written scores with their oral scores. Evaluation favored women in male-dominated subjects such as mathematics, and men in female-dominated subjects such as literature.

Delivery also leaks into marks. When one hundred and fifty-eight college instructors were each shown one of four randomly assigned videos of a student presentation, a presenter with high communication competence was rated more positively whether or not the student stuttered. The UK Quality Assurance Agency notes that orals are challenging for students with speech or hearing disabilities, and that students with different accents must be treated equitably.

And anxiety. San Diego's three-year project put four thousand and twenty engineering students in forty-six classes through orals. Before their first one, fifty-six point one percent of survey respondents expected excessive stress. Afterwards, twenty-four point one percent said the orals had caused excessive stress, against sixty-two point three percent for written exams.

Two cautions. Fewer students answered the later surveys, sixty percent after an exam and thirty-six percent at the end of term, against seventy-two percent for the first. And in the course designs the team described in 2022, orals counted for only two to fifteen percent of the course grade, so lower stress may partly reflect lower stakes.

The tension 5:45

The big problem is cost. Dublin City University advises allowing about fifteen minutes per student for an interactive oral, ten for the conversation and five to mark it. By our calculation, that is twenty-five examiner hours for every hundred students, and one hundred and seventy-five hours for seven hundred, before any preparation.

Yet some institutions have made it work. At George Washington University, three staff examined fifty-eight students in ten half-hour sessions over two days, by seating five or six at a time. In a San Diego course of about three hundred, orals were offered only to the twenty-six percent who had failed a first written test.

Which brings in the obvious shortcut. Can an AI run the exam? At NYU Stern, a voice AI agent examined two cohorts, seventy-three students in all. Grading cost under one dollar per exam, on top of a ninety-nine-dollar monthly voice subscription. But in the first cohort the agent packed several questions into thirty-one percent of its question turns, interrupted students who paused to think, and spoke in a cloned professor's voice that students found harsh.

After fixes, stacked questions fell to eleven percent, and the share of surveyed students who found the exam more stressful than a written one fell from eighty-three to sixty-three percent. But agreement among the grading models is reliability, not validity, and the authors say so. They did not have experts re-grade blind, did not analyze scores by student group, and acknowledge that a student coached by a second AI through a headset could not be reliably detected.

And the integrity claims for human orals are thinner than they sound. They rest largely on the absence of reported cases. The paper found no study testing how well examiners detect a student who has rehearsed AI-written answers.

The implication 8:05

An oral exam is not a cheap fix for AI. It moves risk from students who cheat to students who are anxious, less fluent in the language of instruction, or disabled. And an institution that mandates orals without funding examiner hours will get short, unstructured conversations, exactly the kind the reliability research warns against.

But it is also worth remembering what the oral is up against. Orals are usually judged against an idealized written exam. In many courses the real alternative is unsupervised written work, and the Brown class, ninety-six at home and forty-eight in the room, shows how little that can certify.

The WAC view 8:55

In our white paper, The Return of the Oral Exam, we propose a Live Check Ladder with four rungs, and one principle. Use the lowest rung that certifies the claim.

Rung one is a spot check. A three-to-five-minute micro-viva on submitted work, when stakes are low and authorship is the question. Rung two is a defense, a ten-minute conversation about a major piece of open work. Rung three is a structured oral exam of twenty to twenty-five minutes that replaces a written exam. And rung four is a panel viva, several orals with different examiners, for when progression or an award depends on the result.

Three tests decide whether to climb. The claim test. Name what a live check certifies that supervised writing cannot. If the answer is nothing, use in-class writing, which costs less. The stakes test. A single short oral samples too little to carry a high-stakes decision, so if progression or an award depends on the result, move to rung four or combine the oral with other secure evidence. And the capacity test. Count examiner hours before announcing the format, and if the hours do not exist, target or sample, or pay for the oral by ceasing to mark something else.

Whatever the rung, we recommend questions drawn at random from a bank, a rubric that marks reasoning, not fluency, a recording or a second examiner, a practice oral that does not count, alternative routes published in advance, and results compared every term by language background, disability, gender and examiner.

And for AI examiners, before their results count toward awards, we would want to see agreement with blind human marking and results by student group, including second-language speakers.

What to watch 11:04

One. Whether anyone runs the comparative trial that is missing, randomizing courses to oral, supervised written and open assessment, and following learning, fairness and cost together.

Two. Whether quality agencies and regulators recognize structured, recorded orals as a secure format, and set expectations for validating AI-conducted exams.

Three. Who carries the cost. Watch whether universities fund examiner hours, or simply mandate orals and hope.

The sources for every figure in this episode are listed on the episode page, along with the full white paper, The Return of the Oral Exam, at worldassessmentcouncil.org.

WAC Signal is produced by the World Assessment Council, with AI-assisted research and AI voices, and every claim is checked against its source before publication.

This has been WAC Signal. Evidence and ideas for the changing world of learning.