Microsoft study finds AI outperforms doctors in diagnosis

A striking result, with a narrow meaning

Microsoft researchers have reported one of the strongest recent benchmark results for artificial intelligence in medicine: an AI system designed to reason through cases step by step diagnosed complex medical scenarios more accurately than participating physicians in a study published in Science on October 22, 2026 [1]. The system, called Diagnostic Orchestrator or DxO, reached 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases, while generalist physicians using search and reference tools scored 36.1% on the same type of cases [1].

The comparison was also extended to a second dataset of emergency department admissions. There, DxO reached 80.3% accuracy, compared with 46.8% for physicians, according to the study summaries published this week [1]. Those numbers explain why the announcement immediately drew attention: they suggest that AI systems may be moving beyond multiple-choice medical exams and toward a more realistic simulation of diagnostic work.

But the key phrase is “simulation.” This was not a clinical trial in which an AI system diagnosed real patients in a hospital and improved outcomes. It was a controlled, text-based benchmark in which both human clinicians and the AI system worked through documented cases under structured rules [1]. That distinction matters because diagnosis in the real world involves incomplete histories, anxious patients, physical examination, imaging, institutional workflows, legal accountability and follow-up care—none of which can be fully captured by a benchmark.

What Microsoft actually tested

The study’s central contribution is not just the AI score. It is the format of the test. Previous medical AI evaluations often handed a model a full written vignette and asked for an answer. Microsoft’s team instead used a benchmark called Sequential Diagnosis Benchmark, or SDBench, that turns diagnosis into an interactive process [2].

SDBench included 629 clinical cases: 304 NEJM clinicopathological conference cases and 325 emergency department-to-admission encounters from MIMIC-IV-ED [1]. Each case began with limited information. The diagnostician—human or AI—then had to decide what to ask, what tests to order, and when enough evidence had accumulated to commit to a diagnosis [1].

That design is important because good diagnosis is not only about knowing rare diseases. It is also about asking the right question at the right time, not ordering every possible test, and revising a differential diagnosis as evidence changes. The AAAS summary of the Science paper emphasized that conventional static benchmarks can overstate AI competence because they compress the diagnostic process into a single inference step [2].

DxO was built to mimic a panel of medical experts. It keeps track of possible diagnoses, challenges leading hypotheses, selects questions and tests, and considers cost while proceeding through a case [1]. In the main results, the system was powered by GPT-5 [1]. This is not a standalone “AI doctor” in the consumer-app sense; it is an orchestration system sitting on top of a large language model and forcing that model into a structured diagnostic workflow.

The cost question is more complicated than the headline

The headline result says AI was more accurate than doctors. The cost result is more nuanced. Without budget limits, DxO’s average testing cost per case was higher than the physicians’ on both sets of cases: $5,431 versus $4,725 for the NEJM cases, and $4,979 versus $3,400 for the emergency department cases [1].

However, when researchers imposed budget constraints, the AI system reduced costs while still outperforming the physician average. In the NEJM cases, budget-limited configurations reached 80.9% accuracy at $2,241 per case and 79.6% accuracy at $1,303 per case [1]. On emergency department cases, budget-limited versions scored 73.2% at $2,634 and 69.5% at $2,037, still above the physician average reported in the study [1].

This is one of the study’s most relevant findings for health systems. An AI that is merely more accurate by ordering more tests would be clinically and economically hard to justify. Microsoft’s result suggests that a structured AI workflow may be able to move along an accuracy-cost frontier, preserving much of its diagnostic performance while reducing unnecessary testing [2].

Still, the costs were estimated from standardized U.S. pricing rather than actual hospital billing, insurance contracts, patient discomfort, delays, radiation exposure or downstream consequences [1]. In other words, the benchmark measured a useful but simplified version of cost.

Why the AI may have done better

One notable difference was how the AI and physicians gathered information. Physicians asked an average of 5.6 questions before ordering their first test, while DxO asked 25.5 questions before ordering its first test [1]. The researchers described this as more extensive and structured history-taking [1].

That pattern points to a possible advantage of AI systems: they do not tire, rush or forget to ask a long list of relevant questions. Large language models also contain a broad range of medical associations and can apply them consistently across cases. The study authors suggested that breadth of knowledge and resistance to fatigue or cognitive bias may have contributed to the performance gap [1].

But there is another interpretation: the benchmark may reward exhaustive text-based questioning in a way that does not always translate to a real encounter. A physician in a busy emergency department cannot ask dozens of perfectly structured questions before moving forward. Patients may not know or remember the answer. Some clues are visual, tactile or contextual. DxO could not perform a physical exam or directly observe a patient, a limitation highlighted in the AAAS summary [2].

What makes this study different from the 2025 announcement

Microsoft’s diagnostic system first drew broad attention in 2025, when the company described MAI-DxO as outperforming physicians on difficult NEJM cases. The current Science publication, reported on October 22, 2026, is significant because it places the work in a peer-reviewed journal and adds a broader emergency department dataset to the discussion [3].

Japanese technology outlet AI Jitan, summarizing Microsoft’s October 22 announcement, noted that Microsoft described the new paper as covering both the SDBench framework and the MAI-DxO system, with partial open-source release of the benchmark infrastructure for some NEJM cases [3]. The same report also emphasized a key caveat: these figures are benchmark results, not evidence from live patient care [3].

That distinction should shape how hospitals, regulators and patients interpret the news. A peer-reviewed benchmark can be an important scientific milestone. It is not the same thing as regulatory clearance, hospital deployment or demonstrated improvement in mortality, safety, equity or patient experience.

Microsoft’s broader healthcare AI push

The study is also part of a larger strategic move by Microsoft AI. Current reporting around the announcement says Microsoft is pursuing “medical superintelligence,” a phrase used by Mustafa Suleyman, CEO of Microsoft AI, to describe systems that could eventually support more advanced clinical reasoning [4].

According to AI Jitan’s summary of Microsoft’s October 22 materials, the company also discussed work with Mayo Clinic and a planned prospective evaluation with Beth Israel Deaconess Medical Center in Boston, where patients would interact with the system and results would be compared with independent physician assessments [3]. If carried out, that kind of study would be far more informative than a retrospective benchmark because it would test how the tool behaves in the messy environment of actual care.

This is where the story moves from impressive numbers to implementation questions. Who is liable if the AI is wrong? How should a physician document disagreement with an AI recommendation? Will hospitals use the tool for second opinions, triage, education or patient-facing symptom assessment? How will regulators classify an AI system whose underlying language model may change over time? The Science benchmark does not answer those questions, but it makes them more urgent.

The limitations are not minor

The strongest version of the result is simple: on the tested cases, DxO outperformed individual generalist physicians. The responsible version adds several limits. The comparison was not against matched specialists or multidisciplinary teams [1]. NEJM clinicopathological conference cases are selected for teaching value and often represent unusually difficult diagnostic puzzles [1]. The benchmark was text-only and excluded imaging, waveforms and pathology slides [1].

The study also evaluated diagnosis, not treatment. It did not show that patients would receive better care, avoid harm or have improved outcomes if DxO were deployed [1]. Medical Economics also reported that all authors were Microsoft employees and held equity in the company, and that eight authors were named inventors on a Microsoft patent application related to the evaluation framework [1]. Those disclosures do not invalidate the findings, but they reinforce the need for independent replication.

A breakthrough, not a replacement plan

The Microsoft study is important because it tests AI in a more clinically realistic way than many earlier medical benchmarks. It asks whether an AI system can gather information, update a differential diagnosis, consider the price of tests and decide when to stop. On that test, the system did remarkably well [2].

Yet the practical conclusion is not that AI has “beaten doctors” in the full sense of medical care. The more defensible conclusion is that AI diagnostic orchestration may become a powerful clinical support tool, especially for difficult cases, second opinions and medical education. The next milestone will not be another headline about a higher benchmark score. It will be prospective evidence showing that doctors and AI together can improve patient outcomes safely, affordably and equitably.

For now, Microsoft has shown that a well-structured AI system can outperform generalist physicians in a controlled diagnostic benchmark. That is a major signal for healthcare. It is not the end of the doctor’s role; it is the beginning of a harder debate about how medicine should use machines that can reason impressively, but still cannot take responsibility for a patient.

Sources from the last 72 hours

1. [1]AI system outperforms physicians on diagnostic accuracy in Microsoft studyOct 22, 2026, 2:00 AM
2. [2]AI System Improves Diagnostic Accuracy, Cuts Testing CostsOct 22, 2026, 8:06 PM
3. [3]Microsoft AI、診断AI「MAI-DxO（マイ・ディーエックスオー）」の評価をScienceに発表NEJMの難症例で正答率83%、医師は22%と36%（同社測定）Oct 21, 2026, 5:00 PM
4. [4]Microsoft says AI tool diagnoses conditions up to four times more accurately than doctorsOct 22, 2026, 2:00 AM

AI-generated article based on recent web research, then preserved as a dated editorial snapshot.