Ask Rupert
When people hear "AI" in medicine, they picture something that understands. What most deployed systems actually do is far narrower: statistical pattern matching.
A convolutional neural network trained on millions of labeled scans doesn't understand anatomy. It learned that certain pixel patterns correlate with "nodule" or "fracture," and it flags them. That's closer to a very sophisticated anomaly detector than to anything that reasons about what it's seeing. The radiologist still supplies the meaning.
A simple test separates the two: if the system can't explain why it flagged something, it's not reasoning — it's matching. That single distinction is worth making publicly, because "AI" borrows credibility from science fiction while the actual product is statistical pattern-matching with a narrow lane. The marketing outruns the capability, and patients and the public deserve to know the difference before they trust any of it with their own scans.
This isn't a knock on the technology. Anomaly detection at this scale is genuinely useful — it catches things tired eyes miss at two in the morning, and consistency across thousands of reads beats any single human radiologist on fatigue. But usefulness and understanding are different claims, and conflating them has consequences. A clinician who treats a flagged region as "the AI saw something" without interrogating what it saw is practicing a different kind of medicine than one who asks, "what pattern triggered this, and does it fit this patient?"
The failure modes are instructive. These systems are brittle outside their training distribution. A chest X-ray AI trained on hospital equipment may misread images from a portable unit in a rural clinic. A mammography model trained on dense-breast populations may underperform on fatty tissue. The error doesn't announce itself — there's no confidence interval attached to the flag, no footnote saying "this was trained on European data." The output looks identical whether the underlying evidence is strong or thin.
Regulators reinforce the confusion. The FDA clears these tools as medical devices, which quietly reinforces the "AI doctor" framing even when the underlying technology is pattern matching. The clearance process evaluates sensitivity and specificity — does it catch what it's trained to catch — not whether it understands. That's the right question for a detector. It's the wrong question for something marketed as intelligence.
Naming it honestly — "software trained to recognize patterns, not a doctor in a box" — lets people value the tool without overestimating it. Precision in language is the first form of patient safety. And it's the foundation for everything that follows: you can't have an honest conversation about what AI should do in medicine until you've been honest about what it is.