Key points
- AI’s confident answer may hide the rare condition it barely considered.
- LLMs are built to narrow toward the typical, common answer.
- Seeking the atypical clinical insight may add depth to the model.
Medical students learn one of medicine’s oldest lessons almost as soon as they step onto their clinical rotations. It’s even become part of folklore and sound advice. When you hear hoofbeats, think horses, not zebras
Most patients have common diseases, and good clinicians learn to recognize familiar patterns before chasing unlikely ones
But the counterpoint is also important. Medicine has always lived with a sort of tension between probability and possibility. Every experienced physician remembers a patient whose diagnosis arrived late, when everyone had accepted the first reasonable explanation and the zebra never got invited into the room. Interestingly, artificial intelligence may be creating a version of that same moment
One Question, One Answer
More physicians are turning to large language models to support their practice—from data organization to clinical thinking. Most of those interactions follow the same path. A question goes in, and an answer comes back. The conversation often ends there—except for some conversational iteration, the model had said everything it had to say. But I’m not sure that assumption stands up to a little clinical and computational scrutiny. And I realize this can be a bit esoteric, but can also be important.
Large language models don’t retrieve facts the way a textbook index does. One interesting variable behind how they generate a response is called temperature. Despite the name, it has nothing to do with medicine. It doesn’t determine how deeply a model reasons. It determines how predictable its next word is. At low temperatures, the model sticks close to the most expected phrasing. At higher temperatures, it becomes willing to say something less obvious
That’s a modest and almost mechanical fact about language generation. But it produces an interesting side effect worth looking at. An LLM running hot will sometimes name a diagnosis that a model running cold never mentions at all. I don’t think that means the model is reasoning more broadly. I think the model isn’t reasoning more. It’s just less guarded about what it says
Most models default to a temperature near 1.0. That’s the “factory setting” if you never touch the dial. ChatGPT’s consumer interface runs cooler than that, closer to 0.7, which nudges it toward more typical, structured answers. Tools built for clinical or factual work tend to run cooler still, often well under 0.5, on the theory that consistency matters more than range. Nobody, as far as I know, is deliberately running a model hot for a second opinion that might conceive a deliberately wider variation and find a zebra.
Three Sweeps
That distinction is what got me thinking that maybe a patient deserves more than one pass at an AI’s attention. Picture asking the same clinical question three times, and letting the model answer a little differently each time, on purpose
A cold sweep stays close to consensus. Common diseases, familiar presentations, the cases a clinician sees every week. For most patients, that’s exactly where the answer should end. The cost is equally clear, as the farther a disease lies from that center, the less likely a cold, careful model is to say it out loud
A standard sweep loosens the grip modestly. Familiar diagnoses still lead, but less common possibilities start to compete for a mention. This is probably close to what most physicians already experience without realizing it, since few clinicians are thinking about temperature when they type a question into an LLM
A hot sweep pushes further. Here the model may name uncommon diseases or even odd constellations of symptoms. Some of it will be noise. Some of it will cost a clinician added time for nothing. But every so often, this is where the diagnosis nobody was looking for gets said out loud, perhaps even for the first time
Artificial Intelligence Essential Reads
Humanity’s Most Dangerous Delusion About AI Is Us
AI and the Two Forms of Authorship
Whose Job Is This, Really?
Here’s the potential problem. None of this should require a physician to become a temperature engineer, running the same question through a model three separate times and then comparing notes. That’s a burden nobody asked for, and it’s exactly the kind of thing that gets a good idea dismissed as clinically impractical. The burden belongs to the system
A differential diagnosis tool doesn’t need to hide its range behind one confident answer. It could surface all three tiers natively, in a single pass. The system provides common diagnoses up top, a middle tier of reasonable alternatives, and a labeled set of lower-probability possibilities the model considered but didn’t lead with. That range should come from how the system is built
That’s a design question more than a clinical one. Most of these tools are built to converge on the single most probable answer. And that’s the same way a search engine converges on its top result. And it’s important to recognize that while convergence feels like confidence, it can be far from complete. And somewhere in that gap is the zebra that goes unseen
What Physicians Already Do
In many ways, physicians already build this range-checking into their thinking. A difficult case gets discussed with a colleague or presented at grand rounds. The patient’s story rarely changes overnight, but what changes is the vantage point. It may be worth asking whether AI systems should be built to offer that same range on their own, rather than waiting for someone to think to ask twice
I want to be careful here; this is a design proposition and not a clinical recommendation. Temperature is a clue I noticed, not a mechanism I fully understand. I don’t know of any study showing that varying it, or structuring outputs around it, improves diagnostic accuracy or shortens the path to a rare diagnosis. As far as I can tell, this exact clinical interface hasn’t been built
Medicine has never trusted a single vantage point on a hard problem. Patients and clinicians ask for a second opinion, or perhaps a more eclectic perspective. Maybe the tools we’re building should carry that same technological instinct
Occam was right. The first answer is usually the most probable one. But today, medicine still needs to know what often disappeared beneath the clinical logic of the moment
