Artificial intelligence is changing medicine, and doctors are using algorithms to predict everything from sepsis to falls to death. Often embedded directly into electronic health records, these predictions are easy to incorporate into care — and their performance is often taken for granted.
But James Deardorff, a geriatrician and assistant professor in the division of geriatrics at the University of California San Francisco, warns that accuracy alone doesn’t guarantee a good outcome. He has built several models aimed at predicting outcomes for older adults, from mortality to the need for nursing home care, and he says clinicians need to pay attention to both the numbers behind the model and how they are used.
This month, Deardorff wrote a commentary on a large analysis of Epic’s proprietary end-of-life prediction model, published in JAMA Network Open. His point is simple: a model can contribute to a poor outcome even if it’s accurate.
How the Model Works
Epic’s model is proprietary, which means its workings aren’t publicly available. Deardorff’s commentary focuses on how the model’s predictions get used, not on how it was built.
The model generates a one-year mortality prediction for individual patients. That number can trigger a conversation about care goals. It can also influence decisions about transplant priority, where the stakes are much higher.
Deardorff’s argument is that the difference between those two uses is enormous. A conversation about care goals has few downsides. A decision about transplant priority could have profound impacts.
Accuracy vs. Outcomes
The problem isn’t that the model is wrong. The problem is that being right doesn’t protect you from harm.
Deardorff’s commentary highlights how a model’s accuracy can mask serious risks when the stakes are high. A patient’s one-year mortality prediction might be accurate, but using it to decide who gets a transplant is a very different matter. The model didn’t misread the patient; the harm came from what the clinician did with the information.
This is a subtle but important distinction. Most models are evaluated on their performance, measured against some benchmark. That tells you how well the model predicts, not how well its predictions are used.
Deardorff wants clinicians to think about both. He wants them to know the numbers, but he also wants them to know what those numbers mean when they land.
The Commentary’s Core Argument
Deardorff’s co-author made the point alongside him. The pair argued that a mortality prediction used to prompt an open-ended conversation about care goals has few downsides. The same prediction used to inform a transplant priority decision could have profound impacts.
The contrast is stark. One use is low-risk. The other is life-changing.
Deardorff’s position is that clinicians need to be aware of both the performance of an algorithm — including in subgroups like older patients — and how to responsibly use its output. That’s a full plate of responsibility, and it asks more of doctors than simply trusting a number.
Why Geriatrics Matters
Deardorff’s work sits squarely in the field of geriatrics, building models for mortality and nursing home care.
His commentary on the Epic model extends that focus. He is looking at how a widely used tool performs in a population that has historically been underserved by medical research.
The JAMA Publication
The analysis of Epic’s model was published in JAMA Network Open, a peer-reviewed journal. That gives Deardorff’s commentary a solid foundation. He is not criticizing a model he built himself; he is responding to a published study of someone else’s work.
The commentary itself is available to STAT+ subscribers. It is a reminder that the academic literature is where these questions get worked through, and that clinicians should be reading it.
The Limits of Accuracy
Deardorff’s point is that accuracy is necessary but not sufficient. A model can be accurate and still produce bad outcomes if its predictions are used irresponsibly.
That is a hard lesson for anyone who thinks technology solves problems automatically. A model is a tool, and tools can be used badly. The model doesn’t make the decision; the person holding it does.
What Clinicians Should Do
Deardorff’s advice is straightforward. Know the numbers, but don’t stop there. Understand what the model is telling you and what you are doing with that information.
He also wants clinicians to consider subgroups. Older patients may respond differently to models trained on younger populations, and ignoring that difference can lead to worse outcomes.
The commentary is a call to responsibility. It is a reminder that a good prediction is not the same as a good decision.
The Broader Picture
AI is becoming standard equipment in medicine. Models are embedded in electronic health records, and they are making decisions that affect patients daily.
That is a shift, and it requires new habits of mind. Doctors need to treat model outputs as evidence, not commands. They need to understand what the model knows and what it doesn’t know. And they need to keep asking whether a prediction improves care or just replaces it with a faster version of the same mistake.
Deardorff’s work is part of that broader conversation. He is a practitioner who builds models, which gives his warnings weight. He knows what the tools can do, and he knows what they can’t.
Key Points From Deardorff’s Commentary
- A model can contribute to a poor outcome even if it is accurate.
- Using a one-year mortality prediction to prompt a care goals conversation has few downsides.
- Using the same prediction to inform a transplant priority decision could have profound impacts.
- Clinicians need to know the performance of an algorithm, including in subgroups like older patients.
- Clinicians also need to know how to responsibly use the model’s output.
This article is adapted from coverage of STAT’s report, which is gated content. The full commentary is available to STAT+ subscribers, and the JAMA Network Open analysis it responds to is also behind a paywall.
Source material: “STAT+: A geriatrician explains why AI for older adults deserves careful scrutiny,” STAT.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.