Hospitals Are Deploying AI Like Crazy. Nobody Knows If It Actually Helps Patients.

I don’t need to tell you that AI is everywhere. You’ve seen the headlines, the vendor pitches, the breathless conference talks. And you know it’s showing up in hospitals too.

Doctors are using AI to take notes during appointments. Algorithms are scanning patient records to flag people who might need extra support. Other tools are reading chest x-rays and lab results. The list keeps growing.

A pile of studies show these tools can be accurate — at least in controlled settings. But accuracy isn’t the same as usefulness. The real question is whether using them actually leads to healthier patients. And we don’t have a good answer.

That’s the argument Jenna Wiens, a computer scientist at the University of Michigan, and Anna Goldenberg of the University of Toronto make in a paper published this week in Nature Medicine. Wiens has spent over a decade trying to get clinicians interested in AI. For years it was an uphill battle. Then, she says, “a switch flipped.” Suddenly health-care providers are not just interested — they’re deploying these tools at scale.

The problem is they’re not rigorously checking whether the tools work in practice.

Take “ambient AI” scribes. These tools listen to doctor-patient conversations, then transcribe and summarize them. Multiple vendors offer them, and adoption has been fast. A few months ago, a staffer at a major New York medical center told me doctors are “overjoyed” — the tech lets them focus on the patient during appointments instead of typing notes. Early studies back that up, showing reduced burnout.

Great. But what about patient health? “Researchers have evaluated provider and clinician or patient satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens told me. “We just don’t know.”

That uncertainty extends beyond scribes. Predictive tools that forecast a patient’s trajectory, algorithms that recommend treatments — they’re all designed to make care more efficient. But even a tool that’s technically accurate might not improve outcomes. An AI might speed up chest x-ray interpretation, sure. But how much does the doctor rely on that analysis? Does it change how they interact with the patient? Does it shift treatment decisions? And what does that mean for the person lying on the hospital bed?

The answers probably vary by hospital, department, and even individual doctor. A seasoned physician might use an AI scribe differently than a resident still learning how to build a patient history. Wiens points to research on AI in education, which suggests these tools can change how people process information. Could the same happen in medicine? “We like things that save us time,” she says, “but we have to think about the unintended consequences.”

A study published in January 2025 by Paige Nong at the University of Minnesota found that about 65% of US hospitals use AI-assisted predictive tools. Of those, only two-thirds evaluated the tool’s accuracy. Even fewer checked for bias. And that was over a year ago — the number of hospitals using AI has likely grown since.

Wiens isn’t calling for a moratorium. She believes AI has real potential to improve care. What she wants is evidence — actual measurement of how these tools affect patients in real-world settings. “I have to believe that in the future it’s not all AI or no AI,” she says. “It’s somewhere in between.”

That middle ground requires something we’re not doing much of right now: asking hard questions before, not after, we roll out the technology.