Google’s AMIE Tried Real Clinic Duty: Here’s What Happened

Google’s been working on AMIE, their conversational medical AI, for a while now. We’ve seen it show off in simulated settings — handling diagnostic challenges, chatting with patient actors. But everyone knows the real test is whether it can function in an actual clinic without causing chaos.

That’s exactly what Google Research and Beth Israel Deaconess Medical Center (BIDMC) just did. They published results from a prospective, single-center feasibility study where AMIE took patient histories before real primary care appointments. This isn’t another demo. This is the first time AMIE has been let loose on actual patients in a clinical workflow, with all the safety oversight you’d expect.

How the study worked

The setup was straightforward but carefully controlled. Patients booked for new, non-emergency appointments — either in-person or telehealth — were invited to participate during booking. If they agreed, they’d interact with AMIE via a secure web link before their actual consultation.

Here’s the key safety measure: a physician was watching the entire AI-patient chat via live video with screen sharing. This “AI supervisor” could jump in at any moment based on predefined safety criteria. Think of it like a supervising doctor watching a trainee — except the trainee is an AI.

AMIE would generate a transcript and summary of the pre-visit interaction, which went to the patient’s actual doctor. The idea is straightforward: let the AI handle the tedious history-taking, freeing up the clinician for the more nuanced parts of the visit.

What they found

The study was designed to assess feasibility, not to prove AMIE is ready for prime time. And the results are… cautiously positive.

Patients generally found the interaction acceptable. The AI managed to collect relevant clinical information without major issues. But let’s be real — this was a feasibility study with a small sample size and tight supervision. It tells us more about whether the approach is worth pursuing than whether it’s actually better than current practice.

What’s interesting is that this follows the pattern we’ve seen in other AI-in-medicine studies. The technology works in controlled settings, but the real world throws curveballs. Patients don’t always answer clearly. They get confused. They say things that require human judgment to interpret.

The supervision question

The live physician oversight is both a strength and a limitation of this study. On one hand, it’s responsible — you don’t want an unsupervised AI messing up a patient’s history. On the other hand, if you need a doctor watching every interaction, you’re not really saving that much time.

Google argues this is analogous to how clinicians-in-training work under supervision. Fair point. But the goal here should be to eventually reduce the need for constant oversight, not to build a system that requires a human babysitter.

What this means for the field

This study is an important step, but it’s exactly that — a step. The paper is upfront about limitations: single center, small sample, highly controlled conditions. We’re still a long way from seeing AMIE deployed in a busy clinic without a physician hovering over its shoulder.

What I appreciate is that Google is actually doing the hard work of clinical validation rather than just releasing flashy demos. Too many AI health products skip this phase and then fail when real patients show up with real complications.

The next question is whether this approach scales. Can AMIE handle different languages, health literacy levels, and clinical specialties? Will patients trust an AI with their symptoms? Will doctors actually read the summaries, or will they just ignore them like they do with so many other clinical decision support tools?

This is higher risk, higher reward territory than most AI applications. Get it right, and you could dramatically improve access to care and reduce clinician burnout. Get it wrong, and you’re putting patients at risk. Studies like this are how we figure out which direction we’re heading.