In 2026, the FDA rewrote its clinical decision support guidance, the AMA's House of Delegates passed policy demanding "meaningful physician oversight" of every clinical AI tool, and five states now require disclosure or human review of AI-generated care. This isn't background noise — it's the new baseline for how any AI medical scribe has to be built. Here's what "clinician in the loop" actually means in 2026, why the term itself has come under fire, and how CRUSH, S10.AI's ambient AI medical scribe, is architected so the note gets faster while the clinical decision stays exactly where it belongs — with you.
A regulatory reckoning, not a marketing tagline
For the last few years, "human-in-the-loop" has functioned as a reassuring footnote — the line every AI vendor added to a slide deck and moved past. That changed in 2026.
On January 6, 2026, the FDA issued a rewritten final guidance on Clinical Decision Support Software, reissued January 29, that goes well beyond the 2022 version it replaces. The new guidance spends real space on automation bias — the well-documented tendency of busy clinicians to over-trust a confident-sounding recommendation — and tightens what it means for a clinician to "independently review the basis" of an AI output rather than simply defer to it. Software that can't show its logic, its inputs, or its limitations in plain language now carries real risk of being classified as a regulated medical device.
Five months later, the AMA's House of Delegates went further. Meeting at its 2026 Annual Meeting, delegates adopted policy requiring that AI integrated into clinical practice be evidence-based and subject to "meaningful physician oversight," explicitly opposing the use of autonomous or semi-autonomous AI as a substitute for physician review — even in back-office functions like coverage determinations. AMA CEO John Whyte, MD, put it plainly in the policy's rollout: augmented intelligence has "enormous potential in healthcare," but it "cannot replace physician judgment."
At the same time, ECRI — the patient-safety organization that has published an independent hazards report since 2008 — named misuse of AI chatbots and large language models the single largest health technology hazard for 2026, ahead of cybersecurity failures and medication mix-ups. And state legislatures didn't wait for Washington: Texas's TRAIGA now requires that a licensed practitioner review every AI-generated clinical record before it's relied on. Utah requires hospitals to disclose AI use directly to patients. California's AB 3030 requires disclosure whenever a patient-facing clinical communication was generated or substantially drafted by AI. Colorado and Maryland have gone further still, requiring a licensed clinician to sign off on AI-influenced coverage and medical-necessity decisions.
The pattern across every one of these developments is the same: regulators, medical associations, and legislatures are no longer taking "clinician in the loop" on faith. They're defining it, in statute and guidance, as a specific set of design requirements — and vendors are going to be held to them.
That's the environment CRUSH was built for. Not because "human-in-the-loop" is a good marketing phrase, but because it's fast becoming the law.
What "clinician in the loop" actually means — and where the phrase falls short
"Clinician in the loop" gets used to mean almost anything: a checkbox before a note saves, a radiologist glancing at an AI-flagged scan, a physician who technically has the power to override a recommendation they never had time to actually evaluate. That vagueness is a real problem, and it's worth taking seriously rather than waving away.
A widely discussed 2026 analysis from Hardian Health made the sharpest version of this critique: putting a "clinician in the loop" can function less as a safeguard and more as a way for manufacturers to shift liability onto the person doing the reviewing, without giving them a real, unbiased opportunity to catch an error. The concerns are specific — a confident AI suggestion can bias the reviewer's own judgment (a form of automation bias), a time-pressured clinician has every incentive to accept rather than re-derive an answer from scratch, and the vendor rarely specifies whether the clinician is supposed to independently evaluate the case or merely check the AI's work.
That critique is correct, and it's exactly why the design of the review step matters more than the existence of one. A note that's technically "reviewed" in the two seconds it takes to tap "accept" isn't a safeguard; it's theater. The FDA's 2026 guidance makes essentially the same point when it insists that a clinician must be able to independently review the basisof a recommendation, not just the recommendation itself — and that automation bias has to be actively designed against, not assumed away.
So the honest question isn't "does S10.AI have a human in the loop?" Every credible vendor in this category can say yes to that. The real questions are: what, specifically, is the clinician being asked to review; how much of their own independent judgment does that review actually require; and what does the AI never do without them?
Two different jobs, two different risk profiles
Part of what makes this debate confusing is that "healthcare AI" covers two very different categories of tool with very different liability exposure, and conflating them muddies the conversation.
Decision-recommending AI — diagnostic support tools, prior-authorization engines, treatment-recommendation software — proposes a course of action. This is where most of the liability research concentrates. A 2021 study in the Journal of Nuclear Medicine found that physicians who accepted an AI's recommendation for standard-of-care treatment reduced their liability exposure with mock jurors, but got no similar protection for rejecting a recommendation of nonstandard care — a structural incentive to defer to the machine even when independent judgment says otherwise. A 2026 legal analysis described this as "doctrinal collapse": AI blurring the traditional legal lines between malpractice negligence, vicarious liability, and product liability once diagnostic reasoning is co-produced by physician and algorithm. A 2024 paper in Frontiers Health Journal warned that clinicians risk becoming "liability sinks" — absorbing legal and moral responsibility for systems whose actual decision logic they can't fully inspect.
Documentation AI — ambient medical scribes like CRUSH — does a fundamentally different job. It doesn't recommend a diagnosis, propose a treatment, or suggest an order. It listens to an encounter that has already happened and structures what was already said and decided into a note. That's a narrower, lower-bias task, but it isn't a risk-free one: a 2025 mixed-methods study out of UC Berkeley and UCSF, analyzing more than 50,000 real ambulatory encounters with an AI scribe in production, found that when clinicians did flag safety-relevant problems, medication name, dosage, and titration errors were the single largest category (18.5% of flagged issues), followed by fabricated or misattributed diagnoses, exam findings, and history details showing up in the History of Present Illness and Assessment & Plan sections.
That study is worth sitting with, because it's the strongest evidence available that ambient documentation carries a real accuracy tax, not a hypothetical one — and it's precisely why the quality of the human review step matters more than its mere existence.
CRUSH is built for the second category, and it's engineered around that specific risk: not "will the AI make a bad clinical call" (it doesn't make clinical calls), but "will the note it drafts accurately reflect what happened in the room."
What the law now requires, state by state
If you deploy an AI scribe or clinical AI tool in 2026, "we have a human in the loop" is no longer a design choice you get to make unilaterally — in a growing number of states, it's a compliance requirement with teeth:
- Texas (TRAIGA, effective January 1, 2026): a licensed practitioner must review every AI-generated clinical record before relying on it, and the law separately bars offshoring electronic medical records.
- Utah (HB 452 and the amended AI Policy Act): hospitals must disclose AI use in patient care directly to patients; suppliers of AI-driven mental health tools must disclose that a user is talking to a machine, not a person.
- California (AB 3030): health systems must disclose when a patient-facing clinical communication was generated or substantially drafted by AI, with a path for the patient to reach a human.
- Colorado (HB 26-1139, HB 26-1195, 2026): AI-assisted coverage decisions must account for the patient's individual clinical history, a licensed clinician must review medical-necessity denials, and AI-delivered psychotherapy is barred from payer reimbursement outright.
- Maryland (HB 820, effective April 2026): adopts CMS-style guardrails requiring licensed review of AI-influenced coverage decisions.
- Arizona (Board of Behavioral Health Examiners, November 2025): behavioral health professionals must obtain and document informed consent before using ambient listening or AI recording tools with a patient.
- Illinois: flatly prohibits AI from making independent therapeutic decisions — arguably the clearest bright-line rule in the country.
Some of these apply directly to documentation tools (Texas, Arizona, California); others govern adjacent AI use like coverage determinations and chatbots. Taken together, though, they describe a single emerging standard: the clinician has to see, and be able to independently evaluate, whatever the AI produced — before it becomes part of the record or the care decision. That standard isn't unique to any one state, and it isn't going away; if anything, the 2026 legislative wave (Colorado, Maryland, and a wave of "fast follower" states adopting similar language) suggests it's converging into something close to a national floor.
How CRUSH is actually built around this standard
The design principle behind CRUSH is simple to state and easy to check against the workflow: the AI structures what happened in the encounter; the clinician decides everything about the patient.
The draft is reviewed while the encounter is still fresh, not batched for later. CRUSH generates a structured note — HPI, exam findings, assessment, plan — within seconds of the encounter ending, while the clinician is still in the room or immediately after. That matters for exactly the reason the FDA's automation-bias language flags: review quality degrades the further removed it is from the original context. A note reviewed against a fresh memory of the actual conversation is a materially different check than one skimmed hours later from a queue, and it's the difference between the kind of "genuine independent review" regulators are now asking for and the rubber-stamp version critics like Hardian Health are right to worry about.
CRUSH never originates a diagnosis, an order, or a prescription. It maps the conversation against a medical knowledge graph to structure clinical language correctly — placing a stated symptom in the HPI, a negative finding in past medical history, a medication change in the plan — but it does not generate a new clinical recommendation for the physician to accept or reject. That keeps CRUSH out of the highest-liability category described above (decision-recommending AI) entirely. The clinician isn't being asked whether to trust an AI's medical opinion; they're being asked whether the note accurately reflects an encounter they just conducted.
SHINE flags; it doesn't decide. SHINE audits the finished chart for documentation quality, coding and compliance gaps, and medical-legal exposure, and cross-checks decisions against current clinical guidelines and literature — but it surfaces those flags for a human to act on rather than silently editing the chart or auto-submitting a code. That's a meaningful distinction under the FDA's non-device CDS criteria, which draw a hard line between software that displays information for a clinician to independently evaluate and software that drives the decision itself.
BRAVO routes, rather than resolves, anything requiring clinical judgment. When a call, message, or after-visit question touches something that requires a licensed professional's judgment — a symptom that needs triage, a medication question, anything outside routine scheduling or reminders — BRAVO hands it to staff with full context attached instead of attempting to answer it. The AI's job is narrowing the funnel of what reaches a human, not shrinking the number of things a human actually decides.
Every note carries an editable, auditable trail. Because state and federal guidance increasingly expect an attestable, reviewable record — not a black box — CRUSH preserves what was AI-drafted, what was clinician-edited, and when, so a practice can demonstrate exactly what a document from FDA's guidance calls "independent review of the basis" actually looked like, if it's ever asked to.
Speed is part of this story, but it's worth being precise about what the speed actually buys. A faster, more accurate first draft reduces the administrative burden of documentation — the retyping, reformatting, and hunting for what got missed that turns "pajama time" into two or three extra hours a night. It does not, and isn't meant to, reduce the clinician's role as the final decision-maker on the patient in front of them. Those are two different problems, and collapsing them is exactly the kind of ambiguity the FDA, the AMA, and critics of shallow "clinician in the loop" claims are all, from different directions, pushing the industry to stop getting away with.
What real oversight looks like versus what's cosmetic
Given the Hardian Health critique above, it's fair to ask: how would a practice actually tell the difference between meaningful review and a rubber stamp? A few concrete markers, drawn from AMA governance guidance and the operational practices emerging across the ambient-scribe category in 2026:
- A specific attestation, not a generic "accept" button. The clinician should be affirming that they reviewed the specific content of this note, not clicking through a blanket disclaimer.
- High-risk content routed for mandatory manual verification — anticoagulant dosing, procedural risk discussions, anything tied to informed consent — rather than treated identically to routine documentation.
- Retained edit history and timestamps, so a practice (or a plaintiff's attorney, or a state regulator) can reconstruct exactly what was reviewed and changed.
- Informed consent for ambient recording, documented before the encounter, not assumed.
- Periodic, structured audits of AI-generated notes against the source encounter — not just relying on whether clinicians happen to submit feedback, which research on AI scribe safety signals suggests significantly understates the true error rate, since most clinicians who catch and fix an error never report it.
- Clear, written policy on what the AI is never permitted to do autonomously — no auto-submitted diagnosis codes, no auto-sent patient communications without a review step, no self-resolving triage.
None of this is exotic. It's the same governance discipline the AMA's own STEPS Forward toolkit and ECRI's 2026 recommendations for AI in diagnosis both describe: define the tool's role narrowly, make the review step specific and verifiable, and never let convenience quietly expand what the AI is trusted to decide on its own.
What this means for your liability exposure
For decision-recommending AI, the liability research is genuinely unsettled — jurors in the Tobia et al. study rewarded physicians for accepting standard-care AI advice but not for correctly rejecting nonstandard advice, and legal scholars describe malpractice, vicarious liability, and product liability doctrines struggling to divide responsibility when a diagnosis is co-produced by physician and algorithm.
For documentation tools used the way they're designed to be used, the exposure looks different — and in most respects, more familiar. The clinician's signature still means what it has always meant: this note accurately reflects the encounter, and I stand behind the medical decisions in it. An AI-drafted note doesn't change that standard; it's still the physician's attestation that creates the legal and clinical record. What does change is the practical burden of meeting that standard well — a faster, more complete first draft, reviewed while the encounter is still fresh, is easier to genuinely verify than a rushed one written from memory at 9 p.m. That's the actual liability case for a well-built ambient scribe: not that it removes your responsibility, but that it gives you a better starting point for discharging it.
The bottom line
The industry spent the early 2020s treating "human-in-the-loop" as a box to check. 2026 is the year that stopped being enough — for regulators, for the AMA, and for the researchers documenting what happens when review is real versus cosmetic. CRUSH is built on the harder version of that standard: an AI that drafts faster and more accurately so the clinician's review is worth something, that never originates a diagnosis, order, or prescription, and that keeps every clinical decision exactly where the law, the evidence, and plain professional responsibility all say it belongs — with the clinician who signs the chart.
FAQs
Does S10.AI's AI ever make a diagnosis or treatment decision? No. CRUSH structures and documents what was said and decided during an encounter; it does not generate a new diagnosis, order, or treatment recommendation for a clinician to accept or reject. Every clinical decision remains the physician's.
Is a clinician legally required to review AI-generated notes before they're finalized? Increasingly, yes, depending on your state. Texas's TRAIGA requires a licensed practitioner to review every AI-generated clinical record before relying on it, and several other states now impose disclosure or review requirements for AI used in patient care or coverage decisions.
What's the difference between AI clinical decision support and an AI medical scribe? Clinical decision support tools propose a recommendation — a diagnosis, a treatment option, a prior-authorization decision — that a clinician evaluates. An AI medical scribe like CRUSH documents an encounter that has already happened. The liability and bias considerations for the two are genuinely different, and conflating them is one of the more common sources of confusion in this debate.
Can "clinician in the loop" become a rubber stamp instead of a real safeguard? Yes, and that's a legitimate, well-documented concern — a quick "accept" click on a confident-sounding AI output is not meaningful review. What makes review real rather than cosmetic is design: point-of-care review while the encounter is fresh, specific attestation rather than a blanket disclaimer, mandatory manual verification for high-risk content, and a retained audit trail.
Does using an AI scribe increase my malpractice exposure? Used as documentation support, an AI scribe doesn't change what your signature has always meant — that the note accurately reflects the encounter and the decisions you made. The liability research on AI is far less settled for tools that recommend diagnoses or treatments, which is a different category of risk than ambient documentation.
What does the FDA's 2026 guidance actually require of AI documentation and decision-support tools? The FDA's January 2026 Clinical Decision Support Software guidance emphasizes that clinicians must be able to independently review the basis of any AI-generated recommendation — not simply defer to it — and calls out automation bias directly, requiring greater transparency about how a tool's outputs are generated.
Sources: FDA, Clinical Decision Support Software Guidance for Industry (Jan. 6 / Jan. 29, 2026); American Medical Association, 2026 Annual Meeting policy actions and 2026 Physician Survey on Augmented Intelligence; ECRI, Top 10 Health Technology Hazards for 2026; Dai, Huang, et al., "Patient Safety Risks from AI Scribes: Signals from End-User Feedback" (UC Berkeley / UCSF, 2025); Tobia, Nielsen & Stremitzer, "When Does Physician Use of AI Increase Liability?", Journal of Nuclear Medicine (2021); Hardian Health, "Clinician in the AI loop: a faulty solution to a thorny problem" (2026); Holland & Knight, "States Continue Efforts to Regulate AI in Healthcare" (2026); Olson et al., JAMA Network Open (2025).