Clinician editing of ambient AI drafts is not a final polish step; it is where clinical accuracy, specialty convention, billing justification, and medicolegal exposure are reasserted. Interviews with 30 clinicians at University of California, Irvine Health documented edits aimed at improving accuracy and specialty-specific precision, reducing liability risk, and meeting billing and coding requirements [1]. The practical question for health systems is therefore not whether drafts require editing, but how to structure review so that high-risk errors are caught while documentation time remains lower than manual writing. A useful workflow separates transcription verification from clinical judgment, standardizes the order of checks, and measures what the review sequence actually changes. Quantitative data show why this matters: in a pragmatic pilot, average percentage of words changed by physician ranged from 1.9% to 69.3% [4].
The Edit Workflow Is Two Cognitive Passes
A reliable editing sequence treats the AI draft as unverified data first and clinical prose second. The first pass checks whether the system captured what was actually said and who said it. UCSF program guidance instructs clinicians to review for missing, incorrect, or irrelevant information and to watch for content that was not said during the visit [5]. Clinician interviews reinforce that speaker attribution is a persistent failure point, including encounters where another individual is in the room and the note must be corrected to reflect the patient [1]. Checking speaker labels before reading clinical meaning prevents a clinician from silently adopting a misattributed statement.
The second pass is clinical validation: deciding what belongs in the note, where it belongs, and how strongly it should be stated. The UC Irvine authors distinguish accurate conversation capture from clinically appropriate documentation, which requires additional judgments about relevance, placement, source, and uncertainty [1]. This explains why a draft can be transcriptionally faithful yet still require substantial edits. UCSF materials list role and context misattributions, unsuitable tone, and improper section placement among errors to identify during review [5]. Treating these as separate passes prevents transcription errors from contaminating clinical reasoning and vice versa.
How Edit Patterns Vary by Specialty and Note Section
Edit volume is not evenly distributed across specialties. In a pragmatic prospective pilot at UC Davis Health, 31 physicians generated 7,545 notes, and vendor metrics for a subset showed median percentages of words changed of 8.3% for family practice, 9.5% for internal medicine, 34.4% for otolaryngology, and 22.4% for pediatrics [4]. The same source reported average percentage of words changed by physician ranging from 1.9% to 69.3% [4]. These figures describe one vendor deployment at one academic medical center and reflect technical reporting limits; percentage edited was unavailable for several specialties, including dermatology, obstetrics and gynecology, and endocrinology [4].
Why otolaryngology and pediatrics diverge from primary care is not established by these sources, but the qualitative data suggest mechanisms. Clinicians whose documentation required more complex differential reasoning and greater billing and medicolegal scrutiny reported spending more effort correcting errors, restoring missing clinical details, and rewriting content to align with specialty norms [1]. Selective use across note sections reflects systematic gaps between conversational summarization and documentation requirements [1]. One clinician described the AI making a medication look inappropriate when it was prescribed for overactive bladder rather than recurrent UTIs [1]. Edit workflows should therefore route specialty-specific checks earlier than generic grammar review.
A Concrete Evaluation Example
A suggested evaluation for an ambulatory multispecialty group would sample 100 consecutive draft-review cycles per specialty over two weeks, with the reviewing clinician logging the first pass and second pass separately. First-pass logger entries would code errors as misattributed speaker, hallucinated content, or missed exam finding, matching the UCSF safety checklist categories [5]. Second-pass entries would code section misplacement, uncertain phrasing, and specialty convention gaps, consistent with the UC Irvine finding that accurate capture does not guarantee clinically appropriate documentation [1]. The vendor would report the same word-level retention metric used at UC Davis, defined as one minus word-level edits divided by words in the final note [4].
This exercise would produce specialty-specific data on which pass consumes the most corrections and whether high edit percentages concentrate in transcription or clinical judgment. A dermatology or otolaryngology group that spends most second-pass effort reordering sections and tightening language has a template problem; a primary care group that spends most first-pass effort restoring speaker labels has an attribution problem. The distinction matters because the fixes are different: prompt or audio configuration for attribution versus specialty template redesign for section placement. No named vendor meets this test absent local data.
Technical Decision Points in Review Design
Health systems face a decision about whether editing happens inside the ambient tool, inside the EHR, or across both. UCSF's program guidance addresses a concrete technical constraint: copy-paste can result in loss of formatting, and Smartlinks are recommended to bring content into notes for a better experience [5]. This matters because a review workflow that depends on copying drafts introduces formatting defects that clinicians must then fix, adding a third layer of correction unrelated to clinical accuracy. Systems should identify where discrete fields such as medications, problem lists, and family history are updated; UCSF directs clinicians to update those discrete elements in the EHR as they normally do, separate from the note draft [5].
A second technical decision is whether the vendor provides usable edit metrics. The UC Davis pilot relied on a vendor measure of percentage retained, converted to percentage of words changed, but technical issues limited that measure to 960 of 7,545 notes [4]. That denominator failure is itself an implementation finding: a health system cannot monitor review burden if the vendor cannot reliably deliver edit data. Any editing workflow should specify which events are logged, at what granularity, and for which clinicians. Without that, post-deployment evaluation collapses into anecdote, and specialty outliers remain invisible.
Evidence Limits and Governance Implications
The strongest editing-behavior evidence comes from interview and observational studies at single institutions with commercial tools. The UC Irvine study involved 30 clinicians in routine outpatient care and drew on their personal experience and review of real-world examples [1]. The UC Davis pilot was a quality assessment designed to support safety before wider deployment [3]. Neither study establishes a generalizable error rate across vendors. A simulated-case evaluation in the Veterans Health Administration found lower quality scores for AI-generated notes than human-produced notes across five standardized primary care cases, with the largest deficit in acute low back pain [7]. The authors of that study explicitly note that cases were simulated and human notes were not generated under real-world constraints [7].
These limits should inform governance rather than disqualify the technology. The UC Irvine authors recommend improving transcription and speaker attribution accuracy, using default language that reflects clinical uncertainty, and supporting adaptable specialty- and encounter-specific templates [1]. UCSF's operational guidance requires full review and validation of every draft, with consent documented for each visit [5]. What the evidence cannot yet supply is a validated threshold for acceptable edit burden by specialty or a standardized instrument that predicts patient harm. Systems that proceed without that instrumentation are testing clinical documentation, not just software.
Sources & references
- Clinicians’ rationale for editing ambient AI–drafted clinical notes: persistent challenges and implications for improvement - PMC · pmc.ncbi.nlm.nih.gov
- Clinicians’ rationale for editing ambient AI–drafted clinical notes: persistent challenges and implications for improvement | Journal of the American Medical Informatics Association | Oxford Academic · academic.oup.com
- JMIR Medical Informatics - Quality of Clinical Notes Created by Ambient Listening Generative AI: Pragmatic Prospective Pilot Study · medinform.jmir.org
- JMIR Medical Informatics - Quality of Clinical Notes Created by Ambient Listening Generative AI: Pragmatic Prospective Pilot Study · medinform.jmir.org
- AI SCRIBES - community-affiliates-internal.ucsf.edu · community-affiliates-internal.ucsf.edu
- Implementing Artificial Intelligence-Enabled Ambient Documentation Technology for Ambulatory Clinicians: An Innovation Evaluation | Journal of General Internal Medicine | Springer Nature Link · link.springer.com
- Rapid Evaluation of Artificial Intelligence Technology Used for Ambient Dictation in Primary Care: Comparing the Quality of Documentation of Artificial Intelligence–Generated and Human-Produced Clinical Notes | Annals of Internal Medicine · www.acpjournals.org
- Impact of Unrestricted Artificial Intelligence Scribe Deployment in a Single Health System on Clinician Workload and Productivity: A Quality Improvement Study: Annals of Internal Medicine: Vol 179, No 11 · www.acpjournals.org