Ben Scharfe, executive vice president for artificial intelligence at Altera Digital Health
Photo: Ben Scharfe
While the healthcare industry rapidly integrates artificial intelligence into everything from clinical documentation to patient triage, a critical and largely unaddressed risk is emerging: automation complacency, said Ben Scharfe, executive vice president for artificial intelligence at Altera Digital Health.
Altera Digital Health – in booth 4431 in the exhibit hall at HIMSS26 this week – is a vendor of clinical, financial and interoperability systems and services to hospitals, health systems and large physician practices. Scharfe believes automation complacency is among the most important issues facing HIMSS26 attendees.
Similar to alert fatigue
"Much of the current discourse around risk centers on issues like algorithmic accuracy and bias rather than what happens after implementation, when clinicians begin to interact with these AI tools daily," Scharfe explained. "This phenomenon mirrors the long-recognized problem of alert fatigue, where a constant barrage of notifications leads to desensitization.
"Similarly, as clinicians are presented with a continuous stream of AI-generated outputs, their level of scrutiny could diminish," he continued. "This fading attention could have serious consequences for patient safety."
The core of the automation complacency issue is not technological but psychological, rooted in the human-machine interaction within routine clinical workflows, he added.
"As we rush to scale and embrace AI, the risk of becoming too comfortable with its output grows," Scharfe noted. "This can lead to an accumulation of small, nuanced errors that go undetected over time. For instance, with the rise of ambient listening technologies for clinical notes, subtle inaccuracies in transcription – a misheard word or phrase – may not be caught by providers if they no longer meticulously review the output.
"While the note may appear correct and function for billing purposes, these small inconsistencies can propagate throughout a patient's record, leading to a cascading effect of misinformation that is difficult to trace and can eventually result in downstream harm," he continued.
Legal and patient safety risks
The legal and patient safety risks associated with automation complacency vary depending on the specific AI use case, Scharfe added.
"The level of risk in using AI for billing or scheduling is substantially different from employing it to advise a clinician on a patient's health risks," he said. "As these technologies become more integral to clinical decision-making, they may fall under stricter regulatory oversight. Consequently, we will see a complex distribution of liability among providers, health systems and technology vendors, determined by the specific use case.
"Addressing automation complacency is therefore not just a matter of technological refinement but a critical imperative for ensuring patient safety and navigating the evolving legal landscape of AI in healthcare," he continued. "This shift in focus is a natural consequence of the real-world learnings that follow pilot programs and initial adoption."
As healthcare moves through 2026, it is entering a more mature phase of AI integration where these psychological and practical human challenges are no longer theoretical – they are the new operational reality IT professionals must solve for, he concluded.
Follow Bill's health IT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.
WATCH NOW: The model context protocol at work in AI and clinical workflows
