Real progress, from real institutions, faster than anyone could reasonably have expected two years ago. None of it independently tests a live system. Here’s the honest read of what each framework hands a hospital — and the specific gap it leaves for Gate Zero.
Defines the govern / map / measure / manage functions a trustworthy AI program should have in place, across the model lifecycle.
A well-built, genuinely useful registry baseline for documenting a tool’s intended use, training population, and known limitations.
The Responsible Use of AI in Healthcare certification, launched June 2026. Voluntary, but surveyors expect dated, current artifacts — not a policy binder.
Requires an AI-device manufacturer to file a plan for how a model may change over time, with real-world monitoring “as applicable.”
Federal transparency rules require EHR vendors to disclose validation and monitoring information to the hospitals buying their tools.
A growing set of state requirements that clinical AI use be disclosed to patients and providers.
“Owning the software and running the software weren’t proof of anything.”
— FROM “THE PILOT PROBLEM,” THE LAB’S FOUNDING ESSAYThe Institute of Medicine’s To Err Is Human put a number on how many patients were dying every year from preventable medical error. Employers and health plans didn’t just ask hospitals to try harder — they formed the Leapfrog Group and built something the industry didn’t have: a standardized way to test, from outside both the hospital and the vendor, whether a piece of safety technology actually did what it was supposed to do.
Computerized physician order entry was one of the first things they pointed it at. Hospitals had already bought it, installed it, and trained staff on it — and that didn’t automatically make care safer. Early systems introduced their own new failure modes, from alert fatigue to interfaces that made it easy to select the wrong patient or the wrong order. Owning the software and running the software weren’t proof of anything.
So Leapfrog built the CPOE Evaluation Tool: a standardized simulation that ran a defined set of test orders — including orders built to trigger known, dangerous error patterns — through a hospital’s live system to see whether it actually caught what it was supposed to catch.
Healthcare AI is running into the same question CPOE did: not whether a tool works in a demo, but whether anyone is actually positioned to check what it does once it’s live. Unlike CPOE, an AI tool doesn’t hold still once it’s deployed — it can be retrained by its vendor, and its behavior can shift as it meets more patients and situations than a short pilot ever covered. That’s part of why the frameworks above ask for more than a one-time sign-off.
Read together, the frameworks above add up to a specific, unglamorous skill set: clinical statistics, bias and drift detection, adversarial and edge-case test design, and the discipline to keep checking after go-live instead of only before it. That is not the skill set of a typical hospital IT or informatics department — which is staffed and funded to implement software and keep it running, not to independently evaluate whether a live clinical model is still behaving the way it did during its pilot.
Industry survey data points in the same direction: most system-affiliated hospitals now report doing some version of accuracy and bias evaluation — real progress — but adoption is uneven between system-affiliated and independent hospitals, and dedicated, independent testing capacity remains the exception rather than the rule. Ask what “monitoring” means in most of the places that report doing it, and the honest answer is often a team watching a vendor’s own dashboard — which is not the same as independently verifying it.
None of this is a failure of effort. CHAI, the Joint Commission, and federal regulators have each built real, substantive pieces of what oversight should look like. What none of them can do is manufacture, inside every hospital, a skill set that is currently in short supply across the entire industry. A framework can describe what good monitoring looks like. It cannot hand a community hospital the statisticians and clinical-informatics specialists needed to run it.