“When an AI makes a mistake in coding, they’re not uniformly random… They make telltale mistakes.” Neither do devices. Here is every device class the FDA heard about in 2024, plotted by how loudly it fails and how badly.
Aron Szanto’s claim is the most useful sentence anyone said about AI code this week, and it isn’t really about AI. Random failure you can only catch by reading everything. Patterned failure you can build a detector for. The whole argument for clinician-built software surviving review rests on which of those two you are in.
So: is failure patterned? We have a public answer, and it has been sitting in a free JSON API the whole time. Every year US device manufacturers, hospitals and patients file a few million adverse event reports into MAUDE, and every one is tagged with exactly three possible outcomes: malfunction, injury, death.
If failure were random, every device class would land near the same mix. It does not. Each class has a signature — and the signatures are so sharp they are almost fingerprints.
Continuous glucose monitors: 340,130 reports in 2024, 98.6% malfunction, 18 deaths. Endosseous dental implants: 536,012 reports, 99.7% injury, 2 deaths. Same database, same year, opposite shapes.
Below, every dot is one device class. Left to right is how many reports it generated (log scale — the range is three hundred to half a million). Bottom to top is what share of those reports involved a human being harmed rather than a box misbehaving. Dot size is deaths.
339 of 560 device classes — 61% — are more than 90% a single event type. Forty-four of them are 100% one type: every single report, all year, the same outcome. If the three outcomes were assigned at random, the equivalent figure would be about 2%.
That is what “telltale, not random” looks like when you can count it. And it is the whole argument. Szanto’s team could build Argus because AI coding errors fall into a small enumerable set. MAUDE is what the same claim looks like after thirty years of somebody actually writing the taxonomy down.
Your error taxonomy is a design document, not a postmortem artifact. Before you write the happy path, spend thirty seconds on the API above and pull the failure distribution for every device your tool takes a feed from. If your infusion-pump integration assumes failures announce themselves as injuries, MAUDE will tell you that 98.6% of the time the pump just quietly misbehaves — which is the failure mode with no alarm attached to it.
A dataset this clean-looking is exactly the kind you should distrust. Four things are wrong with the chart you just used, and all four matter more than the chart.
1. There is no denominator. Nothing above is a rate. MAUDE counts reports, not events, and it has no idea how many devices are in service. Continuous glucose monitors top the malfunction chart because millions of Americans wear one, not because they are the least reliable object in medicine. Every ranking on this page is a ranking of reporting, not of risk.
2. Small numbers manufacture monsters. Pull the threshold slider to the left and MAUDE will tell you the deadliest device class in America is “VALVE REPAIR” — 269 reports, 27.5% of them deaths. Push it right and that claim evaporates; by 20,000 reports the worst class in the country sits at 4.1%. Nothing changed except how much evidence you demanded. Any dashboard that ranks anything without a sample-size floor is generating monsters for a living.
3. The taxonomy is not a taxonomy. generic_name is a free-text field. In 2024 alone it held 14,902 distinct strings for roughly 6,000 real FDA product codes. “VENTRICULAR (ASSIST) BYPASS” and “VENTRICULAR (ASSISST) BYPASS” are both in there, as separate classes, with a typo between them. So are “NO MATCH” (15,276 reports), “UNKNOWN”, “ACCESSORIES”, and hundreds of catalogue SKUs sitting where a device category should be — toggle hide brand-name strings and 153 of the 560 “device classes” vanish, because they were never device classes. The enumerable taxonomy Szanto is describing does not come free with the data. Somebody has to build it, and here nobody did.
4. Almost nobody in this dataset is a clinician. 93.2% of 2024 reports came from manufacturers and another 6.0% from distributors. Voluntary reports — the physician who noticed something — are 0.5%. The pattern you are looking at is largely the pattern of what manufacturers are legally obliged to file, filtered through their own adjudication of what counts as an injury.
Spinal cord stimulators appear in MAUDE twice. Under the generic label “STIMULATOR, SPINAL-CORD, TOTALLY IMPLANTED FOR”: 12,058 reports, zero deaths. Under a manufacturer’s brand string, “NEVRO SENZA”: 1,148 reports, 205 deaths — 17.9%. Both figures are real; we re-pulled them from openFDA and they match exactly. They are not describing different levels of danger. They are describing two different filing conventions inside one company’s regulatory department. If a benchmark can be moved this far by a naming choice, the benchmark is measuring the filer.
Ranked by share of that class’s reports coded as deaths. Move the slider above and this table rewrites itself — that instability is the finding.
| Device class (as MAUDE spells it) | Reports | Deaths | Death % |
|---|
Not that devices are dangerous, and not that any class is worse than another. Only this, which is enough: failure in a bounded domain has shape, the shape is stable, and the shape is cheap to look up. That is the precondition for a detector. It is why Argus can exist for AI-written code, and it is why the equivalent for clinical AI output — hallucinated citations, anchoring on the chief complaint, silent unit conversions — is a taxonomy problem long before it is a model problem.
MAUDE took three decades and a federal mandate to get this far, and it still can’t spell “assist” consistently. Whoever writes the clinical-output version should budget accordingly.