Somebody always waits longer (external link)
This one is my actual job, so forgive the enthusiasm.
A team took 313,966 ED visits over four years, trained a LightGBM model to predict whether a CT study would turn out to be clinically actionable — at the moment the order is placed, before anyone scans anything — and then ran a discrete-event simulation of the department under two policies: first-in- first-out, and AI-ordered.
The headline results are good. Median wait for actionable studies down 10.75 minutes. The 90th percentile down 43.36 minutes, which matters far more than the median, because the 90th percentile is the patient your morbidity and mortality conference is about. Actionable findings resulting within the hour went from 48.33% to 57.30%.
Two things I want to draw out, because they are the difference between reading this as a press release and reading it as an implementation.
First: it predicts at the moment of order — before the study for that patient exists — from multimodal data the department already collects. Whatever else is in that feature set, it cannot be the scan it is prioritising. So this is not the usual shape: not a box that reads images and writes findings back, but something that sits upstream and changes the sequence in which work gets done. Every conversation I have had about "imaging AI" for four years has assumed a box that reads scans and writes findings back. This one reorders a list, which is architecturally trivial and operationally enormous, and it is the kind of thing I could actually deploy without a twelve-month integration project.
Second: somebody waits longer. For non-actionable studies the median wait improved by 5.80 minutes, but the 90th percentile got worse by 14.46 minutes. That is the honest cost, stated plainly in the results, and it will be the first thing anyone asks me in a governance meeting. Prioritisation is zero-sum on a fixed scanner. You cannot move somebody up without moving somebody down, and the somebody moving down is at the tail of the low-acuity distribution — the patient nobody writes a paper about, waiting an extra quarter of an hour because a model said their order looked boring.
I think the trade is defensible. I do not think it is the department's decision to make quietly, and it should certainly not be made by whoever configures the worklist rules, which is to say me.
The ceiling analysis is the number I would take to a business case: the model captured 80–87% of the benefit available from perfect predictions. That is the useful framing. Not "how accurate is it" but "how much of the achievable gain does it get". Very few vendors present it that way and they should.
Caveat that matters: this is a simulation, calibrated to real operations but still a simulation. Queues in a real department are shaped by things a discrete-event model does not capture, including the fact that people reprioritise by hand when they think the system is wrong — which, after the first bad flag, they will.