Eight of eighteen models had drifted (external link)
ReclAIm is a multiagent LLM system that watches medical imaging models for performance decline and kicks off a fine-tuning run when it finds one. A master agent, three task-specific agents, natural language interface, benchmarked across brain MRI, chest CT and chest radiography.
My honest first reaction was that adding a multiagent system to supervise your classifier is a way of having two things to debug at 4pm on a Friday. I still think that. But the finding underneath the framework is the part worth your attention, and it is not really about agents at all.
Performance discrepancies turned up in eight of eighteen models.
Not eight models that were badly built. Eight of eighteen, on the researchers' own benchmark data, drifting between their test partition and their inference partition. One case — cardiomegaly, InceptionV3 — had declined by 40.6%.
Forty percent. If that happened to a tool in my department, I would like to believe I would notice. I am not confident I would. What I would see is a gradual increase in radiologists ignoring the flags, and I would probably attribute it to the radiologists.
That is the uncomfortable thing about decline. A model that stops working outright is easy — the results stop arriving, someone raises a ticket, I find the failed job. A model that gets worse keeps producing output in exactly the right format, on schedule, attached to the correct accession. Every system I have to monitor it says green. The only signal is clinical, it is slow, and it arrives as an impression rather than an alert.
Which is precisely the gap that the PCCP review found missing from the public documentation: predefined drift triggers and continuous performance monitoring. Here is a paper demonstrating both that drift is common and that detecting it is tractable, and there is a regulatory mechanism for updating models that does not require you to publish how you would detect it.
The correction results are genuinely encouraging: fine-tuning with parameter anchoring against catastrophic forgetting restored performance to within ±2% of baseline. So the loop closes, in a lab.
What I want, and what nobody sells, is far less clever than ReclAIm. I want a dashboard showing, per deployed tool, flag rate over time against the same period last year. Not accuracy — I do not have ground truth in real time, nobody does. Just volume. Because every drift story I have described here shows up as a change in how often the thing fires, long before anyone can prove it got worse. That is a chart, not a multiagent framework, and I would take it tomorrow.