The Pinnacle of Mediocrity

We were promised the singularity, or at least a digital god that could solve cold fusion by lunch. Instead, we got a $7 trillion valuation for a collection of models that are currently being outpaced by a Convolutional Neural Network (CNN) from the era when people were still doing the Ice Bucket Challenge. A recent study pitted frontier Large Language Models against a specialized 10-year-old diagnostic tool, and the results were exactly as embarrassing as you’d hope. The 'cutting-edge' AI, designed to understand the very fabric of human thought, proved significantly worse at identifying malignant melanomas than a piece of software that could probably run on a smart refrigerator from the Obama administration.

It is a beautiful irony. We have built the world’s most sophisticated autocorrect, trained it on the collective output of Reddit and Wikipedia, and are now shocked—shocked!—that it lacks the clinical nuance of a narrow tool trained solely on dermatological data. The frontier models are busy trying to convince us they are sentient while the 2014 CNN is busy actually being useful. It’s like hiring a philosopher king to fix a leaky pipe; he’ll give you a breathtaking lecture on the fluid nature of existence while your basement fills with three feet of water.

The Generalist Trap

The obsession with 'Generative AI' has led to a peculiar form of brain rot where we assume that because a model can explain the French Revolution in the style of Dr. Seuss, it must be inherently better at everything else. This is the 'General AI' tax. To make a model good at everything, you inevitably make it mediocre at the things that actually require precision. In the world of skin cancer, 'mostly right' is just a polite way of saying 'lawsuit.' A CNN doesn't care about the historical context of the mole on your shoulder. It doesn't want to write a haiku about it. It looks at pixels, calculates probabilities based on a hyper-specific dataset, and gives an answer.

  • Specialized AI: Trained on 100,000 high-resolution medical images.
  • General AI: Trained on the entire internet, including 4chan and recipes for banana bread.
  • Results: The specialist wins, the generalist hallucinates a new type of mole that doesn't exist.

We are currently pouring venture capital into models that prioritize 'emergent behavior'—which is Silicon Valley speak for 'we don't actually know how it works, but look, it can do math now.' Meanwhile, the boring, reliable, narrow AI tools are sitting in the corner, doing the heavy lifting without demanding a seat on the board of directors. The sheer audacity of expecting a model trained to predict the next word in a sentence to also predict the cellular mutation of a skin lesion is a testament to our current era of tech-bro hubris.

A Decade of Progress Nowhere

There is something deeply satisfying about a tool from 2014 outperforming the 2024 hype machine. It suggests that perhaps, just perhaps, throwing more parameters and more electricity at a problem isn't the solution to everything. The 2014 CNN didn't need a cooling forest or a dedicated nuclear power plant to function. It just needed a clear goal and a curated dataset. In the intervening ten years, we’ve optimized for engagement, chatty personalities, and the ability to generate fake images of pugs in space suits. We haven't necessarily optimized for accuracy in the fields where accuracy is the only metric that matters.

If you’re a patient, you probably don’t care if your diagnostic tool can pass the Bar Exam or explain the plot of Inception. You want the tool that has a lower false-negative rate. The fact that we are even having a 'debate' between narrow and general AI in medicine is a sign that we’ve lost the plot. The generalist models are currently sitting at a 0.1% improvement rate per billion dollars spent, while the 'old' tech is still holding the line. It’s not a paradox; it’s just basic engineering. If you want to drive a nail, you use a hammer, not a Swiss Army knife that’s also trying to be your therapist.

What This Actually Means

This isn't just a win for 'legacy' tech; it's a sobering reality check for the 'AI will replace doctors' crowd. We are witnessing the limits of scale. You can double the size of an LLM every six months, but if the underlying architecture is designed for conversation rather than classification, you are just building a louder, more confident idiot. The frontier models are great at pretending to be experts, which is exactly why they are so dangerous in a clinical setting. They provide an answer with the same level of confidence whether they are quoting a medical textbook or a post from a lifestyle vlogger.

In the high-stakes world of medical diagnostics, 'good enough' is a death sentence. The persistence of the CNN's dominance proves that specialized, narrow intelligence is not a stepping stone to something better—it is the destination. We don't need a computer that can do everything poorly. We need a dozen computers that each do one thing perfectly. But of course, you can't raise $10 billion for a 'skin cancer app' in the current climate unless you promise it will eventually also write screenplays and manage a hedge fund.

We are currently trading precision for vibes, and the 2014 CNN just sent us the bill. It turns out that a decade of progress in 'General AI' has mostly just made it harder to find the tools that actually work. Maybe in another ten years, we'll invent a model that's as good as the one we already had in 2014. One can only dream.

Quick Answers

Is the old AI actually better?
Yes, in this specific clinical task, the 10-year-old CNN had higher sensitivity and specificity than the latest multi-billion dollar LLMs. It turns out specialization beats a diet of the entire internet.

Why can't the new models win?
LLMs are built on probability and language patterns, not the raw spatial analysis required for image-based diagnostics. They are trying to 'read' an image like a story instead of analyzing it like a map.

Should we stop using LLMs for medicine?
We should probably stop asking them for medical advice until they can reliably distinguish between a freckle and a death warrant. Using them as a fancy filing cabinet is fine; using them as a doctor is a gamble with bad odds.