For most of the past five years, the prevailing assumption in AI has been simple: bigger models are better. Bigger meant smarter, more capable, more “general.” The major players — OpenAI, Anthropic, Google — pushed that narrative hard, and they had the benchmarks to back it up.

But that story’s wearing thin.

The latest data in the 2025 AI Index Report from Stanford HAI points to something much more interesting: small models are catching up fast — and in a growing number of cases, they’re the more practical choice for real-world use.

Here’s the stat that stopped me:

Microsoft’s Phi-3-mini, a 3.8 billion parameter model, is matching the performance of Google’s PaLM, which has 540 billion parameters, on the MMLU benchmark — a standard test of general knowledge and reasoning​.

If that’s not a turning point, I don’t know what is. We’re looking at a model that’s 142x smaller, performing on par with one of the largest models from just 2 years ago. It signals huge strides in model efficiency and training techniques, making smaller models far more viable for enterprise and on-device use.

Figure 1: Graphic showing the benefits of LLMs vs SLMs.

The performance gap is shrinking

The report shows this isn’t an isolated case. Across the board:

  • The performance difference between the #1 and #10 ranked models shrank from 11.9% in 2023 to 5.4% in 2024.
  • The gap between open-weight models and proprietary closed models dropped to just 1.7%, down from 8% last year​.

Translation: the “best” model is increasingly a matter of fit, not just firepower.

What this means for tech leaders

If you’re making decisions about AI architecture, model selection, or deployment in 2025, here’s what matters:

1. You might not need a foundation model

For a lot of enterprise tasks — chat, summarization, classification, retrieval — you no longer need GPT-4 or Claude 3. You can do the job with a small model, fine-tuned and deployed internally.

It’ll run cheaper, faster, and probably leak less data.

2. Control is back on the table

Smaller models mean you can host them yourself. That opens up options in regulated industries where cloud dependency, privacy, and IP protection are deal-breakers. If you’re in healthcare, finance, defense, or govtech, this is huge.

3. Open-source is real competition

It’s not just Anthropic and OpenAI anymore. LLaMA, Mistral, Gemma, and Phi are putting pressure on the incumbents — and they’re doing it with smaller models that can be audited, forked, or customized.

If you’ve been putting off a serious evaluation of open models, now’s the time.

The big picture

This isn’t the end of frontier models. There will still be use cases — complex reasoning, multi-modal interactions, long context — where size wins. But the reality in 2025 is more nuanced:

In many real-world scenarios, smaller, well-trained models are now “good enough”, and significantly easier to work with.

That’s a big shift. It means architecture decisions shouldn’t default to “whatever’s biggest.” They should start with the question: what’s the smallest model that does the job?

Because the answer to that might be smaller than you think, and a lot more efficient.

‍