Artificial Intelligence
Small Language Models Are Quietly Winning
For two years the answer to every AI problem was a bigger model. That reflex is fading. Teams shipping real products are discovering that a well-tuned small model, running close to the user, beats a giant general-purpose one on latency, cost and predictability.
The economics changed first
Inference, not training, dominates the lifetime cost of an AI feature. A model that answers in 200 milliseconds for a tenth of a cent changes what you can afford to build: autocomplete on every keystroke, classification on every row, summaries on every ticket.
Once a task is narrow enough to describe in a page of instructions, a 3B-parameter model fine-tuned on a few thousand examples usually matches its much larger cousin.
Where small models excel
Structured, repetitive, well-bounded work
Extraction, routing, tagging, tone rewriting, and schema-constrained generation are the sweet spot. These tasks have a clear right answer and a narrow output space, which is exactly what fine-tuning rewards.
The bonus is determinism. Smaller output spaces mean fewer surprises in production, and far easier evaluation.
Where they still fall short
Long multi-step reasoning, open-ended research and code that spans many files still favour frontier models. The pragmatic pattern is a router: a small model handles the ninety percent, and escalates the rest.
How to migrate without regressions
Build the eval before you switch
Collect three hundred real requests, label the expected outputs, and freeze them as a test set. Swap the model, run the set, and compare. Anything below your threshold routes upward.
Teams that skip this step usually roll back within a week, not because the small model was bad but because nobody could prove it was good.
