Small models are becoming the default edge runtime

On-device models have crossed a practical threshold. They can classify intent, extract structured fields, rewrite short passages, and coordinate simple tools without paying a network round trip. That changes architecture, not just latency.

Local first, cloud when necessary

A resilient AI product can route private, frequent, and latency-sensitive work to a small local model. Complex synthesis and tasks needing fresh knowledge still belong in the cloud. The boundary should be explicit, measurable, and visible to the user.

The best edge workload is one whose value improves when the network disappears.

What teams should measure

Track task success, energy use, first-token latency, and fallback rate together. A tiny model that fails silently is not cheaper. A slightly larger model with predictable routing often produces the better system.