When a vendor tells you which model they use, it sounds like an answer. It is closer to a starting condition. The model is one component in a system that also has to retrieve the right documents, respect who is allowed to see what, hand results to a person at the moment they need them, and stay affordable at ten thousand requests a day instead of ten.
There is a version of AI implementation that consists of pointing the largest available model at every task and calling the result a strategy. It demos well, because the largest model is genuinely very good and because a demo runs at low volume. Then it goes to production and the bill arrives, or the interface feels slow, or the compliance team asks where the data went, and the choice that was never really made becomes the thing everyone is arguing about.
Frontier models belong at the centre of serious systems. They are what makes any of this work. The mistake is treating one of them as the answer to every question inside the system rather than as the most expensive tool in a set.
What most capable is measuring
Model rankings are built on benchmarks: broad academic tests, reasoning problems, coding tasks, long context recall. They measure general capability across a wide spread of work, which is exactly what you want when the question is which model is strongest overall.
Your system does not do work across a wide spread. It classifies incoming email into six categories. It extracts nine fields from a purchase order. It checks whether a clause appears in a contract and flags the ones where it does not. On tasks that narrow, the gap between a large model and a small one often collapses to nothing measurable, because the task never touches the capability that separates them.
The useful question is not which model is better. It is whether the difference between them shows up on your work, on your documents, at your acceptable error rate. That is answerable, and the answer is frequently no.
Four costs a demo will not show you
Latency. A model that takes six seconds to respond is fine in a demo, where the audience is watching and waiting. It is not fine inside a form a person fills in forty times a day. Slow tools get abandoned quietly, and the abandonment is usually reported as poor adoption rather than as a design fault.
Unit cost at volume. Cost per request is a rounding error during a pilot and a line item in production. The arithmetic is unforgiving: a workflow that runs twice a minute across a business day, on a model priced for hard reasoning, doing work a cheaper model handles identically, is a permanent tax paid for a capability that was never used.
Variance. Larger models are not uniformly more consistent. For structured extraction with a fixed output shape, a smaller model constrained tightly to the format will often produce more stable results than a more capable model given room to be discursive. Consistency is worth more than peak quality in anything that feeds a downstream process.
Data boundaries. Where a request goes, what is retained, which jurisdiction it lands in and what contractual terms apply are questions your risk function will ask. They constrain the option set before performance is considered. In financial services, professional services and legal they sometimes settle the question entirely.
The mix matters more than the model
Well built systems rarely use one model. They route.
The pattern that works looks like this. Cheap, fast models handle the high volume mechanical steps: classification, extraction, routing, formatting, first-pass filtering. A capable model handles the parts where judgment actually lives: synthesis across sources, ambiguous cases, anything a person will read and act on directly. Somewhere there is a rule for escalation, so that a low confidence result from the cheap path gets a second look from the expensive one rather than going out wrong.
That design costs a fraction of routing everything to the top, and it is usually faster where speed is felt. It is also more work to build, which is the honest reason it often does not get built.
Assume the answer changes
The model you would pick today is not the model you would pick in nine months. Prices fall, new versions land, capability moves. That is a good thing, and it is only a good thing if your system can take advantage of it.
Which means the model should be a configuration choice, not an assumption baked through the codebase. Prompts, retrieval and business logic sit on one side. The model sits behind an interface you can swap. Your evaluation set stays fixed, so when something new comes out you rerun it and get a number instead of an opinion.
Systems that cannot be re-evaluated cheaply do not get re-evaluated. They stay on whatever was chosen at the start, quietly paying more than they need to, until someone rebuilds them.
What to ask
If someone is building this for you, four questions separate a considered design from a default one.
Which model handles which step, and why that one. Look for different answers for different steps.
What does this cost per thousand requests at our expected volume. A vendor who has not run the number has not thought about production.
What happens when a better or cheaper model appears. The answer should describe a swap and a test, not a project.
Show me the evaluation set. If model choice was made by impression rather than measurement, it will become clear here.
Where we come in
Alppoint puts engineers on the problem and ships the system into your environment. Model selection is part of that work, decided per task against your data and your volumes, with the routing and the evaluation harness built so the decision can be revisited later without a rebuild. The system is yours, and so is the ability to change it.
If you are being quoted for a build and want a second read on the design, that is a conversation we are glad to have.
