API, self-hosting or hybrid AI: the evidence to gather before choosing
TokenShift Executive Note
AI-translated from the original French version.
Choosing between an API, self-hosting and a hybrid architecture starts with a specific use case. An API bill and a server budget do not show what it costs to deliver work to the required standard, on time, with data sent only to authorised destinations. That is the outcome against which the options must be compared.
The same cases, the same quality standard
Consider a hypothetical example: an insurer uses AI to extract information from supporting documents before a claims handler reviews a case. An acceptable response must capture the correct amounts, link each piece of information to its source document and flag anything missing. Plausible values are not enough.
These criteria must be identical for the API, the self-hosted model and any combination of the two. The full cost per accepted case can then be calculated: all spending on processing the cases, including unsuccessful attempts, divided by the number of cases that meet the criteria. If human correction makes a case acceptable, that time belongs in the calculation.
This ratio cannot be read in isolation. The acceptance rate, errors found after validation and turnaround times, especially during peaks, also matter. An option that costs little for straightforward documents may leave teams with most of the difficult work. The comparison must therefore use the same set of cases and include the existing process.
The cost of the model is not the cost of the service
Self-hosting brings costs for capacity, integration, monitoring and continuity. Reserved capacity still costs money when idle; a new model version must be evaluated before it goes into service. According to the DeepInfer documentation cited in the sources, the published version supports offline inference, but other components are needed to build a complete service. This observation applies to that specific version, not to every self-hosted solution. DeepSeek, 2026.
The question for the business is concrete: who takes responsibility for those components, incidents and backup capacity? Steady volumes can help make use of dedicated infrastructure. Uneven traffic can make reserved capacity less attractive. Neither point settles the choice without measured volumes, the required service level and the cost of the team providing it.
An API also leaves work for the business: integration, response checks, error handling and a fallback. Setup or migration costs must be spread over an explicit time horizon for each option.
Hybrid AI adds a decision to every case
A combination could use a self-hosted model for routine processing and send certain cases to an API or a claims handler. It is worthwhile only if routing identifies the cases that need different treatment well enough, without creating more rework, delay or cost than it avoids.
The RouteLLM work cited in the sources studies routing between models on MT Bench, with a cost estimate specific to its protocol. Even if confirmed, its results establish neither the quality achievable on a company’s cases nor the economics of a hybrid service in operation. RouteLLM, 2025.
In the hypothetical insurer example, an illegible or contradictory document could trigger a check. The test must show whether such cases are actually detected. A silent error in a misrouted case matters as much to the decision as an API call avoided.
In a regulated business, routing can select only destinations permitted for the data concerned. An API that can handle a document better is not an option if that document cannot be sent to it. The CNIL recommendations cited in the sources advise examining how the provider may use the data and involving business, IT, security and data protection leads. CNIL, 2024. The review must also cover logs, backups and support access: the model’s location alone does not describe where the data goes.
Measure before migrating
The decision calls for a trial using the same cases, including routine and difficult ones, with the same acceptance criteria. It must compare the API, self-hosting and hybrid approach with the current process: accepted cases, human rework, errors, turnaround times and full cost. Varying the volumes or the share of complex cases can then show whether the reported advantage still holds.
The business team sets the required quality standard. Those responsible for the data define the permitted destinations. The operations team confirms what it can support; finance checks the costs used. If these elements are missing, the priority is to measure them within a limited scope before funding a migration.