For most businesses, calling an LLM through an API is cheaper, quicker to ship and workable under GDPR. That holds if you choose an EU data-residency option, sign the provider’s processor terms and keep personal data out of prompts where you can. Self-hosting an open-weight model starts to pay off in three cases: you have high and steady token volume, you need hard control over where data goes, or you need latency that a shared endpoint can’t guarantee. Even then, most of the cost is engineering time rather than GPUs. For many teams the sensible answer is a hybrid, with an API by default and a self-hosted model for the narrow workloads where control genuinely matters.
The short version
Here is how the options compare on the dimensions that usually decide the question. Note the middle column: a managed open-weight deployment in an EU region is the option people most often forget exists.
| Dimension | Public LLM API | Managed open-weight model (EU region) | Fully self-hosted (your hardware or your cloud GPUs) |
|---|---|---|---|
| Cost at low or spiky volume | Lowest: pay per token, nothing when idle | Moderate: per-token or per-hour depending on provider | Highest: you pay for the GPU whether it’s busy or not |
| Cost at high, steady volume | Grows linearly with usage | Often cheaper than frontier APIs | Can be lowest, if utilisation stays high |
| Quality ceiling | Highest: frontier models are usually API-only | Strong open-weight models, below the frontier on hard reasoning | Limited by what fits on your hardware |
| Latency | Good typical latency, variable tail, rate limits | Similar to API, region-dependent | Predictable once tuned; slow if under-provisioned |
| Data control | Contractual: DPA, residency, retention settings | Contractual, with fewer parties involved | Physical: prompts never leave your boundary |
| Ops burden | Minimal | Low to moderate | High: serving stack, patching, monitoring, upgrades |
| Time to first working version | Days | Days to a couple of weeks | Weeks, plus hardware lead times |
Cost: utilisation decides everything
The common mistake is to compare an API invoice against the sticker price of a GPU. What you are really comparing is a variable cost against a fixed one, and the deciding variable is how much of the time that fixed capacity is doing useful work.
What an API really costs
API pricing is per million tokens, usually with separate input and output rates, and output is typically several times more expensive. Your real bill depends on prompt length, which grows quietly as you add system instructions, retrieved documents and conversation history. It also depends on retries, and on whether you use cheaper tiers such as batch processing or prompt caching where the provider offers them. For most early-stage workloads, such as a support triage agent, a document extraction pipeline or an internal assistant, the monthly API bill is small next to the salary cost of the people building it. Our breakdown of what it costs to build an AI agent goes through where the money actually goes.
What self-hosting really costs
Self-hosting has four cost lines, and the GPU is only one of them. You have the compute itself, whether bought or rented. You have the serving stack: an inference server such as vLLM, TGI or llama.cpp, a gateway, authentication, logging and autoscaling if you need it. You have the engineering time to benchmark models, tune quantisation and batch sizes, and keep the whole thing patched. And you have the opportunity cost of the quality gap. If the open-weight model you can afford to run gets a few more extractions wrong, someone has to catch and fix them.
That third line is the one people underestimate. A model server is production infrastructure with an attack surface like any other. Gateway and proxy layers need patching on the same cadence as the rest of your stack, as our note on PHP2Shell, LiteLLM 1.107.0 and what to patch now shows.
A break-even sum you can do on a napkin
Take your all-in monthly cost for a self-hosted node, including a fair share of engineering time. Divide it by the blended per-token price of the API you would otherwise use. The result is the monthly token volume you need just to break even. As an illustration only: if the node costs €2,000 a month all-in and the API works out at €1 per million blended tokens, you need two billion tokens a month before self-hosting saves anything. Then check that the node can actually serve that volume at your required latency. If your traffic is business-hours only, the GPU sits idle roughly two-thirds of the week, and the sum moves sharply in the API’s favour.
The pattern holds across most realistic numbers. Self-hosting wins on cost for large, steady, batchable workloads such as overnight document processing, bulk classification or embedding generation. It rarely wins for interactive, spiky, low-volume use.
Latency: closer isn’t always faster
Intuition says a model on your own network must be faster than one across the internet. Sometimes it is. A small model on a local GPU can return the first token very quickly, with no network hop and no shared queue. That matters for voice interfaces, autocomplete and tight agent loops where a single task makes dozens of calls.
Large models are different. Hosted frontier APIs run on hardware you are unlikely to match, so their raw generation speed usually beats a modest self-hosted box running a comparable model. What APIs give you less of is predictability. Tail latency varies with the provider’s load, rate limits cap your throughput, and an outage you don’t control becomes your outage. Self-hosting trades peak speed for consistency, provided you size it for your peak concurrency rather than your average.
Measure time to first token and total completion time separately, because they matter differently depending on whether a human is watching the output stream. And remember that most latency problems in real systems come from architecture rather than hosting: serial calls that could run in parallel, oversized prompts, or an agent that re-plans at every step. Fix those before you buy hardware.
GDPR and data residency: what hosting does and doesn’t change
This is where the debate gets muddled, so it is worth being precise. GDPR does not ban sending personal data to an LLM API, and self-hosting does not make you compliant. In both cases you are the controller. In both cases you need a lawful basis, purpose limitation, data minimisation, retention rules and, for higher-risk processing, a data protection impact assessment. What hosting changes is who else is involved and where the data physically goes.
Using an API lawfully
With an API, the provider is your processor, so you need an Article 28 data processing agreement. That is normally the provider’s standard DPA on business and enterprise terms. If data leaves the EEA, you need a valid transfer mechanism under Chapter V, such as an adequacy decision (including the EU-US Data Privacy Framework for certified US companies) or standard contractual clauses. Many providers now offer EU data residency for processing and storage on suitable plans. That is the single most useful setting for LLM data residency in the EU. The other things to check are how long prompts and outputs are retained, including for abuse monitoring; whether your data is used for training (on business tiers it generally isn’t by default, but confirm it in writing); and which sub-processors are involved. Our guide to data security and privacy in AI projects covers the wider controls.
What self-hosting buys you
Self-hosting removes the model provider from the chain entirely. Prompts never leave your boundary, nobody else retains them, and there is no cross-border transfer to document. For special-category data such as health or biometrics, for client data under strict confidentiality terms, or for organisations whose own customers forbid third-party AI processing, that physical guarantee can be worth more than any contract. Note the caveat, though: if you self-host on a cloud GPU provider, that provider is still your processor. You have reduced the number of parties, not eliminated them.
Self-hosting also gives you new obligations to meet yourself. Your logs now contain every prompt, so you own their retention, access control and deletion. A careless debug log on a self-hosted model can be a worse GDPR LLM hosting problem than a well-configured API. The EU AI Act sits alongside all of this, and your obligations as a deployer don’t depend on where the model runs. Our EU AI Act compliance checklist for SMEs sets out the dates.
Open-weight licences are a separate question
“Open weight” is not the same as open source. Licences range from permissive ones such as MIT and Apache 2.0 to custom licences with usage restrictions, attribution rules or thresholds based on company size. Before you build a product on open weight models for business, read the actual licence for the exact model version. Releases like DeepSeek V4.1-Flash under MIT weights make this easy. Others need a proper read by someone who understands what your product does.
Quality, model size and the middle options
Open-weight models have closed much of the gap for well-defined tasks: extraction, classification, summarisation, routing and structured output against a schema. They remain behind the frontier on long, multi-step reasoning and the hardest agentic work. The size of the model you can run also matters. A release like Mistral Large 4 at 1T parameters is open weight, but serving it yourself is a multi-GPU undertaking. Most practical self-hosting uses models small enough to fit comfortably on one or two GPUs, often quantised.
That is why the middle options deserve attention. A managed open-weight deployment in an EU region, or a dedicated instance of a commercial model hosted in the EU, gives you most of the data-control benefit without running a serving stack yourself. A hybrid router goes a step further. Requests containing personal or confidential data go to a local model, and everything else goes to the strongest available API. Built behind a single internal interface, it also means you can switch providers without rewriting your application. We run open-weight models alongside hosted APIs in our own lab on Proxmox and a DGX Spark, which is where we learnt what self-hosting really demands.
A decision checklist
Work through these in order. If you can’t answer one, that is the first thing to find out.
- Data class: does the workload involve special-category data, client-confidential material or contractual bans on third-party processing? If yes, self-hosting or a dedicated EU deployment moves up the list.
- Residency: does your preferred API offer EU processing and storage on the plan you’d use, with retention you can control?
- Volume shape: estimate monthly tokens and how flat the curve is. Steady, batchable volume favours self-hosting; spiky, interactive volume favours APIs.
- Break-even: run the napkin sum above with real quotes, including engineering time.
- Quality bar: run your own evaluation set of 50-200 real examples through a candidate open-weight model and through an API model. Compare error rates, not impressions.
- Latency target: define acceptable time to first token and total time at peak concurrency, then test both options against it.
- Ops capacity: who patches, monitors and upgrades the model server at 2am? If nobody, don’t self-host yet.
- Licence: confirm the open-weight model’s licence permits your use, in writing.
- Exit: can you swap models or providers without rewriting the application?
If this were our business, here’s what we’d do first
We would start on an API with EU data residency switched on and the DPA signed. We would put a thin internal gateway in front of it so every model call goes through one place, and keep personal data out of prompts wherever pseudonymisation or field-level redaction makes that possible. That gets a working system in front of users in days and produces real data on token volume, latency and error rates. Next, we would build an evaluation set from real cases and run a candidate open-weight model against it in parallel, without users seeing the output. Only when one specific workload clears the bar would we consider self-hosting: high steady volume, data that can’t leave the boundary, and an open-weight model that matches the API on your own evaluation. Even then, the first step would be a managed EU deployment, with our own hardware reserved for when the numbers clearly justify it. This order means you never buy GPUs on a hunch, and you never ship personal data somewhere you can’t explain to a regulator.
This is the work D3X Solutions LABS does from Ireland. We design the gateway and routing layer, run the evaluation, do the break-even and GDPR analysis against your actual workloads, and build whichever hosting option the evidence supports. We let your data pick the option, not our preferences. If you’re weighing this decision for a real system, book a free 30-minute call using the form below and we’ll work through your volumes, data classes and constraints together.
Frequently Asked Questions
Is it cheaper to self-host an LLM or use an API?
For low or spiky volume, an API is almost always cheaper because you pay nothing when idle. Self-hosting only becomes cheaper with high, steady token volume that keeps GPUs busy most of the time. Once you include engineering and operations time, the break-even point is usually higher than people expect.
Can I use ChatGPT or Claude APIs under GDPR?
Yes, provided you have a data processing agreement with the provider, a valid transfer mechanism if data leaves the EEA, and appropriate settings for retention and training use. Many providers offer EU data residency on business plans, which simplifies the transfer question. You remain the controller, so lawful basis, minimisation and a DPIA where needed are still your responsibility.
Does self-hosting an LLM make me GDPR compliant?
No. Self-hosting removes the model provider from the processing chain and avoids cross-border transfers, but every other GDPR obligation still applies. You also become responsible for securing and deleting prompt logs yourself. If you host on a cloud GPU provider, that provider is still a processor you need a contract with.
Are open-weight models good enough for business use?
For well-defined tasks such as extraction, classification, summarisation and structured output, many open-weight models perform well. They still trail frontier API models on long multi-step reasoning and complex agentic work. The only reliable way to know is to test candidate models on a set of your own real examples.
What hardware do I need to self-host an LLM?
It depends on model size and concurrency. Smaller and quantised models can run on a single modern GPU, while very large models need multi-GPU servers. Size for your peak concurrent requests and latency target, not your average load.
What is a hybrid LLM setup?
A hybrid setup routes each request to the most suitable model behind a single internal gateway. Requests containing personal or confidential data go to a self-hosted or EU-dedicated model, and the rest go to the strongest available API. It balances data control, quality and cost, and makes switching providers much easier.
Want this working in your business?
D3X Solutions LABS builds AI agents and automation that pay back fast. Not sure where to start? A free 30-minute call is enough to find the first thing worth building.
Get in touch
