When a local LLM is worth it, and when the cloud wins

"Should we run our own AI model, or just use a cloud provider?" is one of the first questions we hear. The honest answer is that it depends, and that the deciding factor is usually not the one people expect.

This is how we think about it, with numbers you can check.

The short answer

• Run a model locally when your data really can't leave, when the work is routine (reading, sorting, drafting, answering from your own documents) and when you want costs that don't depend on usage.
• Use the cloud when you need the hardest reasoning available, when usage is low or spiky, or when you'd rather not run hardware at all.
• Most projects end up with both, with a simple rule that decides which requests go where.

Local models are not a cheaper copy of the big ones. They are a different trade: you give up some capability and get control in return.

Security vs capability

"Cloud is unsafe" is too simple. The large providers offer no-training terms, EU regions and data processing agreements. The real difference is how short the answer is to the question where did our data go?

If your documents contain patient records, contracts under NDA or anything you would not paste into a web form, the local column is worth a serious look. If they don't, the capability row usually wins.

What about €15,000 buys

A budget of around €15k buys one machine that can serve a handful of people at the same time. It is enough for the everyday work that makes up most of what companies want from AI:

• summarising and rewriting text
• pulling fields out of emails, PDFs and forms
• sorting and routing requests
• drafting replies in your tone and language
• answering questions from your own documents
• translating, including Dutch and Hungarian

What it is not good at: long chains of reasoning, hard maths, large-scale code work, or the best writing quality. For those, a top cloud model is still clearly ahead.

You don't need to spend €15k to start. The speed of a local model depends mostly on how fast the machine can read its own memory, so the hardware choice is a trade between memory size and speed:

(Memory size and speed are separate things. A big model fits into 128 GB of shared memory but can crawl. One test of a large 70-billion-parameter model on a Spark-class box managed only about 3 tokens a second, while smaller models or mixture-of-experts models feel instant.)

These prices move quickly, so treat the table as a starting point and check before you buy.

Qwen is a capable model

The open model family we use most is Qwen. It is a good example of how far open models have come. On our own machine we measured:

It also has limits. In one test it labelled a pure batch job as "mixed", which is the kind of judgement call a top cloud model would get right more often. That is why we put exact things (dates, prices, calculations) in code and keep a person in the loop where it matters.

You are looking at the proof: the assistant on this website runs on a model on our own hardware in the Netherlands. How we built it explains the details.

Does it pay off?

Here we have to be careful, because "local is cheaper" is only true in some cases.

We compare a local machine with three kinds of cloud model, using the published list prices (USD per million tokens, October 2026):

*Blended at four input tokens for every output token, which is typical for agents that read a lot and write a little.

For the local machine we assume €15,000 spread over three years, plus about €65 a month for electricity (300 W, running all day at €0.30 per kWh). That is about €480 a month. We treat a dollar as a euro; the real exchange rate shifts the lines a little but doesn't change the picture.

What this tells us:

• Against a small cloud model, local almost never wins on cost. Small models have become very cheap. If a small model can do your job, use it.
• Against a mid-tier model, you need real volume: about 130 million tokens a month. At roughly 5,000 tokens for each request, that is around 900 requests a day.
• Against a top model, local breaks even early, but the comparison is unfair. A local model can't do what the top tier does, so you can't swap one for the other.
• Prompt caching and batch discounts make cloud even cheaper than these list prices. Your own time to run a machine is not included either.

So local rarely wins on price alone. It wins on privacy, on predictable costs (an agent that loops by mistake runs up electricity, not a bill) and on independence from a provider's prices and policies.

One more number. At our measured 69 tokens a second, a single stream can produce about 180 million tokens of output a month, and a machine like this serves several streams at once. Capacity is not the limit for most small teams.

The middle path

You don't have to choose one. A common setup is:

• Local for the sensitive and routine work: reading your documents, extracting data, drafting.
• Cloud for the few requests that need top-level reasoning, with any private details removed first.

The important part is that code decides the route, based on the kind of data, not the model itself. That keeps the rule simple and checkable.

A quick way to decide

Go local if at least two of these are true:

1. The data is sensitive and shouldn't reach a third party.
2. The work is routine and a mid-sized model does it well enough.
3. You expect steady, high volume, or you want a fixed monthly cost.
4. You want to stay independent from one provider.

Go cloud if you need the best reasoning, usage is small or unpredictable, or nobody wants to run a machine. If you are somewhere in between, start in the cloud with the data that is safe to send, and add a local model when a real need shows up.

Not sure which side your project falls on? Ask the assistant. It will ask a few questions about your data and your tasks, and tell you plainly if cloud is the better fit.

Sources and assumptions

• Cloud prices: Claude platform pricing, list prices checked in October 2026. Other providers' small, mid and top tiers are in a similar range, but check their current price lists.
• Hardware: NVIDIA DGX Spark price history and a review of its speed limits. Prices and speeds vary by source and software.
• Speed rule of thumb: how fast a model answers is roughly the machine's memory speed divided by the model's size.
• Our own measurements come from the assistant on this site, in October 2026.
• Costs exclude staff time, the cost of an outage, and discounts negotiated with providers.