Self-Host an LLM with Ollama on a VPS: Real Numbers From 6 Servers

Ollama is the easiest way to run open LLMs on a machine you control. One command installs it, one command pulls a model, and it speaks an OpenAI-compatible API. Your prompts stay on your box, and no per-token meter is running.

The question everyone argues about is whether running it on a rented server is worth it. The top result for "self host ollama" is literally a Reddit thread asking that. The answers are opinions. Mine were too, and I run most of my stack self-hosted, so I wanted real numbers.

So I measured it. Six servers, same model, same benchmark scripts, one day: tokens per second alone and under concurrent load, exact VRAM per model, the point where each box falls over, and what a million tokens actually costs on each one. Along the way I hit four traps that no install guide mentions. All four are documented below with their fixes.

The short answer

If you just want the verdict before the evidence, here it is.

A cheap CPU VPS runs an LLM for one person. llama3.2:3b on a $48/month droplet generates 5 to 7 tokens/sec. Usable for background jobs and personal tools. The 8B model on the same box crawls at 2.8 tokens/sec. And no CPU box can serve two people at once: throughput stays flat around 7 tokens/sec total while latency explodes.

A GPU changes the game completely. The cheapest GPU droplet DigitalOcean rents (RTX 4000 Ada, 20 GB VRAM, $0.76/hour) ran the same 8B model at 63 tokens/sec for one user, and served 16 concurrent users at reading speed. The H100 served 32 at once, at 529 tokens/sec combined.

On cost, the API usually wins. My best box produces tokens at $0.84 per million, but only when fully busy. gpt-4o-mini costs $0.60 with zero servers to babysit. Self-hosting wins on privacy, on flat predictable cost, and on control over your own stack. It rarely wins on cost per token. Now the evidence.

Pick your server: the six boxes I tested

All six are stock DigitalOcean droplets anyone can rent today. Three CPU, three GPU:

Two practical notes from creating these. GPU droplets live in fewer regions (mine were Toronto; my CPU boxes were Frankfurt), and they boot from DigitalOcean's "NVIDIA AI/ML Ready" image, which ships Ubuntu 22.04 with the driver and CUDA preinstalled. Nothing to install for the GPU: nvidia-smi worked 70 seconds after I clicked create.

One measured surprise worth your money: on CPU, dedicated cores beat shared cores by about 30% at the same core count. The c-8 ran the 8B at 6.8 tokens/sec against 5.4 on the shared 8-vCPU box. Shared vCPUs time-share a physical core with the neighbors; sustained inference is exactly the workload that notices.

Install Ollama and pull your models

The install is genuinely one command, and it behaves correctly out of the box: the API binds to localhost only, so nothing is exposed yet. This is my timed run on the fresh 16 GB droplet:

$ curl -fsSL https://ollama.com/install.sh | sh
>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
>>> Creating ollama user...
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.
# 19 seconds, start to finish. On the GPU boxes the same script found CUDA on its own.
$ ollama pull llama3.2:3b && ollama pull llama3.1:8b
# 21s and 41s. DigitalOcean pulls the model registry at about 230 MB/s (1.3 GB/s on the GPU boxes).
$ ollama list
NAME           ID              SIZE      MODIFIED
llama3.1:8b    46e0c10c039e    4.9 GB    Less than a second ago
llama3.2:3b    a80c4f17acd5    2.0 GB    41 seconds ago
$ curl -s localhost:11434/api/generate -d '{"model":"llama3.2:3b","prompt":"Say hello in five words.","stream":false}'
"response": "Hello, how are you today?"

That is a working, private LLM. Under a minute and a half on a rented box, including both model downloads.

Expose the API safely (HTTPS + key)

Ollama has no authentication. None. Anyone who can reach port 11434 can run your models, read your prompts, and delete your downloads. And a fresh server gets found fast: when I once left a new droplet exposed on purpose, the first scanner hit it in under 3 minutes.

The safe setup is three layers, and the whole thing takes about five minutes:

Layer 1: leave Ollama on localhost. The default. Don't set OLLAMA_HOST=0.0.0.0, whatever a tutorial tells you.

Layer 2: firewall. Only SSH and the web ports are open:

ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable

Layer 3: Caddy with HTTPS and an API key. Point a DNS A record at your server, install Caddy, and use this config. Caddy fetches the certificate on its own (Let's Encrypt, free; mine was issued 4 seconds after reload):

ollama.example.com {
    @authed header Authorization "Bearer YOUR-LONG-RANDOM-KEY"
    handle @authed {
        reverse_proxy 127.0.0.1:11434 {
            header_up Host 127.0.0.1:11434
        }
    }
    handle {
        respond "unauthorized" 401
    }
}

Verified from outside the box, all three checks:

$ curl -s -o /dev/null -w '%{http_code}' https://ollama.example.com/api/version
401
$ curl -s -H "Authorization: Bearer $KEY" https://ollama.example.com/api/version
{"version":"0.32.9"}   HTTP 200
$ curl -s --max-time 8 http://SERVER-IP:11434/api/version
unreachable (good, the firewall is doing its job)

If you would rather not expose the API at all, there is an even simpler pattern for a single developer: an SSH tunnel. ssh -L 11434:127.0.0.1:11434 root@your-server and your laptop's localhost:11434 IS the server's Ollama. No ports opened, no proxy, no key. I use both, and the app section below shows each one working.

The benchmarks: CPU vs GPU, same model

Same llama3.1:8b, same prompts, six boxes. One chart:

The shape of this chart is the whole CPU-vs-GPU decision. The three CPU bars are not "slower". They are a different category: 2.8 to 6.8 tokens/sec means watching words appear one by one. The cheapest GPU is 9 times faster than the best CPU box, and the H100 is 38 times faster than the $96 droplet.

Two model-ladder numbers worth knowing. The small llama3.2:3b is the honest CPU recommendation: 12 to 15 tokens/sec on the dedicated box, real reading speed, for $168/month. And OpenAI's gpt-oss:20b, a 13 GB download, ran at 74.6 tokens/sec on the $0.76/hour GPU. A 20-billion-parameter reasoning model, faster than the 8B dense model, because only a fraction of its experts fire per token. VRAM proof from my run:

$ nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
NVIDIA RTX 4000 Ada Generation, 20475 MiB, 580.173.02
$ curl -s localhost:11434/api/ps | python3 -m json.tool
"name": "llama3.1:8b"
"size":      5271715839
"size_vram": 5271715839   ← 100% of the model in VRAM, zero CPU spill
"context_length": 4096
$ nvidia-smi --query-gpu=utilization.gpu,memory.used --format=csv,noheader,nounits
95, 5226
# mid-generation: 95% GPU utilization, 5.1 of 20 GiB used

How many users can one box handle?

This is the question that decides real deployments, and it is the one no guide measures. Ollama processes OLLAMA_NUM_PARALLEL requests at once and queues the rest. So I set that to each test's stream count and pushed every box up a ladder: 1, 2, 4, 8, 16 streams, and 32 on the H100.

Read the left panel first. The CPU line is flat: 7 tokens/sec total whether one person or four are asking. CPU inference is bound by memory bandwidth, so concurrency just divides the same speed between users, and the right panel shows what that does to their waiting time: p95 latency of 128 seconds at 4 streams. Four users on CPU is not slow. It is broken.

The GPUs scale the way you would hope. The $0.76/hour card climbs to 252 tokens/sec at 16 streams, every user still getting 13 to 30 tokens/sec. The H100 reaches 529 tokens/sec at 32 streams, p95 under 9 seconds for full 150-token answers:

offload_check[level=32]: offloaded 33/33 layers   (NUM_PARALLEL=32, CONTEXT_LENGTH=1024)
$ python3 bench_concurrent_level.py --model llama3.1:8b --level 32 --requests 3
"ok": 96, "failed": 0
"agg_eval_tok_s": 528.86
"per_stream_tok_s": [17.4 ... 22.7]   # all 32 users at reading speed
"ttft_p95_s": 0.89
"latency_p95_s": 8.73

Getting to those numbers taught me the two GPU lessons of this whole project.

Put together, here is the capacity table I wish had existed before I started. "Comfortable" means the biggest tested load where p95 time-to-first-token stayed at or under 3 seconds and p95 latency at or under 12 seconds. "Sustained users" assumes an active chat user sends 8 requests an hour at about 220 output tokens each:

The row to stare at is the RTX 4000 Ada. For a SaaS with a few hundred active users of an 8B-class feature, the cheapest GPU droplet DigitalOcean sells is genuinely enough. That surprised me.

Is it cheaper than the API?

Take each box's best measured throughput, divide the hourly price by tokens per hour, and you get an honest dollars-per-million-tokens figure to hold against the API price lists:

Three honest readings. CPU self-hosting never wins on cost: $4.41 to $9.04 per million tokens against $0.60 for gpt-4o-mini. The budget GPU at $0.84 per million beats Claude Haiku 4.5 by 6x and gpt-5-mini by more than double, but still loses to gpt-4o-mini. And every self-hosted number assumes the box is busy. At 10% utilization, multiply by 10.

So why do I self-host anyway? Because the meter is the point. A flat $48 or $555 a month is a number you can plan a product around, and nobody re-prices it mid-quarter. Your prompts and your users' data stay on hardware you control. And an 8B model you own completely is enough for summarization, extraction, tagging, drafts, and most internal tools. The same logic that moved my whole stack off managed clouds applies to the model layer too.

What specs do you actually need

The same data, folded into three honest recommendations:

Just you, personal tools, background jobs: the $48 CPU box. Run llama3.2:3b, not the 8B. 5 to 7 tokens/sec is fine for automations that nobody watches (summaries, tagging, drafts landing in your inbox). Add a 2 GB swapfile, stick to one model, and it will quietly do its job. If you already run n8n or PyRunner on a VPS, this is the same move for the model layer.

You want real chat speed, or a small team: the RTX 4000 Ada at $0.76/hour. 63 tokens/sec single-user, 16 people served at once, room for gpt-oss:20b at 74 tokens/sec. If it only needs to run during work hours, snapshot it and power it off at night: GPU droplets bill hourly, so 8 hours a weekday is about $130/month.

Production SaaS feature for hundreds of users: start with one 4000 Ada, and know your two knobs. Set OLLAMA_NUM_PARALLEL to your real concurrency, size OLLAMA_CONTEXT_LENGTH so context times slots fits VRAM, and load-test with the ladder pattern from this guide before launch. Past ~500 active users, an H100 at $2.32 per million tokens is still cheaper per token than Claude Haiku, or you scale sideways with two smaller GPUs behind a load balancer.

Consume it from a real app

The part that makes this a product feature and not a toy: your app talks to your server. Ollama ships an OpenAI-compatible endpoint at /v1, which means every OpenAI SDK on earth already supports your private LLM. Change the base URL, pass your Caddy key as the API key, done.

I verified it three ways from my laptop against the secured server. First with SimplerLLM, my open-source Python library, through both routes (the public HTTPS endpoint and the SSH tunnel), then with a raw curl so you can see exactly what any SDK does under the hood:

$ python consume_ollama.py
=== Route A: OpenAI-compatible endpoint (HTTPS + API key) ===
    base_url = https://ollama.example.com/v1
    [60.0s] The main benefit of self-hosting a Large Language Model (LLM) is
    that it allows for full control and customization over the model's
    functionality, data usage, and deployment environment ...
=== Route B: native Ollama provider via SSH tunnel ===
    OLLAMA_URL = http://127.0.0.1:11435/
    [25.0s] The primary benefit of self-hosting a Large Language Model (LLM)
    is having complete control and ownership over the model ...
$ curl -s https://ollama.example.com/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Reply with exactly: SELF-HOSTED OLLAMA API OK"}]}'
"content": "SELF-HOSTED OLLAMA API OK"
"usage": {"prompt_tokens": 23, "completion_tokens": 10, "total_tokens": 33}

Those 60 and 25 second timings are the CPU box generating at its measured 2.5 tokens/sec, a nice accidental cross-check of the benchmarks. On the GPU box the same calls come back in about 2 seconds.

The Python side is four lines with SimplerLLM (any OpenAI client looks the same). Set OPENAI_BASE_URL=https://ollama.example.com/v1 in your environment first, then:

from SimplerLLM.language.llm import LLM, LLMProvider

llm = LLM.create(provider=LLMProvider.OPENAI, model_name="llama3.1:8b",
                 api_key="YOUR-CADDY-KEY")
print(llm.generate_response(prompt="Summarize this ticket: ..."))

My take

I went in expecting the GPUs to win on speed. They did. What I did not expect was how completely the "worth it" question splits into two different questions.

For cost per token, the API providers have won, full stop. They run these models at a scale and utilization you cannot match with one rented card, and gpt-4o-mini at $0.60 per million is the proof.

But that was never really the question. The question is whether $48 a month buys you a model that is yours: prompts that never leave your box, costs that never surprise you, a stack with no usage dashboard and no deprecation emails. Measured against that, yes. A 3B on a cheap droplet for your automations, or the $0.76/hour GPU serving 16 real users, both do honest work. In my experience the builders who self-host their LLM do it for the same reason they self-host anything: not to save pennies, to own the thing.

What this is part of

An LLM server is one tenant on a box you own. The full skill is running your whole stack this way: the analytics, the automations, the backups, the hardening, all next to each other without the tenants fighting. That is what I teach in Self Hosting 2.0: 34 lessons from a blank server to a stack you fully own, including the security playbook this guide applied in one section.

FAQ

• Updated August 2026
• Ollama 0.32.9
• Boxes tested 6 (3 CPU, 3 GPU)
• 8B speed 2.8–206 tok/s measured
• Best value RTX 4000 Ada, $0.84/1M tok
• Difficulty Intermediate

Related

• Self Hosting Hub every self-hosting tutorial on the site
• What Is Self-Hosting? the hub entry point, proven on a $4 droplet
• Secure Your VPS the hardening pass every public server needs
• Free SSL Certificates the Let's Encrypt lifecycle behind the Caddy magic here

Own your model.

Hasan Aboul Hasan builds open-source tools and teaches solo developers how to build, host, and sell AI-powered products. Founder of LearnWithHasan.com, creator of SimplerLLM and PyRunner.