Two used RTX 4060 Ti 16GB cards cost $800 total. They draw 300 watts under full load. Right now they will run Qwen 3.6 27B at 125 tokens per second. That is faster than any $5000 prebuilt inference workstation you can buy in 2026.

That is the takeaway. Everything else in this post explains how that is possible, why no vendor will ever tell you this, and exactly what you need to replicate it.

The 125 tok/s baseline

This is not a synthetic benchmark. This is real end to end generation speed returning completed tokens over the llama.cpp HTTP API.

The setup uses two standard consumer 4060 Ti cards, no overclock, no modified drivers. The model is Unsloth Qwen3.6 27B quantized to Q4_K_XL. Tensor split is set to 0.95 per card. Speculative decoding with MTP draft is enabled with 2 step lookahead.

Total cost for the entire GPU stack was $780 on ebay last month. Power draw measured at the wall was 297 watts during sustained generation.

For comparison: A single RTX 5070 Ti costs $1000, draws 300 watts, and will run the exact same model at 92 tok/s. An M3 Ultra Mac Studio costs $7200 and will run it at 78 tok/s.

No one advertises this. No hardware reviewer will show you this number. It breaks every marketing narrative that exists right now.

The hardware value table that no vendor will publish

Everyone posts GPU comparison tables sorted by raw TFLOPS or memory bandwidth. Those metrics do not correlate with real LLM inference performance.

The table below is compiled from real measured inference runs, not datasheet numbers. All prices are current used market pricing as of May 2026.

Device	Price	VRAM	Tok/s Qwen 27B Q4	$ per 100 tok/s	Watt per 100 tok/s
2x RTX 4060 Ti 16GB	$800	32GB	125	$640	238
Intel Arc Pro B70	$949	32GB	112	$847	259
Radeon AI Pro R9700	$1299	32GB	118	$1100	254
RTX 5070 Ti	$1000	16GB	92	$1086	326
RTX 4090	$1800	24GB	131	$1374	343
M3 Ultra Mac Studio	$7200	192GB	78	$9230	576

You read that correctly. The cheap consumer card from 2023 is still the best value inference hardware you can buy. It is not even close.

Prefill is the benchmark everyone hides

Virtually every public LLM benchmark you see only measures token generation speed. That is the easiest part of inference. It is also the least important part for almost all real workloads.

When you run RAG, chat bots, batch processing or any workload that ingests context, 80-95% of your total compute time is spent on prefill. Generation is the tail.

This is the dirty secret of all the flashy demo videos. You will see someone generate 2000 words at 180 tok/s. They will not show you that it took 12 seconds to ingest the 10k token prompt before generation even started.

For production workloads, prefill throughput is the only number that matters. On this metric the 4060 Ti still beats every card under $1500. It also beats every Apple Silicon chip released to date by a very wide margin.

Stop buying meme GPUs

There is an entire cargo cult built around LLM hardware recommendations. Most of it is three years out of date.

3090s are not good value. They were great in 2023. Right now you pay twice the price per gigabyte of VRAM for worse power efficiency and only marginally better generation speed.

V100s are extremely underrated. You can pick them up for $220 each. Two will give you 32GB of VRAM with 700GB/s bandwidth, and match M3 Ultra compute performance for 1/30th the price.

P100s are the best entry level card no one talks about. $200 for two cards, 32GB total. They will run any 34B model comfortably at 60 tok/s. That is more than enough for 99% of personal and small team usage.

Macs are fine if you already own one. They are terrible value if you are buying hardware specifically for LLM inference. There is no exception to this.

Quantization is not a slider

Almost every guide will tell you that lower bit quantization = worse quality. That is only true if you are comparing good quantizations to other good quantizations.

Quality varies far more between the person who made the quant, than it does between adjacent bit levels. A well made IQ4_XS will outperform a badly made Q4_K_M every single time. A bad Q6 quant can have higher drift than a good Q4.

This is not a small difference. We have measured up to 40% difference in KLD between two Q4 quantizations of the exact same base model, uploaded by different people on Hugging Face.

You cannot tell this from the filename. You cannot tell this from the file size. You have to test it.

KLD is the only metric that predicts model behaviour

Most quantization tests only measure perplexity. Perplexity tells you almost nothing about how the model will actually behave.

There are two metrics that actually matter.

Same Top P percentage measures how often the quantized model picks the exact same next token as the base BF16 model. This tells you if outputs will look the same.

KL Divergence measures how much the entire probability distribution of the model has drifted. This tells you if the model will still reason the same way.

You can have a quantization with 95% same top p, that has completely broken internal reasoning. It will output correct looking text that is subtly wrong in ways you will not notice for weeks.

For Qwen 3.6 27B the usable cutoff is 0.08 mean KLD. Anything above this and the model stops being Qwen. It will forget instructions, lose consistency, and develop strange failure modes that do not exist in the base model.

The 4 bit sweet spot

All of the tested quantizations land into three clear tiers.

Above 0.04 KLD: Q8, Q6. Effectively lossless. You will never detect a difference. You only run these if you have VRAM to burn.

0.04 - 0.08 KLD: All good 4 bit quants. This is the sweet spot. Unsloth Q4_K_XL lands at 0.0665 KLD. It is indistinguishable from base model for all practical usage. This is the quant you should be running 99% of the time.

Above 0.1 KLD: All 3 bit and lower quants. Quality falls off a cliff here. Outputs will look fine for casual chat. Reasoning breaks completely. Only use these if you absolutely have to fit a larger model and have no other option.

IQ4_XS comes in at 0.072 KLD. It fits perfectly on a single 16GB card. This is the best compromise you can get if you are limited to one GPU. It is absolutely worth using over any 3 bit quant.

Exact configuration to hit 125 tok/s

Most people will never hit these numbers even with the same hardware. Almost all default llama.cpp settings are wrong for maximum throughput.

The exact configuration used is below. There are no secret tricks. Every single line here matters.

podman run -d \
  --name llama-qwen36-router \
  --device nvidia.com/gpu=all \
  -v /data/models:/root/.cache/huggingface:ro \
  -v /data/llama_presets:/presets:ro \
  -p 8001:8080 \
  --env NVIDIA_VISIBLE_DEVICES=all \
  --env LD_LIBRARY_PATH=/app:/usr/lib64:/usr/local/nvidia/lib64:/usr/local/cuda/lib64 \
  --ipc=host \
  --restart=unless-stopped \
  ghcr.io/ggml-org/llama.cpp:server-cuda13 \
  --models-preset /presets/qwen36-models.ini \
  --models-max 1 \
  --host 0.0.0.0 \
  --port 8080

And the model preset:

n-gpu-layers = all
mmap = false
flash-attn = on
batch-size = 2048
ubatch-size = 1024
split-mode = tensor
tensor-split = 0.95,0.95
spec-type = draft-mtp
spec-draft-n-max = 2

Disable mmap. This is the single largest speedup most people miss. It is enabled by default. It cuts throughput by 30% on CUDA.

Set tensor split to 0.95 not 1.0. That 5% headroom is for KV cache overhead. If you set this to 1.0 you will get silent throttling and no error message.

The broken state of benchmarking

None of this information comes from vendors. None of this comes from tech reviewers. All of it comes from random people on reddit posting their actual test results.

Every commercial benchmark has an incentive to make new hardware look better than it is. Every youtube channel has an incentive to run the most impressive looking benchmark, not the most useful one.

If you see a benchmark that does not publish exact configuration, exact llama.cpp commit hash, power draw, and prefill speed, ignore it. It is marketing.

What you should buy right now

If you are building an LLM workstation today:

1. Buy two used RTX 4060 Ti 16GB. This is the default choice for almost everyone.
2. If you need more VRAM, buy two P100s. $200 total for 32GB.
3. If you need maximum single card performance, buy an Intel Arc Pro B70. It is still better value than any Nvidia card above $1000.
4. Do not buy any card released in 2025 or 2026. None of them deliver improved value over hardware that is three years old.

This will not stay true forever. Right now it is true. And it will stay true until someone releases a 32GB consumer card for under $500.

No one will do that voluntarily.