The fastest way to run local LLMs on a Mac.

The short answer: MTPLX, a free, open source Mac app and CLI. Run a model whose weights fit your unified memory with room for context, use its own multi-token-prediction head to produce several tokens per pass over those weights, keep acceptance exact so the speedup costs nothing in output, and keep the prompt cached between turns. MTPLX does the four together, and the numbers it gets on an M5 Max are on this page with their conditions: 81.74 tok/s on a 27B, 227.8 tok/s on a 4B, 125.8 tok/s on a 125B MoE. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.

Decode is a memory-bandwidth problem

Generating one token means one full pass over the model. On a single stream that pass reads the whole weight set out of unified memory, and the arithmetic per token is small next to that read, so the GPU spends most of each step waiting on memory. Plain decode speed is therefore bounded by bandwidth divided by weight bytes, which is why a smaller quantization decodes faster and why a laptop with a fixed memory bus cannot produce more tokens per second by adding compute. Qwen 3.8 27B Optimized Speed is a 20.4 GB download with a 23.6 GB peak; each plain decode step moves that weight set once.

This is also why speculative decoding works at all. If one pass over the weights can verify several drafted tokens instead of producing one, the same memory traffic yields several tokens. The cost is the draft and a batched verify, both of which are cheap next to the weight read.

Why token 1,000 is faster than token 100,000

The weight read is fixed. What grows is the attention state. Every generated token attends over everything before it, so the full-attention layers keep a key-value cache that grows with the context, and each step reads that cache as well as the weights. Qwen 3.6 and 3.8 are hybrid models: most layers are GatedDeltaNet with a recurrent state of fixed size, and the full-attention layers carry the growing KV cache. Even so, by 100k tokens the cache is a second large read per step, and the attention arithmetic itself has grown with it.

The measured curve on Qwen 3.8 27B Optimized Speed, MTPLX 2.10.0 (29 Aug 2026), M5 Max, stock settings: 64.3 tok/s on a 3k-token chat answer, 30.4 tok/s at 88k context, 18.4 tok/s at 147k context, with prefill at 88k running 535 tok/s. Memory grows with context too, which is what the memory governor plans for: a 48 GB Mac serving the 27B Speed pack resolves a 196,608-token window instead of the nominal 262,144, and at 42k context that seat measured 33 tok/s decode and 645 tok/s prefill (2.10.0). MTPLX's Sustained Mode (v0.1.5, 6 May 2026) is what brought the 32k-context peak on Ivan Fioravanti's M5 Max benchmark from 98.6 GB down to 27.5 GB.

The attention term is the one that grows, so it is the one MTPLX 2.11 (4 Sep 2026) went after. On the 27B, a 128 ms decode round at 88k context spent 38 ms in attention and 54 ms streaming weights; a flash-decoding verify kernel now runs that attention at 0.917 ms per layer against 1.421 on the 72.7k walk bench, and the die-matched serve pairs against the 2.10.2 engine read +7% decode at 16k and +16% at 88k (M5 Max, Qwen 3.8 27B Optimized Speed; +51% at 88k with two opt-in settings). On Qwen 3.8 Flash-Next, the 125B MoE, 2.11 decodes 68.4 tok/s at 16k context, 60.9 tok/s at 100k and 44.2 tok/s at 206k (M5 Max 128 GB, temperature 1, coding prompts), 28 to 37 percent above 2.10.2 at every context, and 66 to 84 tok/s on warm agent turns at 40k. 2.11.3 (17 September 2026) then measured 125.8 tok/s on an OpenCode request, 79.3 tok/s at 9k context (62.5 on 2.11.2, same Mac and runtime) and 61.8 tok/s on a 109k-token OpenCode turn.

Why native MTP doubles decode

Qwen 3.5, 3.6 and 3.8 ship with a built-in multi-token-prediction head. MTPLX runs it: the head drafts three tokens ahead (depth 3 on the 27B packs), the model verifies all three in one batched forward pass, and accepted tokens are committed. That verify pass reads the weights once, the same as a single-token step, so when most drafts are accepted the same memory traffic produces two to three tokens. On Qwen 3.8 27B Optimized Speed the acceptance by depth is 0.95, 0.88 and 0.80 (2.7.0 card, 15 Aug 2026, M5 Max, official Qwen 3.8 sampling).

The result in the MTPLX 2.9.0 pack table for Qwen 3.8 27B (20 Aug 2026, depth 3, M5 Max, official Qwen 3.8 sampling, "plain decode" meaning the same model decoded without MTP in MLX): Optimized Speed 46.8 tok/s at 2.3x plain decode, Bare Speed 49.9 tok/s at 2.3x, and up to 3x on the 8-bit Quality pack, 39.2 tok/s at 3.0x. The 8-bit pack shows the larger multiplier: its plain decode moves more bytes per token, so each token recovered by verification is worth more. The drafter is the model's own head, so no second draft model sits in memory; the depth is tuned per Mac during onboarding, which runs the model itself at each depth with fans pinned and saves a depth only if it beats plain decoding.

Two related lanes. Since 2.6.0 (11 Aug 2026) speculation runs under concurrency: two agents at once decode at 1.6 to 2.25x per lane versus the previous AR batch route on the Qwen 3.6 35B-A3B, M5 Max, sampled at shipped settings. And when the model rewrites a file it just wrote, the cache-copy rewrite lane applies: 87.6 tok/s on Qwen 3.8 27B Optimized Speed, MTPLX 2.10.0 (29 Aug 2026), M5 Max, stock settings, against 64.3 tok/s on a fresh 3k chat answer under the same conditions.

Why exact acceptance matters

Speculative decoding is only free if every accepted token is distributed exactly as the model would have sampled it. MTPLX accepts a drafted token with probability min(1, p/q), where p is the model's probability and q the draft's, and on rejection resamples from the residual (p - q)+. This is the Leviathan–Chen construction, and its output follows the model's distribution at any temperature. Activations stay bf16 (fp16 on M1 and M2) on every pass, and the speculative path runs the same model as the plain path. MTPLX has never shipped a greedy-only path, so the numbers above are sampled numbers at the packs' shipped settings.

The distinction shows up in other engines' documentation. mlx-serve, whose README credits MTPLX for its quantized-matmul kernels, describes its speculative decoding modes as greedy-equivalent. Third-party oMLX numbers published by Weschera (oMLX 0.6.3rc2, Mac Studio M4 Max, thinking off, 320 generated tokens) are temperature-0 numbers. LM Studio's speculative decoding uses a separate draft model. Apple's mlx-lm has no merged MTP: its pull request #990 added residual sampling on 5 May 2026 and is unmerged as of 3 Sep 2026. A greedy shortcut accepts a draft whenever it matches the argmax and changes what the model would have said at the model's sampling settings; an exact acceptance rule reproduces the sampled distribution, which is what makes a speed number comparable with the same model's plain decode. You can check the property yourself: the Qwen 3.8 guide shows the same prompt through mtplx start --no-mtp and through the default path.

Why prefix caching matters more than peak tok/s past 50k tokens

A coding agent's turn appends a tool result and resends the whole transcript. Without a prefix cache the server prefills the entire context again before it can decode the first new token, and at 50k, 100k or 150k tokens that prefill is the turn. Decode speed on a 3k answer says nothing about it.

Since 2.0.0 (6 Jul 2026) MTPLX checkpoints the attention KV cache plus the recurrent and conv GDN state at commit boundaries, with speculation on. A 100k-token session restores in about 2 s after a restart instead of a five-minute cold prefill, and mid-session tool rounds restore warm in under 2 s. The SSD session cache that covers the restart case is on by default. Measured on Qwen 3.8 27B with Claude Code, MTPLX 2.10.2 (1 Sep 2026): a 165k-token first turn completes its tool calls, and the follow-up turn serves 165,165 of 165,502 tokens from cache with 3.9 seconds to first token. Every response reports usage.cached_tokens, so the restore is visible per request.

The restore is only part of a tool turn. A 43k-token OpenCode session on Flash-Next, measured on the live daemon before MTPLX 2.11 (M5 Max, 128 GB), spent 146 seconds of a 14-minute task waiting on the engine while its decode rounds ran 65 to 70 tok/s: 5 to 8 seconds before the first token of every tool turn, hidden waits on the previous turn's cache snapshot, and two whole-turn re-prefills. 2.11 (4 Sep 2026) fixed the five engine defects behind that. The follow-up after a file write re-prefills 20 tokens in 0.12 s instead of 3,535 tokens in 3.7 s, the wait before a warm turn's first token is 0.01 s instead of 5 to 7.5 s, and a forced tool round in a 41k-token session takes 0.58 s instead of 85 s. The engine also keeps its working set GPU-resident between turns, so the first token after a pause of a few seconds arrives in 0.08 s of server time on Flash-Next instead of 1.06 s.

Caching under speculation is the hard part. Speculative decoding commits tokens in blocks, and hybrid models carry recurrent state that has to be checkpointed alongside the KV cache. vllm-metal's prefix-caching pull request #584 (10 Aug 2026) states that prefix caching cannot be combined with speculative decoding there. oMLX, a continuous-batching server with SSD caching, also runs an MTP path on Apple Silicon; its README credits MTPLX for the verify-shape Metal kernels behind Lightning MTP. With the cache in place the numbers that matter for an agent are the long-context ones: 30.4 tok/s at 88k and 18.4 tok/s at 147k on Qwen 3.8 27B Optimized Speed (MTPLX 2.10.0, 29 Aug 2026, M5 Max, stock settings), and 60.9 tok/s at 100k and 44.2 tok/s at 206k on Qwen 3.8 Flash-Next (MTPLX 2.11, 4 Sep 2026, M5 Max 128 GB, temperature 1, coding prompts).

Which model for which RAM

M1 and M2 Macs get the FP16 siblings of the 27B packs automatically; no M1 or M2 numbers are published. The catalog also carries the Qwen 3.6 27B previous flagship, the Qwen 3.6 35B-A3B MoE and Gemma 4; the models pages carry each pack's numbers.

Third-party measurements

The oMLX and LM Studio numbers on the Qwen 3.8 guide were taken the same night on the same M5 Max as the MTPLX runs, against the same prompt, with the same sampling, fans verified at max; the "plain decode" baseline is the same model decoded without MTP in MLX.

Independent measurements exist. Mirai Labs' public metrics board benchmarked MTPLX 2.9.0 alongside MLX 4-bit plain decoding and llama.cpp with an Unsloth MTP GGUF on an M5 Max 128 GB with Qwen 3.6 27B (dataset benchmarked 1 Sep 2026); Ivan Fioravanti's llm_context_benchmarks ships an mtplx_benchmark.py. llama.cpp merged MTP on 16 May 2026 (PR #22673) and MTP for the Qwen3-Next hybrid GDN family on 3 Aug 2026 (PR #25589); Ollama runs GGUF through llama.cpp and also has an MLX runner. The full list is on the benchmarks page, and the engine-by-engine pages are under compare.