Releases.

Faster responses, smoother streaming, smaller model downloads, and one-click model updates.

• Faster generation. Decode is 15–20% faster on typical workloads and up to 60% faster on code-heavy output.
• Smoother streaming. Visible freezes fell by 95%, and the worst measured stall dropped from 725 ms to 109 ms.
• Much smaller updates. Existing models now update with a 240–450 MB delta instead of downloading 15–21 GB again.
• Smaller models, updated in one click. Qwen 3.8 packs are up to 610 MB smaller, and the app now shows and installs model updates directly.

The streaming-quality release. 2.8.2's field reports kept coming back to one thing: chats that freeze for half a second and land in bursts — but only when a human was actually using the app. Hands-off testing stayed clean, which is exactly how it survived. 2.8.3 closes that whole class, plus every streaming bug found on the way to it.

• Streaming stays smooth while you touch the app. The window-measurement guard ran in a run-loop phase macOS skips while input events keep arriving — precisely when you scroll or move the mouse. Now it runs every turn: 40 s of continuous wheel-scrolling went from 70 UI stalls (18.7 s frozen) to one, and 30 s of cursor movement from 91 stalls to zero.
• Scrolling up mid-generation no longer fights you. User scrolling wins instantly — trackpad, momentum, and classic wheel mice alike — and following re-engages when you return to the bottom.
• The transcript can't go blank, thinking is plain text, tables stream correctly. The mid-generation blank-out, the self-rewriting reasoning ticker, and whitespace-free freezes (tables, URLs, minified code) are all fixed, with a quadratic detokenizer path closed on exactly that content.
• Every request records a stream-smoothness census. Producer gap percentiles land in every request record — including cancelled runs, which used to log nothing and were exactly the runs people complained about.

The 2.8 line. This release is about trust: people started benchmarking MTPLX seriously and running long agentic sessions against it, and both groups found real problems. 2.8 closes that work, and 2.8.1 is the build that ships it to the desktop, together with a vision cache fix our own release gate caught the same morning.

• Long agent sessions no longer stall their cache. Sessions used to quietly stop reusing their prompt cache around 38k tokens and re-prefill a growing suffix every turn. The committed frontier now advances on every turn, tool-call turns included, so a 45k-token conversation keeps turn-delta prefills only.
• Every number the server reports is one you can bench against. mtplx_stats is always populated, temperature 0 is exact, the logprobs contract is parser-safe, and /health now reports any degradation (compiled-verify fallbacks, overridden profile keys, kernel bails) instead of looking like turbo while running slow.
• A different image can never read another image's cache. The 2.8.1 fix: cache keys for vision turns are derived from the actual image bytes, and the raw session frontier no longer commits image histories. Identical pixels still restore the full prefix; different pixels stop cold before the image.
• Sharing the API over your network is a one-liner. mtplx serve --host 0.0.0.0 --api-key-file ~/.mtplx/api-key creates the key on first run and prints the LAN base URL other devices should use. Keyless non-localhost binds still refuse, on purpose.
• 256k context, for real. The flagship's full 262,144-token window fits and runs on a 128 GB Mac: a 260k-token prompt completes end to end with MTP still engaged.

An emergency fix for mtplx pull. On 2.7.1 and older, re-pulling a model that changed upstream can corrupt your local copy. Upgrade before you pull.

• Pull no longer corrupts files that changed upstream. The downloader treated a complete local file whose size no longer matched the server as an interrupted download, and appended the remote tail onto the old content — corrupting config.json and the safetensors index. Stale files are now re-fetched whole; genuinely interrupted downloads still resume.
• Already hit by it? Delete the model's config.json and model.safetensors.index.json, then pull again on 2.7.2. The full notes have the details.
• The Qwen 3.8 models can see again. All six published 3.8 repos were re-published with their vision towers restored, and forge now grafts the tower on every build — failing closed rather than publishing blind. Existing installs pick the repair up as a ~0.9 GB delta.

A bug-fix release. It clears the known-issues list 2.7.0 shipped with.

• xhigh stays selected. Picking it in Inference settings snapped back to medium, and mtplx config set reasoning_effort xhigh was refused. Both places carried their own copy of the effort list and neither knew about xhigh, so the save was rejected whole and the picker reverted. Every place that accepts an effort level now reads the same list.
• KV cache quantization reaches Qwen 3.8. The toggle displayed q8 while the launch path recognized only Qwen 3.5 and 3.6.
• Honest diagnostics. mtplx doctor names the model it actually checked, and turbo's profile note reports the real 32,768-token compiled-verify fence.
• A new build can't offer you an older one. The build number derived for 2.7.1 came out below the shipped 2.7.0, so a fresh install proposed 2.7.0 to itself.

Qwen3.8 support 🎉. Qwen3.8-27B came out on 14 August; this release runs it the way the model card says, with three MTPLX builds tuned for it and FP16 versions of all three for M1 and M2 Macs.

• Qwen 3.8, served properly. Official sampler (1.0 / 0.95 / 20), reasoning effort xhigh, medium and low with medium as the coding default, thinking preserved in history. Bare Speed (16.0 GB), Optimized Speed (20.4 GB, recommended) and Optimized Quality (29.4 GB), each with its calibration stamped in its own metadata.
• New default. Macs with 32 GB or more now default to Qwen 3.8 Optimized Speed; M1 and M2 get its FP16 build automatically. Under 32 GB still routes to the 9B.
• Compiled verify to 32k. The compiled verify window moves from 12k to 32k tokens of context: +6.9% at 20k on Qwen 3.8 Bare Speed, peak memory flat at 20k and lower at 30k.
• Coding agents uncapped. OpenCode and Pi no longer send an output cap for MTPLX models; Pi sessions restore their banked prefix from RAM.
• Fixes. The SSD session cache no longer walks its whole store on every write or health poll (idle CPU 35% down to 0.2% on an 816k-file bank); the macOS 27 slider crash is fixed (thanks @joshlacal); mtplx pull names the mirror knob when huggingface.co is unreachable.

Concurrency. Speculative decoding used to be a single-user feature: the moment two requests hit the daemon at once, everyone fell back to plain batching. 2.6.0 removes that trade-off.

• Concurrent speculative decoding. --scheduler-mode mtp_batch serves independent requests through fixed-width MTP cohorts (three-wide and eight-wide) with per-row stats and honesty controls, and the session bank composes with it.
• Embeddings and reranking. New /v1/embeddings and /v1/rerank endpoints (Cyb3rb1ade).
• LiquidAI LFM2. LFM2 and LFM2.5 models run on MTPLX (davidtai).
• Temperature-0 exactness. A real correctness fix to greedy decoding under prefill partitioning, plus a serial-lane sampling speedup.

Agent sessions got the attention this cycle, especially Pi. Warm turns stay warm.

• Faster warm turns. Tool turns restore from the stable boundary instead of re-processing ~200 tokens per round; a postcommit about to finish is briefly waited for instead of thrown away (a 1,449-token re-prefill became 436 tokens, first token 2.7 s down to 1.1 s).
• No background stalls in your turn. SSD cache work no longer slips into the gap between a request's internal jobs; the SSD tier skips candidates that cannot win.
• The cache tells you what it is doing. Resolved cache budget printed at startup, per-session and total (#229, #230).

A small release focused on the agent lane and the API surface, from a day of head-to-head benchmarking.

• Agent requests no longer stall behind background cache work. Background commits yield the moment any request is admitted, whichever session it belongs to (worst measured case before: a follow-up turn 44% slower).
• Warm follow-ups got faster. A byte-identical transcript hits an exact-match encode cache and gets 77 to 92 ms back per request.
• API honesty. Fixes for places where the API misled external tools.

Hotfix for the 2.5.1 long-response slowdown. Long answers no longer start fast and decay: the speculative verifier now hands off cleanly when a response outgrows its compiled window.

• Long responses hold their speed. The compiled-to-eager verifier handoff settles all state once at the ownership boundary instead of dragging unfinished GPU work through the rest of the answer.
• Identical output. The fix changes execution order only. Tokens, acceptance, and peak memory are unchanged, and the handoff is visible in verifier stats so it cannot regress silently.
• Faster MLX for existing installs. The minimum MLX version is now 0.32, converging older runtime environments to the stack fresh installs already run.

The coding-default release. Qwen 3.6 27B Optimized Speed V2 is now the first recommendation on modern Macs with at least 32 GB of unified memory. The original Optimized Speed model stays directly below it.

• Much higher quality for coding. V2 uses dynamic 4-bit hybrid quantization with hand-tuned sensitive parts kept at up to 16-bit.
• Built for longer agent work. V2 gets stronger as coding tasks become longer. It is slightly larger and can be a little slower for short chats.
• First-class everywhere. Onboarding, the app picker, CLI defaults, quickstart, downloads, inspection, turbo profiles, and OpenCode all use one model identity.
• Memory-aware recommendations. Modern 32 GB Macs get V2 first. Smaller Macs keep the existing 9B and 4B recommendations.
• Focused and regression-gated. Runtime kernels, sampler defaults, and speculative depth are unchanged from 2.5.0. A cold-cache bookkeeping race is fixed, and the full Python, Swift, and signed-app gates passed.

The next-model readiness release. MTPLX is prepared for the shape of Qwen 3.8, coding-agent bridges survive real multi-file work, and new experimental performance lanes land without changing safe defaults.

• Prepared for Qwen 3.8. Multi-layer MTP heads and checkpoint-declared architecture classes remove the known integration blockers; real-weight validation begins when the weights publish.
• Tool calls survive real work. OpenCode CLI completed a ten-action multi-file change with 31 tests passing, OpenCode Desktop ran visibly, and Pi, Hermes, OpenAI, and Anthropic tool paths were exercised end to end.
• HY3 is first-class. Native MTP loading, safe AR fallback, model discovery, think-tag handling, and OpenCode tool calls are wired together.
• DeepSeek V4 gets an experimental fast path. David Tai's shape-specialized optimizations reached about 36 MTP tok/s on the tested 128 GB Mac. The lane stays opt-in while agent-quality calibration continues.
• No measured Qwen V2 regression. Alternating baseline/candidate runs were flat within thermal/order noise for decode, prefill, and memory, followed by the full Python and Swift suites.

The agentic-cache release. If you run MTPLX under a coding agent, this is about the slowdowns you could feel but not see — each one a real, named mechanism, each one fixed or fenced.

• Your warm session stops being evicted mid-run. Recently-active sessions are eviction-last under cross-session pressure, and divergent per-turn snapshots are bounded per session, so a long agent run keeps its warm state instead of paying a surprise full re-prefill.
• Tool-turn commits stop being ghosts. A background commit that could burn a 26-second full-history re-forward without ever storing is fenced off, a bounded grace window lets nearly-finished commits land under fast agent loops, and OpenCode's per-request session headers are honored so consecutive turns stop being treated as strangers.
• Every serve keeps a durable trail. A default-on, content-free request log (rotating, disable with one env) plus the 2.4.1 bit-exact capture make agent-session incidents diagnosable after the fact.
• Experimental DeepSeek-V4-Flash backend. A from-scratch native port with an optional speculative lane, gated on committed-sequence identity — thanks to davidtai.
• The documentation tells the truth. ~450 claims audited against the code; every confirmed drift fixed, from a phantom MLX fork to a wrong Anthropic base URL.

The smooth-streaming release. The chat window stops stuttering and bouncing, streamed markdown grows up, and the 2.4.0 short-turn regression is fixed.

• Streaming stays smooth on long answers. Finalized lines fold into segments so the view count stays bounded, the scroll pin runs in the same display cycle as layout so the bubble can never visibly bounce, a hidden ~50 ms per-update sizing walk is gone, and new text reveals with typewriter pacing that always keeps up.
• Code, tables, and math render properly. Live syntax coloring across twelve languages from a freeze-time lexer with O(new text) cost, streaming code cards that settle in place, real tables, and real math notation — Unicode superscripts, stacked matrices and fractions, no leaked dollar signs. Performance mode remains a true plain-text kill switch.
• 2.4.0 short-turn regression fixed. The compiled-verify path could reserve KV budget above the configured ceiling, taxing short requests with setup work they never used. If short turns felt slower on 2.4.0, this was why. Warming prefills also now yield to real traffic within one small chunk.
• The model chip tells the truth. A derivative artifact whose folder name extends a first-party model name is served under its own id — health payload, OpenAI model field, and app chip all report what is actually loaded.
• Opt-in debugging. Per-request capture for bit-exact failure replay, a frontend stream-performance probe with an on-screen HUD, and two experimental engine policies — all off by default.

The 35B speed release. The 35B-A3B MoE gets a compiled decode stack, the fan bug from 2.3.0 is dead, and tool calling gets another round of contract hardening.

• 35B-A3B compiled decode stack. Target-prefix compiled route, whole-MoE fusion, GDN post-conv fusion, and a row-owned router, plus continuous batched serving with fixed-shape cohorts and ragged KV for the concurrent lane. Contributed by davidtai.
• Fans no longer stay stuck at max (#201). A failed fan restore was silently treated as restored while the hardware stayed pinned. Restores now verify the fans are back on the Apple auto curve and retry with backoff until they are, and a watchdog drops any fan lease held while the engine sits idle.
• Honest finish_reason on cut tool calls. A generation cut by max_tokens mid-tool-call reports length instead of tool_calls, so agent clients continue the turn instead of executing a truncated call. The think-splitter also stops leaking reasoning into content on bare function= strings.
• More tool-call dialects. Bracket-style and Poolside arg_key/arg_value calls parse correctly, incomplete calls buffer instead of double-delivering, and calls to undeclared tools pass through per the OpenAI contract. Contributed by davidtai.
• Structured output cannot be eaten by thinking. The reasoning prelude inside constrained generation is bounded at 4000 characters (tunable, 0 restores unbounded), so an unclosed think block can no longer consume the whole token budget. Contributed by amoreo140.
• Forge probes recover from slow Hugging Face responses with a longer timeout and pinned-SHA retry. Contributed by PhilipJohnBasile.

The agent reliability release. Tool calls stop corrupting, structured output lands at full speculative speed, and agent sessions stop paying re-prefill taxes.

• Tool-call arguments no longer collapse to {}. The intermittent argument corruption behind failed agent edits is root-caused and fixed in both parsing lanes, validated over 168 live agent turns with zero collapses. Nested edits arrays arrive intact.
• Structured output at full speed. response_format with a JSON schema is enforced by grammar masks that compose with MTP speculative decoding: valid JSON, decode parity within noise, under five milliseconds of masking per request. Opt-in strict mode grammar-forces every tool call to a declared tool with schema-valid arguments. Contributed by PhilipJohnBasile.
• Sessions survive history rewrites. When an agent client compacts its transcript, a common-prefix fallback keeps the session identity and its warm cache instead of forcing a cold re-prefill.
• Leaner agent turns. The injected tool contract instructs whole-file reads and bans echoing file contents into visible text. Reasoning share of generated tokens dropped from 75 percent to under 30 in live sessions.
• Silent hangs are contained. A heartbeat watchdog turns a frozen stream into a structured five-minute failure instead of an infinite hang.
• Sharp edges filed. 5-bit and other non-divisor quantizations load correctly (by Jonathangadeaharder), parallel_tool_calls is honored, interrupted downloads are detected as incomplete (by titan550), vision works on a bare pip install, and context-copy drafting can never run past a stop token.

The copy-drafting and small-Mac release. The model now drafts from your prompt as well as its MTP head, and the 4B pair is fixed and fast.

• Context-copy drafting, on by default. When the model restates something already in your context (the function it is editing, a config block, a quote), whole spans are proposed as copy blocks and verified in one pass. Measured +53% on edit-heavy agent turns at temperature 0.6, parity on novel text. Exact at any temperature. Contributed by lBroth.
• The 4B models actually work now. The old 4B drafted at zero acceptance for everyone. The engine heals existing downloads at load (no re-download), the Speed 4B is rebuilt at 227.8 tok/s, and a new Quality 4B ships at 191.7 tok/s with a 2.2x MTP multiplier. Macs under 16 GB finally get first-class recommendations.
• Tune can no longer persist garbage. A zero-acceptance depth can never win or be saved, and poisoned records from earlier versions are quarantined automatically.
• Forge your own FP16 models. New precision option with M1/M2 auto-select.
• SSD session cache crash fixed. The writer thread no longer touches the GPU.
• MoE launches at its measured depth. The 35B-A3B default corrected from the ceiling to measured D2 (by davidtai).

The community-fixes release. Memory is bounded by default, the agent session cache is fixed end to end, and the app startup hang is closed. Most of this release started as community reports.

• Memory is bounded by default. The MLX allocator cache gets a RAM-tiered cap, the per-session admission gate is re-clamped on smaller machines, the paged KV pool stops growing past the context window, and a new --memory-budget flag fits MTPLX inside a declared RAM envelope.
• Agent sessions stay warm. Prefix reuse survives every tool turn, follow-ups on hybrid models restore near the divergence point (0.4s instead of 33.8s on a 22k prompt), and warm state now survives daemon restarts. Cache hits show up in standard usage fields.
• Penalties work in quickstart chat. Presence and frequency penalties were silent no-ops in the batched lane. Small models benefit most.
• The startup hang is gone. Runtime installs run off the main thread and every subprocess wait has a deadline watchdog.
• Hermes Desktop from the app. The Hermes tile launches Hermes Desktop when installed, pinned to the MTPLX backend.
• Two new native backends. qwen3_5_mtp and hy_v3, both community contributions.
• 8 to 10% faster decode under load. The model-owner thread is QoS pinned so background apps stop taxing generation.

The agent-reliability release. The repetition marathons in long agent sessions are fixed at the source, and the biggest quality-of-life issues since 2.0.1 are closed.

• Repetition marathons fixed. Multi-turn reasoning history now renders on Qwen's trained contract (scoped to the active round). The captured looping session went from 3/4 marathons to 4/4 immediate healthy tool calls; root-caused to context construction, not quantization.
• Serve on any host and port from the app. LAN serving (0.0.0.0) no longer misreports ports or kills healthy daemons, and the app explains the API-key requirement up front.
• Warm prefix reuse for every agent client. Pi, Claude Code, Cline, and custom harnesses get the block-prefix warm restores that were gated to OpenCode.
• Settings Off means Off. The SSD session cache setting is passed explicitly, so an explicit Off stays off and session-bank stops growing back. The runtime venv self-heals after updates.
• Community fixes. Streaming reasoning-tag leak fix by @Osamaali313, quickstart host/port rendering by @hasegaw, and a bounded Ctrl-C shutdown under open streams.

Turbo for every Mac. The v2 turbo default now covers every dense model on every Apple Silicon generation, with a load-time kernel safety net.

• M1 and M2 get their speedup. The FP16 27B those Macs route to now defaults to turbo: decode up 19-31% across 0.5k-32k context, roughly 2x over plain autoregressive decode.
• New 6-bit kernels. The 9B tier gains 33-62% decode and 43% faster 2k prefill under turbo.
• New model: Optimized Quality FP16. The missing M1/M2 quality artifact, wired into the picker and the chip-aware routing.
• Kernels prove themselves on your machine. Every turbo lane self-checks against stock MLX at load and falls back per lane if anything disagrees. Worst case is 2.0.0 speed, never wrong output.
• Verified on real M1 hardware. Kernel exactness plus a live turbo smoke now run in CI on M1 runners.

The coding-agent release. Long agent sessions in OpenCode, Pi, Hermes, and Claude Code that stay fast, stay warm, and do not fall over.

• Session cache v2. KV state survives restarts on SSD; a 100k-token session restores in ~2s. Tool-call turns chain warm instead of re-prefilling minutes per turn.
• Turbo decode, on by default. New verify kernels and compiled verify: 27B Optimized-Speed ~45 to 58-60 tok/s, Optimized-Quality 31-36 to 43-44 tok/s on M5 Max.
• Long context. 64k decode +12%, 128k from 17 to 20+ tok/s, peak memory down 8-16 GB. Stock PyPI MLX, no fork, any Apple Silicon Mac.
• Stability. The app no longer kills a healthy engine mid-session; fresh installs no longer crash at model load; SSD restores are corruption-free.
• Agent protocol pass. OpenCode plan-to-build keeps its cache and tools; presence/frequency penalties end-to-end; honest model identity for third-party builds.
• Chat. Markdown renders live while streaming; one compact activity strip per turn with grouped tool rounds and sources.
• Memory that fits your Mac. Cache budgets scale to the machine, with explicit RAM and SSD limits in Settings.
• Vision under MTP tells the truth. No more fabricated differences between similar screenshots.
• Fans behave. Ramp on request arrival, RPM-verified, held through post-response cache work.