MTPLX 2.7.0: Qwen3.8 support 🎉

Qwen3.8-27B came out on 14 August. This release runs it the way the model card says it should be run, with three MTPLX builds tuned for it, FP16 versions of all three for M1 and M2 Macs, and a longer compiled verify window that helps every model. It also fixes a CPU drain in the SSD session cache and a crash on macOS 27.

Every speed number below was measured on one M5 Max with fans verified at maximum, die temperature gated before each run, one request at a time, generation running to the model's own stop. Other Macs will land elsewhere. Nothing here was measured on M1 or M2.

Qwen 3.8, served properly

There is a new qwen3_8 model family in the engine and the app. It carries Qwen's official inference contract instead of the Qwen 3.6 coding defaults:

• Sampling at temperature 1.0, top-p 0.95, top-k 20.
• Reasoning effort levels xhigh, medium and low. Coding sessions default to medium: on the same uncapped agent task, medium finished correct in 51.5 s where xhigh took 314.9 s. You can pick xhigh per request, on the CLI (--reasoning-effort xhigh) or in the app before you start the model.
• Thinking is preserved in the conversation history by default, which is what the model was trained on. Reasoning tokens stay in context and flow through MTP drafting like any other token, so speculation is calibrated on the thinking phase and the answer phase both.
• Qwen's chat_template_kwargs: {"enable_thinking": ...} request field is honored, so client code copied from the model card works unchanged.
• The Qwen 3.8 MTP head is trained for deeper drafts, but depth 4 killed the daemon on drop day, so live serving is capped at depth 3 in this release and Tune offers AR to D3.

The 3.8 trunk keeps the 3.6 hybrid attention layout, so the whole kernel stack transfers as is: compiled verify graphs, the custom verify kernels and the GQA fast paths engage identically, with the same load-time self-check on your own chip.

Three builds, calibration included

• Bare Speed (16.0 GB): flat 4-bit. Quickest burst chat speeds. Lower quality and slower on long coding tasks.
• Optimized Speed (20.4 GB): 4-bit dynamic quant. Great coding speeds and good quality. Recommended. Same hand-tuned layout as Qwen 3.6 Optimized Speed V2 (embeddings, output head, all 48 GDN output projections and the last 8 MLP blocks at 8-bit; GDN convolution and recurrent-state parameters, every norm and the whole MTP head at 16-bit).
• Optimized Quality (29.4 GB): 8-bit dynamic quant. Good coding speeds and perfect quality. Closest to the official bf16 model: KL divergence to the bf16 teacher 0.00105, against 0.0220 for Optimized Speed and 0.0376 for Bare Speed.

Each build states its measured calibration in its own runtime metadata: recommended draft sampler (Bare Speed 0.6, the Optimized pair 1.0), tuned MTP depth (3 for all three), and peak memory measured on that build. The runtime reads that metadata ahead of profile fallbacks, so a build launches at its own tuned depth even when the serving profile disagrees, and the app and the CLI launch every 3.8 build identically because neither pins a draft sampler of its own any more.

Sizes shown in the app and CLI are the exact byte sums of the published Hub files, and the peak memory numbers are measured, not inherited from a 3.6 sibling.

What the default is now

• Modern Apple Silicon (M3, M4, M5) with 32 GB or more: Qwen 3.8 Optimized Speed, downloaded on first use.
• M1 and M2 with 32 GB or more: Qwen 3.8 Optimized Speed FP16 (below).
• Under 32 GB: still the Qwen 3.5 9B route.

mtplx quickstart, mtplx start and the app's first-run picker offer the whole 3.8 line-up in the same order with the same one-line descriptions: Optimized Speed as the recommended default, then Bare Speed, then Optimized Quality (Quality is hidden on 32 GB Macs because its measured 33 GB peak does not fit there). Qwen 3.6 Optimized Speed V2 stays directly below them and keeps its turbo profile.

If you already use MTPLX and your last run used the recommended default, mtplx start says once that the default moved and from which model, instead of quietly relabeling. The app keeps whatever model you had; it does not switch you.

FP16 builds for M1 and M2

M1 and M2 have no native bf16, so every 3.8 build has an FP16 sibling on the Hub (Youssofal/Qwen3.8-27B-MTPLX-...-FP16). They are the identical model: every quantized pack is byte for byte the parent's (498 of 498 per build), and every 16-bit tensor is the bf16 value cast to fp16 (99.992% of elements exact; the remaining 0.008% are magnitudes below 7.6e-6 rounded on the fp16 subnormal grid, largest error 3.0e-8, none overflow). No bf16 tensor is left in any of them.

The M1/M2 tier of the CLI and the app routes to the FP16 siblings automatically: same three picks, same order, same descriptions, and the OpenCode config names the id the server actually advertises (mtplx-qwen38-27b-...-fp16). All three launch on turbo like their parents and pass the fp16 kernel self-check at boot; if a chip ever disagrees, the affected lane falls back to the stock path on its own.

Compiled verify to 32k

Since July the compiled verify graph stopped at 12,288 tokens of context, because a KV copy tax at the time made longer compiled windows a regression. That tax is gone, so turbo now compiles verify to 32,768 tokens. Interleaved A/B on Qwen 3.8 Bare Speed under die-temperature gates: the compiled path beat the eager fallback in every paired epoch (48.5 against 45.4 tok/s at 20k context, +6.9%), with flat peak memory at 20k and lower at 30k (25.4 GB against 28.5 GB; the eager path is the one that spikes). Past the fence the same custom kernels run eagerly, exactly as before. MTPLX_COMPILED_VERIFY can now be set by hand for parity and exactness runs against the shipped profile.

Coding agents

• OpenCode and Pi no longer send an output cap of any kind. OpenCode injected a 32k ceiling even when the model advertised more; Pi silently substituted 16,384 when the metadata omitted it. Both integrations now clear the generated cap for MTPLX models only, so generation runs to the model's own stop. Explicit caps you set yourself still apply.
• Pi sessions carry their real session id to MTPLX, so multi-turn Pi work restores its banked prefix from RAM instead of re-prefilling. Live receipt: 16.9k to 18.5k tokens restored per turn across a five-turn coding task.
• The session bank's background re-render now uses the effort the request actually ran with, in the postcommit path and the idle scheduler lane. Before this, a medium-effort session could run the bank permanently cold while xhigh warm-hit, because the effort instruction is part of the rendered prompt.
• Reasoning that hits the token limit before the closing think tag is now routed as reasoning, not shown as the answer.

App

• Qwen 3.8 launch family: turbo by default, the official sampler preset, the reasoning effort control with xhigh available and medium as the coding default, Tune from AR to D3, and catalog rows with exact sizes and measured peaks.
• The first-run picker shows the 3.8 trio (FP16 siblings on M1 and M2), and the Qwen 3.6 Optimized Quality row on M1 and M2 now resolves to its FP16 build instead of the bf16 one.
• Fixed a crash on macOS 27 when the inference settings overlay opened (#256, #257). SwiftUI 8 traps on a slider whose range has no distinct values; the depth slider was built with 1...1 for models without draft control, and the context-window slider could hit 4096...4096. Both are now built only when there is something to slide. Reported and fixed by @joshlacal.

Fixes

• SSD session cache no longer walks its whole store on every write or every /health poll. On a long-lived bank (816,220 files, 89.9 GB) each walk took 41.7 s. The cap check forced one per write, and the app's health poller kept another running back to back: most of a CPU core, all the time, heating the die under live decode. Reconciliation is now maintenance. It runs only when the store changed and at most 5% of the time, off the writer lock, yielding to live traffic, and the cap gate prices orphan bytes from the last snapshot instead of walking again. Measured on that bank: idle CPU with a health poller 35% down to 0.2%, per-write cap gate 71 to 159 s down to 3 to 6 s, cache-hit restores unchanged.
• mtplx pull names the mirror knob when a download fails for a network reason and no HF_ENDPOINT is set (#259): HF_ENDPOINT=https://hf-mirror.com on the CLI, Settings, Advanced, HF download mirror in the app. Both were already supported and neither was documented; the troubleshooting docs now cover them.
• mtplx doctor judges memory against the model this Mac would actually default to (9B under 32 GB, FP16 on M1/M2) instead of pricing a 27B against 80% of RAM and failing every Mac under about 45 GB. M5 Max is listed in the support matrix.
• Hardware detection calls /usr/sbin/sysctl and /usr/sbin/system_profiler by absolute path, so it works from the app's sanitized environment on any Mac. mtplx doctor and mtplx tune no longer run git outside a repository, which on a Mac without the Command Line Tools used to pop the Xcode install dialog during onboarding.
• Depth-default resolution honors artifact metadata across profile mismatches; the degrade pin (AR mode when the MTP head is missing) and the no-metadata path both survive. An early Quality build ran at depth 2 instead of its tuned depth 3 because of this; the fix is worth +19% on that build.
• The public depth ceiling is decided by the artifact reference, not the served-name alias, so a non-3.8 model served under the default id cannot widen its own depth gate. mtplx tune validates depths against what the model actually supports and takes its sampler from the same family contract as mtplx serve.
• MTPLX_REQUEST_LOG_JSONL=1 means "log to the default file", not a file literally named 1.
• First-live-contact serve fixes for 3.8: xhigh boot no longer trips strict warmup, and the request-log env toggle is honored on the family path.

Thermal honesty

• Every max-fan session now holds an ownership token. A daemon shutting down behind its replacement can no longer switch the fans back to Auto under the daemon that is still serving (or under your benchmark).
• Forge refuses to load a model for benchmarking when verified max-fan mode cannot start, instead of quietly reporting derated numbers. mtplx tune gained --require-max-fans for the same reason.

Forge

• Recipes can carry module_overrides: per-module quantization overrides applied in one conversion pass (suffix match, optional layer list, bits, group size, mode). This is the lane that built Optimized Speed.

Known issues, fixed in 2.7.1

• Choosing xhigh in the app's Inference settings while the model is already running is rejected by the server in this build, and mtplx config set reasoning_effort xhigh is rejected the same way. Set it before starting the model, or pass --reasoning-effort xhigh on the CLI.
• The app's KV cache quantization toggle is not applied to Qwen 3.8 models in this build.
• With reasoning switched off in a plain chat with no tools, Qwen 3.8 emitted a stray tool call and cut the turn short on about half of our coding prompts. Keep thinking on (the default) until 2.7.1.

QA for this release

• Medium-effort coding instrument, identical prompt across engines: Bare Speed 65.2 tok/s, Optimized Speed 58.7 (accepted probability by depth 0.961 / 0.879 / 0.816), Optimized Quality 40.6 (that run was taken before the depth fix, at depth 2; the installed-app line below has the shipped depth-3 number). Same instrument, same night, Qwen 3.6 Optimized Speed V2: 59.9 to 60.1 tok/s. The 3.8 Bare build outruns the 3.6 flagship.
• The installed app end to end (this release's signed bundle, engine started from the UI, defaults resolved purely from artifact metadata, cold sessions): Bare Speed 64.4 tok/s at 17.0 GB peak, Optimized Speed 55.5 at 23.6 GB, Optimized Quality 48.3 at 32.7 GB. The speed the app shows matches the request-log receipt on every run.
• Head to head, same prompt and sampling: oMLX 0.5.7 serving its own Qwen3.8-27B 4-bit MTP quant with its speculative path on decoded 63.3 tok/s. LM Studio on the long-form task: 17.40 tok/s against Bare Speed at 32.4 tok/s sustained over a single 52,740-token response (27.2 minutes, ended at the model's own stop).
• xhigh long form: Optimized Speed 35.1 and 37.3 tok/s over 28k and 20k token responses; Optimized Quality 33.2 and 33.1 at depth 3; Bare Speed 35.7 and 32.0 over 34k and 37k token responses.
• Verify cost per round on the medium instrument: Bare 44.0 ms, Optimized Speed 50.3 ms, against 51.5 and 52.4 ms for 3.6 V2 the same night.
• Live agent QA on ship defaults with no flags: a two-turn OpenCode coding session and a headless Pi session, both with warm session-bank restores from RAM at every turn, no re-prefill storms, no cache poisoning.
• Exactness: acceptance is the exact probability-ratio rule with residual resampling, so sampled output follows the target distribution at every temperature. Fixed-geometry determinism verified byte-identical on all three builds. At temperature 0, MTP-vs-AR argmax can still flip on near-ties across different verify tile geometries (the bf16 rounding property documented in #245), at the same rate as shipped 2.6.0.
• Release pipeline on the shipped commit: full Python battery green, Swift app suite 573 of 573, pillar QA (vision cache, memory ceiling, long-output decay) all pass with fans verified, app and DMG notarized and stapled.

Thanks

@joshlacal for the macOS 27 crash report and fix (#256, #257), and everyone who filed the download and mirror reports behind #259.