MTPLX 2.5.3 is a small release focused on the agent lane and the API surface. A day of head-to-head benchmarking against another engine turned up a set of real latency bugs on our side and a few places where the API misled external tools. This release fixes all of them.
Between requests, the server commits session state to the reuse bank so the next matching request can restore instead of re-reading the whole prompt. Since the 2.4 line, that background commit could only be interrupted by its own session. A request from any other session, a second chat, a subagent, a tool call fired by your editor, could arrive while a multi-gigabyte commit was in flight, wait for it, and then decode slower on top of it. On tight request cadences the worst measured case was a follow-up turn running 44 percent slower with about three quarters of a second of extra first-token latency. This is the class of bug you feel as an occasional dead or sluggish turn in an otherwise fast session.
Commits now yield the moment any request is admitted, whichever session it
belongs to. A session's own follow-up keeps the short grace window it always
had, so streaming tool-call turns still resolve their prefix instead of
re-reading it. Set MTPLX_POSTCOMMIT_CROSS_SESSION_YIELD=0 to restore the
old behavior.
Rendering and tokenizing a long chat transcript costs 77 to 92 milliseconds per request. A repeated request with a byte-identical transcript now hits an exact-match encode cache and gets that work back in under a millisecond (retries, warm re-asks, and benchmark repeats; a turn that appends a new message still re-encodes). In our gate runs, warm follow-up latency went from a 194 to 961 millisecond band to a steady 65 to 74 milliseconds, and a clean warm restore measures 2 to 3 milliseconds server-side.
Three changes, all aimed at the same problem: tools that benchmark or integrate MTPLX through the OpenAI API were seeing things that were not the model.
usage, and broke output
equality checks at temperature 0. The MTPLX app and browser chat keep the
footer. MTPLX_STATS_FOOTER_SCOPE=all restores it everywhere.usage now reports completion_tokens_details.reasoning_tokens, so a
client can separate thinking tokens from visible output instead of
guessing from stream timing.Anonymous API clients now get standard OpenAI semantics: temperature, top_p, top_k, the thinking toggle, penalties, and generation mode set in the request body are applied, instead of being treated as observability hints. Requests that leave a field unset keep the server's launch and live settings. Clients MTPLX manages itself, the app, the browser chat, and the OpenCode and editor lanes it configures, stay server-owned exactly as before, so curated agent sampling does not change.
This closes the reports that MTPLX ignores temperature 0 and the thinking
toggle (issue #241). It also means an external benchmark that asks for
greedy decoding actually gets greedy decoding. Set
MTPLX_CLIENT_CONTROLS_DEFAULT=hints to restore the old policy;
the per-request X-MTPLX-Allow-Client-Controls opt-in still works there.
As part of the same work, temperature-0 requests now run the draft sampler greedy as well, so the speculative window agrees with the target's argmax choices more often. Depth-2 acceptance rose from .526 to .590 in our runs.
Model catalogs, model defaults, memory policy, and every managed-client behavior are unchanged. Each behavior change above has an environment switch that restores the previous policy.
pip install -U mtplxbrew upgrade mtplx