The MTPLX API.

Every endpoint the server exposes, the request fields it reads, and what comes back. mtplx start or the app's play button starts the server; the app and the CLI share it. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.

Base URLs and auth

The Anthropic SDK appends /v1/messages itself, so its base URL is the bare server root; a /v1 base would request /v1/v1/messages, which has no route.

On localhost no key is required. Clients that insist on one can send any non-empty string. Non-localhost binds require --api-key, and requests then authenticate with either header:

Authorization: Bearer <key>
X-API-Key: <key>

To reach MTPLX from other devices on your network, or from a Windows VM on the same Mac, bind all interfaces with a key file. If the file does not exist yet it is created with a fresh key and printed once, and startup prints a Network OpenAI API Base URL for the other machine to use.

mtplx serve --host 0.0.0.0 --port 8000 --api-key-file ~/.mtplx/api-key

Same-origin by default. Since 2.11 (4 Sep 2026) a browser page's Origin must match the Host it used, so a page in another tab can no longer POST to /v1/* or reach /admin/* on the keyless localhost bind. --cors-origin (repeatable) or MTPLX_CORS_ORIGINS allowlists a browser front-end on another origin for the API, never for /admin or sign-in; every other origin gets a 403 with no CORS headers. Requests without an Origin header (the app, OpenCode, Pi, Claude Code, Cline, curl, server-side clients such as Open WebUI) are untouched.

mtplx serve --cors-origin http://localhost:3000

Endpoints

Chat completions

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"mtplx","messages":[{"role":"user","content":"hello"}],"stream":true}'

The server has one chat model loaded and answers with it whatever chat model name the request carries; the served id is listed at /v1/models. Request fields the server reads: messages, stream, stream_options, max_tokens, temperature, top_p, top_k, presence_penalty, frequency_penalty, seed, tools, tool_choice, response_format, reasoning_effort and generation_mode.

Streaming. Server-sent events, one chunk per committed token by default. Start the server with --stream-interval N to batch committed-token chunks when a client prefers fewer events. When the model re-emits text already in context, the context-copy lane commits blocks of 24 tokens per round; since 2.11 the server releases a block's tokens across the round instead of one frame per block, so every client sees a flow (MTPLX_STREAM_PACER=0 restores the old cadence). A streamed response closes within 30 s of its last token even when another request's prefill is queued ahead of its session snapshot (#425).

max_tokens. The request's cap is honored as sent. A client that computes a cap of 1 gets a one-token answer; since 2.11 the server logs one WARNING naming the client's cap, and the generation trace line carries max_tokens, effective_max_tokens and finish_reason (#436). reasoning_effort accepts the off-list values some clients send (PR #433 by @sypsyp97), and finish_stop_origin in the public stats and the request log names a stop's commit path (#414, PR #426 by @atirna).

Sampler. With no sampler fields set, the server uses the pack's own sampler: for Qwen 3.8 the official Qwen 3.8 sampling, temperature 1.0, top-p 0.95, top-k 20. Acceptance of drafted tokens is exact probability-ratio rejection sampling with residual correction, so the output follows the model's distribution at any temperature. The penalties default to 0, an exact no-op that preserves MTP exactness.

Plain decoding per request. generation_mode may be "mtp" or "ar". "ar" uses target-only decoding and reports mtp_depth: 0; it does not unload the MTP weights, so the next request can use MTP again.

Tool calls. When tools are active, Qwen XML tool calls are translated into OpenAI delta.tool_calls chunks as the function name and arguments stream. Unknown or malformed tool-shaped output falls back to assistant content rather than hanging or returning a server 500.

Structured output. response_format with a json_schema has run with full MTP speed since 2.3.0 (21 Jul 2026).

Messages (Anthropic)

curl http://127.0.0.1:8000/v1/messages \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mtplx",
    "max_tokens": 128,
    "system": "Be concise.",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "Write one sentence about native MTP."}]}
    ]
  }'

Requests are translated into the same internal chat path as /v1/chat/completions and returned as Anthropic-shaped message payloads. Supported: system as text or text content blocks; messages[].content as text, tool-result and, since 2.11, image content blocks (base64 or URL, also inside tool_result), which reach the model's vision tower instead of being rendered as base64 text (#441); max_tokens, temperature, top_p, top_k; tools, tool_choice, stop_sequences, thinking; stream false or true.

Streaming. Server-sent events with message_start, content_block_start, content_block_delta, content_block_stop, message_delta and message_stop. Qwen reasoning maps to Anthropic thinking blocks: a content_block_start with content-block type thinking, then thinking_delta events, with answer text resuming in a separate text block. Since 2.10.2 (1 Sep 2026) the bridge emits prefill keep-alives as empty thinking_delta events inside that block, so a first turn whose prefill outlives a client's idle window survives.

Cache accounting. The Anthropic bridge reports cache_read_input_tokens and input_tokens as disjoint fields, so a client's context meter shows the true delta (2.10.2, PR #417). Claude Code's complete client-tool loop has run on this route since 0.3.7 (17 May 2026); setup is on the Claude Code page.

Usage and caching

Every response reports usage.cached_tokens: the number of prompt tokens served from the session cache instead of being prefilled again. Since 2.0.0 (6 Jul 2026) MTPLX checkpoints the attention KV cache plus the recurrent and conv GDN state at commit boundaries, with speculation on. A follow-up turn that extends an earlier prompt restores that prefix: mid-session tool rounds restore warm in under 2 s, and a 100k-token session restores in about 2 s after a restart instead of a five-minute cold prefill. The SSD session cache that makes the restart case work is on by default; disable it with --ssd-session-cache off.

Since 2.11 (4 Sep 2026) a tool-call turn's final state is banked in place, a per-request instruction such as a forced tool_choice no longer changes the session's cache identity (the bank keys on token content alone), and a pending snapshot that is still prefilling when its wait bound expires is followed to completion instead of redone. On a 43k-token agent session on Flash-Next (M5 Max, 128 GB) the follow-up after a file write re-prefills 20 tokens in 0.12 s instead of 3,535 in 3.7 s, and a forced tool round in a 41k-token session takes 0.58 s instead of 85 s. Every request record carries whether it started with the GPU working set warm; MTPLX_GPU_KEEPALIVE=0 disables the keepalive.

Embeddings and rerank

The same daemon can serve retrieval models, so a RAG or agent-memory setup needs no second inference server. Point it at any MLX embedding or reranker model, by Hugging Face id or local path, optionally with a REF=served-id alias. Both flags repeat, and the same reference listed as both roles loads one copy of the weights.

mtplx serve \
  --embedding-model mlx-community/Qwen3-Embedding-8B-4bit-DWQ \
  --reranker-model vserifsaglam/Qwen3-Reranker-4B-4bit-MLX

curl http://127.0.0.1:8000/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen3-Embedding-8B-4bit-DWQ","input":["hello","world"]}'

curl http://127.0.0.1:8000/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"query":"where is the cache?","documents":["the cache lives in ~/.mtplx","unrelated text"]}'

Retrieval models load on first request and are capped by --retrieval-max-resident (default 2), which unloads the least recently used one beyond the cap. With nothing configured the endpoints answer 404 and chat behaves as before. /v1/models stays chat-only by default; list retrieval models with ?capability=embedding or ?capability=rerank, and a chat completion that names a retrieval id gets a clear 400. Checkpoints that bundle their own Python inference code are refused with a 403 until you opt in with --retrieval-trust-remote-code. These models skip the MTP path: multi-token prediction makes next-token decoding cheaper, which means nothing for a model that returns a vector. Persist them in ~/.mtplx/config.toml as embedding_models and reranker_models, or in the app under Settings, Retrieval endpoints.

Health and metrics

GET /health reports model load state, profile, exactness baseline, MLX and runtime information, fan mode and warmup status. The payload includes generation_mode, load_mtp, mtp_enabled, depth, api_key_required, rate_limit_per_minute, stream_interval, warmup and reasoning_parser, so a client harness can confirm the active serving policy.

GET /metrics returns a JSON snapshot of runtime KPIs: latest (the most recent turn), recent (the last 32 turns) and tool_parse_counters.

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/metrics

Python clients

The stock SDKs work unchanged. OpenAI, streaming:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="local")

stream = client.chat.completions.create(
    model="mtplx",
    messages=[{"role": "user", "content": "Write a tiny TOML parser example."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Anthropic, with the bare server root as the base URL:

from anthropic import Anthropic

client = Anthropic(api_key="local", base_url="http://127.0.0.1:8000")

message = client.messages.create(
    model="mtplx",
    max_tokens=256,
    system="Be concise.",
    messages=[{"role": "user", "content": "Write a tiny Python function that clamps a number."}],
)

for block in message.content:
    if block.type == "text":
        print(block.text)

Server flags

mtplx serve --port 8000
mtplx serve --host 0.0.0.0 --api-key "$MTPLX_API_KEY"
mtplx serve --rate-limit 120
mtplx serve --stream-interval 4
mtplx serve --warmup-tokens 16
mtplx serve --reasoning-parser qwen3
mtplx serve --no-mtp

--warmup-tokens runs a small startup generation after model load and reports the result in /health; --strict-warmup makes a warmup failure fatal. --no-mtp serves plain autoregressive decoding on the same loaded model. Sampler defaults can be set at startup with --default-presence-penalty and --default-frequency-penalty, or changed live with mtplx settings set. Every command takes --help. Deeper reference lives in the repository: api.md, server.md, concurrency.md.