The GPU is busy 99% of the time. Its memory is busy 22%.

Correction (2026-09-25, rev 2). The first version computed the bandwidth floor as if all active weights were FP8. The checkpoint’s tensor headers show 22.4 GB of active weights, half of it BF16, so the floor is about 1,180 tok/s and we run at 13% of it; the retracted figure is kept in The instrument, again. Also corrected: the 512-token output limit is a cap, not a fixed length; the long all-reduce waits are the first two calls of each step and were measured only under the profiler; the detail of the two truncated GPQA answers; how the HLE and MMLU-Pro answers cut off at the output limit were re-run; power is the sum of the eight GPUs; the section on our evaluation no longer calls it independent.

GLM-5.3-Flash ships with its own multi-token prediction (MTP) head. With three draft tokens it takes single-stream decode on our 8× H100 node from 156.05 to 317.90 tokens per second. That is the headline, and it is the easy part of this note.

The harder part is what the counters say about the step that MTP amortizes. During those runs the GPUs report a kernel in flight 98.74% of the time, while their memory is read or written 21.80% of the time. Decode at batch 1 is supposed to be a memory-bandwidth problem. On this model and this engine, it is not.

The result

Two arms on the same node and the same server image, one request in flight at a time. Each request sends a synthetic 256-token prompt (a marker word followed by one filler word repeated) to the completions endpoint and is decoded greedily (temperature 0) up to a 512-token cap; every request reached the cap. Thinking was off. Each arm ran three 60-second windows after a discarded warmup, on 17 Sep. Speculative acceptance depends on the text. On this filler 41 of 113 MTP requests ran at the acceptance ceiling; on real agent tasks at T=1.0 the gain was 1.52× per task (below).

The tok/s column counts generated tokens over window time, so it includes the time to first token (0.21 s without speculation, 0.06 s with MTP). The decode-only column is the mean of per-request rates after the first token. With MTP the per-request rate ranges from 114 to 466 tok/s; total decode tokens over total decode time is 330 tok/s (1.98× vs 166.33 tok/s without speculation). The ratio of the two means in the first column is 2.04×; across the worst and best pairing of windows it stays between 2.02× and 2.05×.

2.04

2.02 2.04 2.05

×

n=3 · σ 0.02

Native MTP (k=3), one request at a time: 156.05 ± 0.07 → 317.90 ± 2.92 tok/s (2.04×), three 60 s windows per arm, 17 Sepmeasured

Where the time goes

The floor

At batch 1 every decoded token has to read the model’s active weights once, plus the state its attention layers carry. For this model that is about 22.4 GB of active weights, read from the checkpoint’s own tensor headers: 11.1 GB in FP8 (MoE experts, dense MLPs, most sparse-attention projections) and 11.3 GB in BF16, which is how the checkpoint stores the projections of the 34 linear-attention layers (9.4 GB) and the output head (1.3 GB); plus 0.14 GB of recurrent state and 0.09 GB of KV cache (contexts up to 8K). The total is 22.65 GB per token.

Spread over eight GPUs at 3.35 TB/s each (26.8 TB/s in aggregate), reading that takes 0.85 ms. That is the bandwidth floor: about 1,180 tok/s. We measure 156.05, which is 13% of it. If memory bandwidth were the limit, this node would decode about seven times faster than it does.

That floor counts the checkpoint’s bytes. The engine we run reads a little more: vLLM builds the sparse-attention layers without FP8 and dequantizes their four FP8 projections to BF16 when it loads them, which adds 1.1 GB per token. Against the engine’s own floor, about 23.8 GB and 0.89 ms, or about 1,130 tok/s, our rate is 14%.

The counters

We sampled nvidia-smi once per second on all eight GPUs during the same windows.

utilization.gpu does not mean what its name suggests. It is the fraction of each sample period during which at least one kernel was running. One small kernel on a corner of the chip counts the same as the whole chip working. We had been labeling that column “SM active”. It is not; we had no tool on this machine that reads per-SM activity (DCGM or Nsight), so we cannot say how many SMs were doing useful work.

utilization.memory is the fraction of time device memory was being read or written. Read together, the two lines say the GPU always has a kernel in flight while its memory sits idle about 78% of the time. The step is not limited by HBM bandwidth.

MTP changes the level, not the shape. Memory is busy 26.95% of the time and power rises to 308.1 W per GPU: the same kind of step, doing more useful work.

No speculation: utilization.gpu 98.74%, utilization.memory 21.80%, 256.9 W per GPU (nvidia-smi at 1 Hz, 8 GPUs, three windows)measured

utilization.gpu reported as "SM active"retracted

The trace

To see the step itself we ran the PyTorch profiler over a single request (a 524-token prompt, 64 output tokens, thinking off) and read the trace of rank 7 of 8. Every complete step in the trace launches exactly 1,694 kernels: 1,484 on the main stream and 210 on a second one. The count is the same in all 62 intervals between the 63 step markers. The median kernel lasts 2.56 µs.

Under the profiler a step takes 7.65 ms (median), and the main stream is busy 88.7% of it. Without the profiler a decode step takes about 6.0 ms (1 / 166.33 tok/s). The floor says 0.85 ms of that is moving the bytes the math needs. The rest goes to running some 1,700 small kernels per step, each too short to keep the memory system busy, plus a wait at the start of each step that we measured only under the profiler.

1,694 kernels per decode step (1,484 + 210), median kernel 2.56 µs, one profiled requestsingle-trace

The all-reduce

Tensor parallelism across eight GPUs puts an all-reduce after the attention and after the MLP of every layer. At batch 1 each call reduces one 4,096-wide bf16 vector: 8 KiB, too small for link bandwidth to matter.

In the trace, 89 of the 91 calls in each step take about 5.8 µs (median 5.79 µs). The first two do not: the one right after the embedding waits 1.0 ms (median 1,014 µs over 62 steps) and the next one 0.24 ms (median 242 µs). That is where the eight ranks, which start each step at slightly different times, line up. The profiler slows the host, and the profiled step is 7.65 ms against 6.0 ms without it, so we do not know how much of that wait survives without the profiler.

So

The decode step on this stack is bound by the latency of many small kernels, not by memory bandwidth; under the profiler there is also a wait of about 1.25 ms at the start of each step. MTP pays each step’s fixed cost once for more than one token, whatever that cost is made of.

The profile points at fewer, larger kernels and less synchronization per token. We have not measured a route to that here.

Agentic workloads

The same MTP head inside an agent loop: eight DeepSWE tasks run by our own agent with eight parallel workers, once without speculation (3 Sep) and once with MTP k=3 (4 Sep). Paired task by task, MTP made each task finish sooner. The geometric mean of the per-task duration ratio is 1.52× (95% CI 1.31–1.76, t on the logs, n=8). With MTP every one of the 8 tasks finished sooner; per turn, the gain is 1.64×.

That is one run per arm, on different days. Quality is not comparable at this size: 7 of 8 tasks resolved without MTP, 6 of 8 with it, and exactly one task changed outcome.

We had recorded this as 13.2 → 18.5 tasks per hour (+40%). With eight tasks on eight workers, the wall clock of a run is the duration of its slowest task, and the slowest task was the same one in both arms. That number measured one task.

Per-task duration without speculation ÷ with MTP k=3: 1.52× (95% CI 1.31–1.76), 8 paired tasks, one run per armsingle-run

MTP raised agentic throughput from 13.2 to 18.5 tasks per hour (+40%)retracted

Our own evaluation

We also ran the model through public benchmarks on the same node, with our own runners and our own agent. None of these numbers is comparable to a vendor card or a leaderboard; the last column says why. For K>1 the error bar is 2·sd/√K across runs; for K=1 it is the binomial 2·√(p(1−p)/n).

GPQA: one question did not fit the 65,536-token context window on any of the three seeds: our client estimated its prompt at 1,970 tokens and asked for 63,054 output tokens, but the prompt is about 2,836 tokens. It is left out of the denominator; counting it as wrong gives 89.39%. The run was served with MTP k=5 on. The results file labels the sampling “temp 0”; the effective sampling was T=1.0, top-p 0.95.

HLE and MMLU-Pro: responses cut at 65,536 output tokens were re-sampled up to four times with a 163,840-token budget, keeping only a sample longer than the original cut; items without one count as wrong (122 on MMLU-Pro: 60 cut again, 62 whose re-runs finished shorter and were discarded by design). One MMLU-Pro request returned nothing and is left out. With no judge, HLE exact match only counts answers whose normalized string matches the reference.

DeepSWE: the K runs are independent repetitions at T=1.0, because our agent ignores the seed. The first baseline repetition mixes two serving setups: 29 tasks from a run at 20 workers without MTP and 82 from a run at 8 workers with MTP k=3. Tasks whose agent history outgrew the 262,144-token context (8 of the 13 excluded in the baseline) are left out of the denominator; counted as failures, the baseline is 63.72 ± 2.65% (individual runs 61.47%, 63.64%, 66.06%), scaffold v1lite K=6 is 62.80 ± 5.41%, and the difference is Δ −0.92 pp. The scaffold variant did not move the result.

The instrument, again

In LN-001 we argued that the measuring instrument lies more often than the model. Preparing this note, we recomputed every number above from the raw files instead of copying it from our own records. Six did not survive as recorded.

MATH-500. Our answer normalizer marked equivalent answers as wrong: x = 4 against 4, \dfrac against \frac. Re-grading the same responses with the fixed normalizer moves the score from 76.20% to 99.00%. We checked each of the 148 distinct pairs individually (342 items); every one is a legitimate equivalence. The 15 remaining failures are 7 truncations and 8 answers the grader rejects; one of them (\textbf{(B)}\ 3(1+r/10)^x=7 against (B)) looks like a grader miss.

GPQA Diamond. Our grader credited two answers from responses cut at the output ceiling (the same question in two seeds, both at 64,629 tokens, the full output budget our runner set for that prompt) that never reached the final “Answer: X” line: one letter came from an “Answer:” in the middle of the reasoning, the other from a fallback that reads a conclusion in prose. Counting a response cut at the ceiling as wrong gives 89.85 ± 0.59% instead of the 90.19 ± 0.34% we had recorded.

HLE. The recorded denominators left out 45 items that had been re-run at 163,840 output tokens and still did not finish; their question type had been lost in the results file. The numeric tolerance in our exact-match grader accepted 4 answers whose digits differ from the reference. And there was no judge at all, so exact match is a floor. Corrected: 35.98% multiple choice (recorded as 36.06%, 163/452) and 8.75% strict exact match (recorded as 9.31%, 131/1,407).

MMLU-Pro. Our runner’s answer extractor read no letter from 1,772 of the 11,909 finished responses, including final answers written as \boxed{\text{G. …}}, and its case-insensitive capture turned “The answer is exactly…” into the letter E. Our prompt made it worse: it never asked for the official “the answer is (X)” ending. Re-grading the same responses with the official MMLU-Pro extractor, reimplemented from the benchmark’s repository at a pinned commit, gives 82.96% instead of the 72.59% we had recorded; the verdict changes on 1,686 items. An extractor that also reads \boxed{…} answers gives 83.48%.

Our own records. We had been quoting 1,529 kernels per step over 54 steps. The trace holds 109,211 kernel events; 1,529 is that total divided by about 71, a step count the trace does not support. Counting inside the step markers gives 1,694 in each of the 62 complete steps. The per-category shares of the step we used to quote do not come out of the trace either, so they are not in this note. The DeepSWE K=6 figure was 64.18 ± 4.85; recomputed from per-task results it is 64.21 ± 4.83, because two repetitions had been recorded as 58.95% instead of 59.05%. The “SM active” label and the agentic “+40%” above belong on this list too.

The bandwidth floor. We had calculated the floor assuming all active weights were in FP8, using 18.0 GB of weights. Reading tensor headers directly from the checkpoint shows 22.4 GB of active weights, about half in BF16: the projections of the 34 linear-attention layers (9.4 GB) and the output head (1.3 GB). Adding the recurrent state and KV cache gives 22.65 GB per token, putting the floor at about 1,180 tok/s (0.85 ms) and our measured rate at 13% of it. Counting the bytes the engine actually reads (it holds the sparse-attention projections in BF16), the floor is about 1,130 tok/s and our rate 14%.

GPQA Diamond 90.19 ± 0.34% (two truncated responses credited by our grader)retracted

MATH-500 76.20% (normalizer marked equivalent answers as wrong)retracted

MMLU-Pro 72.59% (our grader; the official extractor on the same responses gives 82.96%)retracted

1,529 kernels per decode step over 54 stepsretracted

Bandwidth floor 1,469.7 tok/s from 18.235 GB per token, all weights assumed FP8; our rate was 10.6% of it (checkpoint headers show 22.4 GB of weights, half in BF16; corrected: 22.65 GB per token, ~1,180 tok/s, 13%)retracted

None of the six was the model. All six were found the same way: by recomputing from the raw files instead of trusting a summary.

What comes next

Next on the same node: SciCode in its official harness; CritPt, whose answers are private and can only be graded by the benchmark’s evaluation server, which we have asked to access; and SWE-PolyBench Verified, which we are preparing as a leaderboard submission. Each will come with its raw files, or it will not appear here. We are not promising dates.

Method

• Hardware. 8× H100 80GB SXM, tensor parallel 8, the model’s official FP8 checkpoint (experts in FP8, linear-attention projections in BF16), served with vLLM from a dedicated image for this model.
• Throughput. One request in flight; synthetic 256-token prompt to the completions endpoint; greedy decoding (temperature 0); capped at 512 output tokens (every request reached the cap); thinking off; three 60-second windows per arm after a discarded warmup. tok/s is generated tokens over window time, including time to first token. ±2·se across windows.
• Counters. nvidia-smi at 1 Hz on all eight GPUs, averaged per window.
• Profile. PyTorch profiler over one request, rank 7 of 8. A step is the interval between consecutive step markers on the main stream. The 347 MB trace is not published; a summary derived from it is.
• Floor. Arithmetic, not a measurement: bytes read per token divided by aggregate HBM bandwidth.
• Evaluations. Our own runners and agent. Datasets that ask not to be published in plain text (GPQA, HLE) are released as per-item verdicts without question, answer or reference text.

Raw files, one package per claim:

• Throughput and counters: telemetria-glm53-c1-20260917-2110
• Step profile (derived from the trace): perfil-glm53-20260828
• Agentic, paired tasks: agentico-glm53-mtp-20260903
• GPQA Diamond: gpqa-glm53-20260827-1653
• MATH-500: math500-glm53-20260919
• HLE: hle-glm53-20260923
• DeepSWE: deepswe-glm53-20260908
• MMLU-Pro, per-item letters from three extractors: mmlu-pro-glm53-20260923

Every headline number is also on the measurements page (rows B8–B10 and C7–C11); the floor arithmetic, the per-position all-reduce and the DeepSWE sensitivity are only here. Nobody outside the lab has reproduced any of it yet.