25 AGO
We scored 0/25 five times. None of them was the model.
Five generations of harness. Exit 0 every time. Five different causes, and not one of them was the thing we were trying to measure.
Measurement9.78×
Notes from a lab
that audits itself.
We train and serve on eight H100s. Everything here carries its error bar — including the numbers we had to correct downward.
§ — the same measurement, twice
claimed
10.1×
measured
9.78×
TP8 no speculation 183.6 → TP2 + DFlash-64 1,795.1 tok/s (9.78×); the TP2 arm alone is 112.7 tok/s, so speculation by itself is 15.5× (same session, n=3). Linear scale 9.40–10.30
latest