# gpqa-glm53-20260827-1653

Public raw data package for this measurement. Every number on the measurements page citing this window comes from these files.

GPQA Diamond, n=198, K=3 seeds (42, 43, 44). The benchmark requests that examples not be published in text: `gpqa_sanitizado.json` does not include question, choices, or response text. It includes, per item and seed, the extracted letter, the target letter (in this run's permutation), the verdict, and infra failure. Full-file sha256 in `seeds[].sha256_arquivo_integral`.

0-shot, with thinking enabled by default on the server (the run does not pass the parameter per request). The 'temp 0' label in `divergencias_da_referencia` is incorrect; effective sampling is that of `sampling_efetivo` (T=1.0, top_p 0.95). The run was served with MTP k=5 enabled. `resumo.json` is derived from this package.

Published figure: `acuracia_so_linha_final_pct`, 89.85 ± 0.59 (2·se) over the 197 valid items per seed: only the letter on the final line «Answer: X» counts, and an answer cut off at the output limit is wrong. Counting as wrong the one question that did not fit the context window: 89.39% (531/594). The old grader (field `score`) credited two responses cut off at the output limit; its 90.19 ± 0.34 in `acuracia_sobre_validos_pct` (89.73 over 198) is retracted, and the 89.39 that appears inside that block is the old grader's seed 42, not the figure above. `extracted_atual` is the letter from the old parser, not a duplicate of `extracted`: they differ in 25 of the 594 items.

| file | what it is | sha256 (first 16) |
|---|---|---|
| `gpqa_sanitizado.json` | items per question or task, without question or response text; file lists what was removed | `4765610ae52167af` |
| `resumo.json` | means, standard deviations, and window result (K repetitions per arm) | `e0f731e95b220d83` |