# math500-glm53-20260919

Public raw data package for this measurement. Every number on the measurements page citing this window comes from these files.

MATH-500, n=500, K=3 seeds (42, 43, 44). `math500_k*.json` contain the model response per item, with full text, without latencies or throughput; removed fields are listed in each file, with full-file sha256.

`reavaliacao.json` reapplies to the same responses the extractor and normalizer from `run_math500.py` (md5 in file); `resumo.json` totals them. The legacy normalizer marked equivalent responses as wrong (e.g. `x = 4` against `4`). Distinct pairs now accepted are listed in `resumo.json` and were checked by hand. Of the 15 items the new normalizer still marks wrong (7 cut off at the output limit, 8 wrong answers), one looks like a grader miss: k=2, `test/intermediate_algebra/754.json`, answer `\textbf{(B)}\ 3(1 + r/10)^x = 7` against gold `\text{(B)}`. Counting it, the score is 1,486/1,500 (99.07%); the published 99.00% keeps it as wrong.

| file | what it is | sha256 (first 16) |
|---|---|---|
| `math500_k1.json` | model responses per item, one repetition; no latencies (see README) | `ea71713234d4b92e` |
| `math500_k2.json` | model responses per item, one repetition; no latencies (see README) | `47136d7e5c0e9a67` |
| `math500_k3.json` | model responses per item, one repetition; no latencies (see README) | `bbffb7b2f1a2b3ce` |
| `reavaliacao.json` | item-by-item re-evaluation (derived): extracted answer, ground truth, old and new verdicts | `53646103025261b3` |
| `resumo.json` | means, standard deviations, and window result (K repetitions per arm) | `4dafefdbecec0cba` |