1c11861902
propagate_to_variants skipped any row with self_eval_samples > 0, meaning "this one was measured directly, do not overwrite it with the family's". But inheritance COPIES self_eval_samples, so an inherited row also has samples > 0 and the test could not tell the two apart. A variant could be inherited once and never again. Found by re-measuring docs_writing. kimi-k3 moved 0.85 (n=2) -> 0.957 (n=7) while kimi-k3-flex sat at 0.85 with a timestamp from 2026-08-16, and the run reported "propagated 0 inherited rows". Flex rows are what `auto:batch` admits, so overnight work was ranking on scores its family had left behind by a week -- silently, since a stale score looks exactly like a fresh one. Provenance is the missing fact, so the table now records it: proficiency.inherited_from names the model a row's scores were copied from, NULL when they were measured on that row. propagate_to_variants refreshes where it is set and still refuses where it is not. The migration ships the repair, not just the fix. ADD COLUMN gives every existing row NULL, which reads as "measured here" -- so on its own it would have left exactly the rows this change exists for frozen forever. Provenance that was never recorded cannot be recovered in general, but it can for the rows that matter, and not by guessing: eval_identities selects standard rows plus flex rows with no standard equivalent, so a flex row WITH one was never a candidate for direct evaluation whatever its sample count says. That is the harness's own selection rule read backwards. Everything else keeps NULL, which is the safe direction -- it means "do not overwrite", so a real measurement is never lost to the backfill. schema.sql is CREATE TABLE IF NOT EXISTS, which defines a new database and does nothing to an existing one, so ensure_columns() carries the ALTER for databases that predate the column. Applied to the live router.db. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WSkcSD2Jzkxo1Kw27ecfXJ
10 KiB
10 KiB