adlee-was-taken 0b85b6366c feat(eval): promote held-out set to evals/heldout.yaml ...

Copy plans/local-decision-classifier-heldout.yaml (30 tasks) to
evals/heldout.yaml unchanged, and add a loader test asserting the
file loads 30 scoreable tasks with the expected categories.

The held-out set is weak: single author, short prompts, no true
distributional shift from the training set. Treat 100% as a ceiling,
not a forecast -- it cannot measure generalization.

2026-09-28 19:56:55 -04:00

..

synthetic

feat(local-encoder): trainable LogisticRegression head with synthetic corpus

2026-09-23 01:22:36 -04:00

heldout.yaml

feat(eval): promote held-out set to evals/heldout.yaml

2026-09-28 19:56:55 -04:00

tasks.yaml

fix(evals,seed): unanswerable diff_checking question + two seed-script defects

2026-09-03 19:33:14 -04:00