test(incumbent): stamp the cache-rate fixture with now, not a fixed date #109

alee · 2026-10-03T14:43:55Z

alee commented

2026-10-03 14:43:55 +00:00

What

tests/test_incumbent_routing.py::TestDebugLog::test_debug_log_emits_incumbent_identity has been failing on main with no code change. This stamps its fixture with the current time so it cannot age out again.

Why it failed

_seed_measured_cache_rate wrote observed_at='2026-09-13T00:00:00+00:00' into energy_observations. _measured_cache_rates only reads rows inside objective.cache_rate_window_hours, which is 168 (7 days). Around 2026-09-20 the seeded rows left that window, so the incumbent was priced at the assumed rate instead of the measured one, and the test asserting measured failed.

No production code is involved. The router behaves correctly: rows older than the window are supposed to be ignored.

Change

One helper in one test file: _seed_measured_cache_rate now stamps its rows with datetime.now(timezone.utc).isoformat(), with a docstring note on why. The other fixed dates in the file are left alone on purpose. The route_decisions seeds (2026-09-13) are read by _session_incumbent_lookup, which filters on session_key and id with no time window, and the models row (2026-09-01) is not time-filtered by anything this file exercises, so neither can age out the same way.

Verification

tests/test_incumbent_routing.py: 26 passed (was 25 passed, 1 failed).
Full suite in this worktree: 2544 passed, 0 failed.
ruff check on the file reports the same three findings as the base commit (import order, unused Objective, a dict() call), shifted by one line. None are new.
The failure was reproduced on a clean export of main before the change, and the file went to 26 of 26 with only the timestamp changed.

Why now

A red main hides regressions: the last several PRs have each had to say "one pre-existing failure". plans/agent-guardrails.md also plans a verify_known_failures.txt that currently lists this test; with this merged, that file can ship empty.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

## What `tests/test_incumbent_routing.py::TestDebugLog::test_debug_log_emits_incumbent_identity` has been failing on `main` with no code change. This stamps its fixture with the current time so it cannot age out again. ## Why it failed `_seed_measured_cache_rate` wrote `observed_at='2026-09-13T00:00:00+00:00'` into `energy_observations`. `_measured_cache_rates` only reads rows inside `objective.cache_rate_window_hours`, which is 168 (7 days). Around 2026-09-20 the seeded rows left that window, so the incumbent was priced at the assumed rate instead of the measured one, and the test asserting `measured` failed. No production code is involved. The router behaves correctly: rows older than the window are supposed to be ignored. ## Change One helper in one test file: `_seed_measured_cache_rate` now stamps its rows with `datetime.now(timezone.utc).isoformat()`, with a docstring note on why. The other fixed dates in the file are left alone on purpose. The `route_decisions` seeds (`2026-09-13`) are read by `_session_incumbent_lookup`, which filters on `session_key` and `id` with no time window, and the `models` row (`2026-09-01`) is not time-filtered by anything this file exercises, so neither can age out the same way. ## Verification - `tests/test_incumbent_routing.py`: 26 passed (was 25 passed, 1 failed). - Full suite in this worktree: **2544 passed, 0 failed**. - `ruff check` on the file reports the same three findings as the base commit (import order, unused `Objective`, a `dict()` call), shifted by one line. None are new. - The failure was reproduced on a clean export of `main` before the change, and the file went to 26 of 26 with only the timestamp changed. ## Why now A red `main` hides regressions: the last several PRs have each had to say "one pre-existing failure". `plans/agent-guardrails.md` also plans a `verify_known_failures.txt` that currently lists this test; with this merged, that file can ship empty. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

alee added 1 commit 2026-10-03 14:43:56 +00:00

test(incumbent): stamp the cache-rate fixture with now, not a fixed date f769eb4a35

_seed_measured_cache_rate wrote observed_at='2026-09-13T00:00:00+00:00', but
_measured_cache_rates only reads rows inside objective.cache_rate_window_hours
(168). Once the date aged out of that window, around 2026-09-20, the incumbent
was priced at the assumed rate instead of the measured one and
test_debug_log_emits_incumbent_identity failed on main, with no code change.

Stamp the seeded rows with the current time so the fixture cannot age out.
The file goes from 25 of 26 to 26 of 26; the full suite is 2544 passed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkCGRantZsSwmcFpet6FTa

alee merged commit 99b01750cf into main

2026-10-03 14:54:47 +00:00

alee referenced this issue from a commit

2026-10-03 14:54:49 +00:00

Merge pull request 'test(incumbent): stamp the cache-rate fixture with now, not a fixed date' (#109) from fix/incumbent-test-time-bomb into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: alee/6krrt#109