fix(classifier): local_encoder default model is gated, switch to bart-large-mnli #48

Merged
alee merged 1 commits from fix/local-encoder-default-model into main 2026-09-07 00:34:48 +00:00
5 changed files with 30 additions and 8 deletions

View File

@@ -867,7 +867,7 @@ function classifierModeFieldsHtml(mode, data) {
<div class="row g-2">
<div class="col-md-6">
<input class="form-control form-control-sm" id="classifier-encoder-model"
placeholder="model (default: MoritzLaurer/deberta-v3-base-zeroshot-v2)"
placeholder="model (default: facebook/bart-large-mnli)"
value="${escapeHtml(enc.model || '')}">
</div>
<div class="col-md-3">

View File

@@ -707,7 +707,7 @@ classifier:
# Read only when mode: local_encoder. Every field has a default, so
# `encoder: {}` is enough to opt in.
# encoder:
# model: MoritzLaurer/deberta-v3-base-zeroshot-v2
# model: facebook/bart-large-mnli
# device: cpu
# confidence_threshold: 0.5

View File

@@ -213,10 +213,24 @@ rarer each time, but never structurally impossible, because a chat-completion
model can always in principle spend its budget thinking instead of
answering. `classifier.mode: local_encoder` sidesteps the whole failure class
instead of picking around it: a zero-shot NLI encoder
(`classifier.encoder.model`, default `MoritzLaurer/deberta-v3-base-
zeroshot-v2`) scores the task directly against `proficiency.categories` and
returns a label plus a confidence — there is no generation step, so there is
no trace to run away.
(`classifier.encoder.model`, default `facebook/bart-large-mnli`) scores the
task directly against `proficiency.categories` and returns a label plus a
confidence — there is no generation step, so there is no trace to run away.
(Was `MoritzLaurer/deberta-v3-base-zeroshot-v2` — smaller, ~184M vs ~407M
params — until that repo started returning 401 on an unauthenticated GET of
its own model page sometime after this project picked it, discovered live
2026-09-06 when it took production down: the startup check correctly
refused to boot rather than fail opaquely on the first request, but the
service still crash-looped until the default was corrected.)
`classifier.encoder.confidence_threshold` is a `[0.0, 1.0]` probability
(`classify_zero_shot`'s own output), not a percent — the admin UI takes 0-100
for a human to type and converts at the save boundary, but a config file
edit or any other caller must use the raw probability. Config load now
validates the range; a value like `80` used to be silently accepted and
would make every real confidence score read as below-threshold, since none
can exceed `1.0` (also caught live 2026-09-06, before a restart made it
active).
The trade is real, not free. It only produces `task_category` — no `task_tier`
signal exists in a zero-shot label score, so tier falls back to

View File

@@ -791,7 +791,15 @@ class LocalEncoderConfig(StrictModel):
learn from without a new, separate opt-in data-capture feature.
"""
model: str = "MoritzLaurer/deberta-v3-base-zeroshot-v2"
# facebook/bart-large-mnli -- the reference model HuggingFace's own docs
# use for this exact pipeline. Was MoritzLaurer/deberta-v3-base-zeroshot-v2
# (smaller, ~184M vs ~407M params) until that repo started returning 401
# even on an unauthenticated GET of its model page -- gated or moved
# sometime after this project picked it. Caught live 2026-09-06: the
# startup check (ensure_available) correctly refused to boot rather than
# fail opaquely on the first request, but it still took production down
# until the default was fixed.
model: str = "facebook/bart-large-mnli"
device: Literal["cpu", "cuda"] = "cpu"
# Below this, the classification is treated as a FAILURE, not a low-
# confidence answer -- the caller cascades exactly as it would for a

View File

@@ -122,7 +122,7 @@ def test_local_encoder_with_empty_encoder_block_loads_with_defaults(raw):
cfg["classifier"]["mode"] = "local_encoder"
cfg["classifier"]["encoder"] = {}
loaded = RouterConfig(**cfg)
assert loaded.classifier.encoder.model == "MoritzLaurer/deberta-v3-base-zeroshot-v2"
assert loaded.classifier.encoder.model == "facebook/bart-large-mnli"
assert loaded.classifier.encoder.device == "cpu"
assert loaded.classifier.encoder.confidence_threshold == 0.5