Yev-Decision-1

A Qwen3.5-4B decision model, trained without Jev.

A local System One decision model. Give it a state, which can be text, images or both, plus named questions. It returns a typed probability distribution for each one: a Choice between options, a Score on an ordered scale, or a Noul (yes or no). It generates no text and samples nothing, and nothing leaves the machine. The call shape mirrors TypeSafe's SDK for its hosted jev-1.13.0, but no Jev output was used anywhere in its training or calibration: its labels come from people, from code, and from open-weight teachers under MIT or Apache-2.0.

This repository is a yev export, not a standalone transformers model. It holds:

• LoRA adapters, rank 128;
• the decision heads and the three temperatures;
• the tokenizer and processor files;
• metadata.json, which records provenance;
• sources.json, every training source with its licence and where that licence was checked.

The backbone, Qwen/Qwen3.5-4B-Base, is downloaded at a pinned revision on first load. It runs with the yev package.

This is the second revision (2026-10-26). It adds a Breadth run on decisions earlier builds never trained on: code, tables, logs, games, spatial and unit reasoning, agent traces and safety. The first release (2026-10-21) is the tag sharp-0.5-verified: load it with Yev("HallD/Yev-Decision-1", revision="sharp-0.5-verified").

Use

uv add "yev[inference] @ git+https://huggingface.co/HallD/yev"

from yev import Yev, Choice, Image, Noul, Score

with Yev("HallD/Yev-Decision-1") as client:
    response = client.system_one(
        state="I was charged twice. Please fix this ASAP.",
        questions={
            "billing": Noul(instructions="Is this ticket about billing?"),
            "tone": Choice(instructions="What is the customer's tone?",
                           criteria={"calm": None, "frustrated": None, "angry": None}),
            "urgency": Score(instructions="How urgent is this ticket?",
                             criteria=["can wait", "this week", "today"]),
        },
    )
    chart = client.system_one(
        state=["Quarterly sales:", Image("chart.png")],
        questions={"growing": Noul(instructions="Do sales rise every quarter?")},
    )

Each request's questions are scored independently, as separate sequences.

• A state and its question read up to 4,096 tokens, and each picture is scaled to at most 256 image tokens.
• For longer text, windows="mean" scores overlapping windows and averages them.
• windows="max" (Noul only) answers from the window most likely to be Yes, such as the email in an inbox that holds an instruction.
• On a CUDA GPU, quantize4bit=True loads the backbone in NF4. On an earlier build it cost 0.8 points on jev-bench, mostly MMLU and BoolQ, and about a point on VizWiz and the people-labelled pictures; it has not been measured on this one.

Results

Clean teacher-check bundle: 5,382 held-out questions with no Jev answers, each model using the temperatures it ships with. The first release is this repository's tag sharp-0.5-verified. yev 4B is HallD/yev-qwen3.5-4b-v2, the same backbone trained in part on Jev's labels.

	This revision	First release	yev 4B
CFPB complaints (3,000): product area against people's labels	88.9%	89.2%	87.8%
jev-bench (12 human-labelled datasets, 2,382 questions): accuracy	84.1%	83.5%	84.2%
jev-bench: expected calibration error	0.015	0.028	0.071
jev-bench: TVD to annotators' vote shares	0.087	0.097	0.085

With temperatures refitted the same way for all three, jev-bench calibration error is 0.018, 0.016 and 0.048, and TVD to annotators 0.087, 0.105 and 0.074.

Strongest	Weakest
SMS spam 97.3%, civil comments 96.5%, FEVER 96.5%, StrategyQA with evidence 95.0%, BoolQ 94.5%, ARC-Challenge 92.0%, MNLI 92.0%	HelpSteer2 helpfulness 43.5%, StrategyQA closed-book 70.0%, HelpSteer2 verbosity 70.5%, MMLU 72.0%

Image questions: dev pictures from the training sets' own dev splits. Accuracy is the share of questions where the people's most common answer comes top.

	This revision	First release	yev 4B
CLEVR (1,000)	97.9%	98.3%	97.7%
TextVQA (1,000)	82.8%	84.0%	83.5%
VizWiz (956)	81.1%	81.4%	77.5%
Pictures of text labelled by people (515): picture / same question as text	89.1% / 90.1%	89.3% / 89.7%	90.3% / 90.9%
Calibration error on those pictures	0.018	0.023	0.016

Definitions: 228 questions whose answer follows from a definition the question gives, so the right answer is known. They cover thresholds ("a parcel is a heavy parcel if it weighs more than 25 kg"), lists of conditions, and features under a definition that can run against the usual meaning, asked mirrored, in the question, in the true description only, and inverted. Flips ask about the same state under two thresholds whose answers differ; both must be right.

	This revision	First release	yev 4B
All 228: accuracy	0.93	0.95	0.93
Threshold flips (20 pairs): both right	1.00	1.00	0.90
Thresholds	1.00	1.00	0.97
Features	0.78	0.83	0.85
Conditions	1.00	1.00	1.00
Inverted questions	0.91	0.96	1.00

Consistency: the average violation over about 150 dev questions per test, where 0 is perfectly consistent.

• Option order: reordering a Choice's options.
• Binary: the same yes/no question asked as a Noul and as a two-option Choice.
• Irrelevant option: dropping one option.
• Definition swap: a Noul asked with its two descriptions exchanged, under neutral instructions.

	This revision	First release	yev 4B
Option order (TVD)	0.000	0.000	0.000
Binary: yes/no against two-option Choice (P(yes) gap)	0.013	0.024	0.023
Irrelevant option dropped (TVD)	0.020	0.023	0.021
Definition swap (P(true) + P(true after swap), gap from 1)	0.055	0.043	0.066

S1-Arena (S1-Arena at 28e16b1, the static sweep run with the benchmark's own tools): accuracy is the leaderboard's, the mean over the 87 leaf datasets of its 13 non-agentic domains. Jev 1.13's figures are the leaderboard's.

	This revision	First release	Jev 1.13
Accuracy (87 leaves)	65.6%	64.4%	72.6%
Brier score	0.413	0.433	0.349
Expected calibration error	0.118	0.126
Static agent steps (7 leaves): accuracy	60.3%	59.5%	67.8%

Against the first release, every domain group the sweep reports is up except forecasting and judging, which are level: embodied 63.2% against 60.0%, safety 77.0% against 74.1%, code 58.5% against 56.7%, routing 56.9% against 55.3% and social 62.1% against 60.5%. The largest gains are BeaverTails (80.4% against 63.6%), ACPBench failure types (44.8% against 30.0%), SemEval-2010 Task 8 (65.7% against 52.4%) and AgentDojo injections (91.0% against 78.9%). The largest losses are floor-plan collisions (64.7% against 73.7%), WMT23 translation quality (28.0% against 35.2%), PlanBench plan checking (60.2% against 66.9%), NSL-KDD intrusion classes (31.1% against 37.1%) and BGL log categories (79.0% against 84.4%). Code vulnerability detection stays at chance (50.0%).

About a dozen S1-Arena leaves draw on a dataset the Breadth run also trained on, among them SemEval-2010 Task 8, EQUATE, WANDS, PRM800K, e-CARE, WiCE, VitaminC and banking77. Only those datasets' training partitions were used, and every Breadth state sharing an 8-word run with an S1-Arena request or a JevBench public item was removed. The blend weight was chosen on the bundle above; S1-Arena was run on this blend and on a 50% blend as a check.

JevBench public items (231), scored for every candidate as a check, not used to choose: 200 right, Brier 0.195, against 197 and 0.192 for the first release and 198 and 0.182 for Jev 1.13.0 (hosted API, 2026-10-11).

Speed: median 51 ms a request, p90 155 ms, from S1-Arena's latency probe (200 requests on one Colab GPU), the same as the first release.

How it was trained

The model is a frozen Qwen3.5-4B-Base with its language-model head removed. On top sit:

• LoRA on the attention and Gated DeltaNet projections;
• a bilinear head over pooled spans, which scores Choice options in sorted order, so their order cannot change the answer;
• a readout of the last token of the question and of each option, added to the pooled spans, which tells apart options that differ by a word ("is a taco" against "is not a taco").

It was trained from the base model with RLCD's objective (Plackett–Luce plus a Brier term):

Run	Data	Updates
Stage A	2.25 million questions from licensed human-labelled datasets, code-labelled definitions and open-teacher logits	71,002
Vision	CLEVR, TextVQA and VizWiz pictures, rendered pictures of text and a text replay (137,072 questions); blended 70/30 with Stage A	6,984
Hard	31,784 questions, about half of them long-document, multi-hop, numeric and judgement questions written and answered by Qwen3.8-27B and DeepSeek-V4.1-Flash, the rest replayed from Stage A's sources; blended 70/30 with its start	1,111
Kev	684,459 answer distributions from Kev-27B plus 768,333 questions replayed from Stage A's data	43,513
Sharp	35,821 questions with one certain answer: both hard sets' questions (the second written and answered by Qwen3.8-27B, with answer-judging questions on GSM8K and ARC-Challenge), each target its checked answer, computed by code or agreed by two blind solves, and the replayed people- and code-labelled questions that have one answer. It starts from 40% of the Kev run blended with 60% of its start	2,194
Breadth	869,458 training questions from 459 sources, one pass: tasksource's typed decisions with each source's own labels, decisions generated with code-computed answers (SQL results, program output and tests, unit and date arithmetic, logs, tables, grids, puzzles and games, agent-trace failures), and the Sharp run's data as replay. It starts from the first release, whose rank-32 adapter it carries inside a rank-96 one with 64 new ranks that start at zero	50,895

This export is an exact blend: 30% of the Breadth run's last update and 70% of the first release (itself 50% of the Sharp run's last update and 50% of its start), its two adapters stacked into one of rank 128. Of blends of 30%, 40%, 50% and 100%, it kept the 50% blend's jev-bench accuracy and the most of the first release's picture, definition and consistency scores.

Temperatures were fitted on 2,752 dev questions whose answer was checked rather than voted on: the hard sets' questions, the code-labelled definitions and the Breadth run's own code-labelled decisions. Choice 1.682, Noul 1.542, Score 1.414. Earlier builds fitted them to annotators' vote shares, which left the model less sure than it is right on questions that have one answer; the Sharp run is there for the same reason, since Kev-27B's answer distributions taught it to hedge.

Data and licences

Every source is in sources.json. The rules for what it could train on:

• Licences that allow commercial use: CC0, public domain, MIT, Apache-2.0, BSD, AFL, ODC-BY, CDLA-Permissive, OANC, CC BY and CC BY-SA.
• Labels from people, from code, or from open-weight teachers under MIT or Apache-2.0. Never from Jev or from a closed model.
• Left out: non-commercial, research-only or unlicensed sets, platform terms (Twitter, Yelp, Yahoo, Quora, Amazon reviews), GPL, mixed sets that cannot be split, unverified sets, and COCO images.

The teachers:

• Qwen/Qwen3.5-35B-A3B-Base (Apache-2.0): logits on question-bank questions;
• Qwen/Qwen3.8-27B (Apache-2.0) and deepseek-ai/DeepSeek-V4.1-Flash (MIT): the hard questions, among them Qwen3.8-27B's answers to GSM8K (MIT) and ARC-Challenge (CC BY-SA 4.0) training questions, graded by code against the datasets' own answers;
• jaredpalmer/kev-27b at af0e6d5 (Apache-2.0): answer distributions over four option orders. Its card says no Jev output was used in its training, and that its consumer-complaint labels were filtered by closed-model judges.

The Breadth run's sources:

• tasksource/tasksource-jev-typed-decisions at 3984508: only rows marked for commercial use whose every recorded licence is on the list above, from each source's training partition. Despite its name, its targets come from each source's own annotations, not from Jev;
• tasksource/procedural-typed-decisions at d24af62 (Apache-2.0) and the tev1 generators at 1dde778 (MIT), whose answers come from code;
• decisions yev generates itself, labelled by the code that computes their answer.

Some source texts were written by models and labelled by people under their datasets' licences: WANLI (GPT-3, CC BY 4.0), hh-rlhf (MIT) and HelpSteer2 and 3 (CC BY 4.0). The pictures are CLEVR (CC BY 4.0), TextVQA (CC BY 4.0, with Open Images pictures listed as CC BY 2.0) and VizWiz (CC BY 4.0). The base model is Apache-2.0.

The weights are released under Apache-2.0. The CC BY and CC BY-SA sources ask for attribution, which sources.json gives. Check each source's terms before redistributing any of the data.

Limitations

• Against the first release it is weaker on definitions that run against a word's usual meaning (78% against 83%) and on inverted questions (0.91 against 0.96), on TextVQA (82.8% against 84.0%), and on the definition swap (0.055 against 0.043). Pin revision="sharp-0.5-verified" if those matter more to you than breadth.
• Annotator agreement: its distributions sit 0.087 TVD from annotators' vote shares on jev-bench, against 0.085 for yev 4B.
• MMLU is 72.0%, against 74.5% for an earlier build without the Kev and Sharp runs.
• Fine-grained ratings are weak: HelpSteer2 helpfulness 43.5%, verbosity 70.5%; WMT translation quality on S1-Arena is 28%.
• S1-Arena is still 7 points behind Jev 1.13. Forecasting (40%), games (42%), routing by difficulty (about 23%) and code vulnerability detection (chance) are the weakest areas.
• Image accuracy is measured on these multiple-choice questions, not with each benchmark's official metric.
• Prompt injection: it isn't a prompt-injection guard. On S1-Arena's injection sets it scores 91.0% on AgentDojo, 63.9% on BIPIA and 84.3% on ToxicChat jailbreaks.
• Validate on your own questions before relying on a threshold.