Instructions to use rycerzes/Qwen3.5-4B-Decision with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rycerzes/Qwen3.5-4B-Decision with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rycerzes/Qwen3.5-4B-Decision")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rycerzes/Qwen3.5-4B-Decision")
model = AutoModelForCausalLM.from_pretrained("rycerzes/Qwen3.5-4B-Decision", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rycerzes/Qwen3.5-4B-Decision with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rycerzes/Qwen3.5-4B-Decision"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rycerzes/Qwen3.5-4B-Decision",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rycerzes/Qwen3.5-4B-Decision

SGLang

How to use rycerzes/Qwen3.5-4B-Decision with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rycerzes/Qwen3.5-4B-Decision" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rycerzes/Qwen3.5-4B-Decision",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rycerzes/Qwen3.5-4B-Decision" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rycerzes/Qwen3.5-4B-Decision",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rycerzes/Qwen3.5-4B-Decision with Docker Model Runner:
```
docker model run hf.co/rycerzes/Qwen3.5-4B-Decision
```

Qwen3.5-4B-Decision

A decision ("System One") model: Qwen/Qwen3.5-4B trained to answer typed questions about a state with a probability for every option, from one forward pass and without generating text:

noul: a yes/no question, answered with the probability of true.
choice: one option among named options.
score: a level on an ordinal scale.

It is trained on the exact prompt of the /v1/systemone endpoint of transformers serve: the prompt describes the question with one label per option (Yes/No, or A, B, C, ...), and the next-token logits of the labels at the end of the prompt, normalized with a softmax, are the probabilities of the options. Serve it as is, with no decision config:

pip install "transformers[serving]>=5.20.0" requests
transformers serve

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "model": "rycerzes/Qwen3.5-4B-Decision",
  "state": "Our checkout has been returning 500 errors since this morning and customers cannot pay. We have lost dozens of orders.",
  "questions": {
    "team": {"type": "choice", "instructions": "Which team handles this?", "criteria": {"billing": "Charges, invoices and refunds", "engineering": "Bugs and outages", "sales": "New contracts"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?", "criteria": ["Can wait", "This week", "Today", "Right now"]},
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}
  }
}'

{
  "team": {"type": "choice", "choice": "engineering", "probabilities": {"billing": 0.004, "engineering": 0.996, "sales": 0.001}, "confidence": 0.994},
  "urgency": {"type": "score", "score": 2.98, "probabilities": {"0": 0.000, "1": 0.000, "2": 0.023, "3": 0.976}, "confidence": 0.975, "legend": {...}},
  "refund": {"type": "noul", "noul": 0.119}
}

(the answers of the response, rounded)

For speed, install the kernels of Qwen3.5's linear attention: pip install flash-linear-attention and pip install causal-conv1d --no-build-isolation.

Training

LoRA (r=16, all linear layers of the text model, merged) with an SFTTrainer subclass from the TRL example examples/sft_decision_model, whose loss is the cross-entropy between the probabilities of the options and the soft targets of the data. The data is 40k questions of tasksource/tasksource-jev-typed-decisions from its sources licensed for commercial use (half choice, a quarter noul, a quarter score), of which about 30.7k have prompts of at most 512 tokens. One epoch, learning rate 1e-4, 32 questions per step: 108 minutes on one L4 (24 GB), 10.6 GB peak. Only the text model is kept: the checkpoint is Qwen3_5ForCausalLM, without the vision tower.

The training curves (loss, accuracy and Brier score on the training questions, learning rate, gradient norm, and CPU, memory and disk usage) are on the Trackio dashboard of the run: the loss goes from 0.98 to 0.59 and the accuracy from 59% to 78%.

Evaluation

Through the /v1/systemone endpoint, on two held-out sets, with all the questions of a state in one request:

LocalLLaMA/typed-decisions, test split: 400 states, 2,000 questions, soft gold from 3 annotators.
JevBench/jevbench: 240 states, the 1,938 questions whose gold is one of their options (302 have the gold unknown).

None of their states is in the training questions, and none shares a 13-word sequence with them. Accuracy counts the questions whose most probable option is the most probable in the gold; the Brier score is the squared distance between the probabilities of the options and the gold ones (0 is perfect, 2 the worst).

	typed-decisions accuracy	typed-decisions Brier	JevBench accuracy	JevBench Brier
Qwen3.5-4B	56.0%	0.331	87.4%	0.184
Qwen3.5-4B-Decision	67.3%	0.137	89.1%	0.141

By question type:

	typed-decisions noul	choice	score	JevBench noul	choice	score
Qwen3.5-4B	65.5%	61.5%	44.9%	97.4%	86.5%	40.8%
Qwen3.5-4B-Decision	74.5%	68.2%	61.4%	98.4%	89.6%	42.9%

Limitations

Read with the /v1/systemone prompt and its labels: another prompt or another label set is not what the model was trained on.
Up to 26 options per question (labels A to Z). Text only; prompts longer than 512 tokens were not trained on.
Ordinal score questions remain the weakest type on JevBench (42.9%).
The probabilities come straight from the softmax of the label logits: no temperature was fitted.