Instructions to use Nicholas55555/qwen2.5-1.5b-weather-intent-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Nicholas55555/qwen2.5-1.5b-weather-intent-v2 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct") model = PeftModel.from_pretrained(base_model, "Nicholas55555/qwen2.5-1.5b-weather-intent-v2") - Notebooks
- Google Colab
- Kaggle
qwen2.5-1.5b-weather-intent-v2
QLoRA adapter for Qwen/Qwen2.5-1.5B-Instruct β a production-hardened weather-intent parser (v2). Maps a user message to a structured JSON intent, and rejects what it can't answer instead of confabulating.
v2 over the v1 SFT baseline (which hit 98.6% exact-match but confabulated on out-of-scope input) adds three independently-measured levers:
- reject training β an explicit
intent: "out_of_scope"class + boundary-negative data (fixes confabulation) - grammar-constrained decoding β XGrammar JSON-schema, making out-of-vocabulary values impossible (fixes surface form)
- a deterministic gate β
route()decides accept vs escalate from the structured output (model-cascade)
{"intent": "weather", "location": "Paris", "date": "this_weekend", "metric": "rain", "time_of_day": null}
{"intent": "out_of_scope", "location": null, "date": null, "metric": null, "time_of_day": null}
Schema
| field | values |
|---|---|
intent |
weather, out_of_scope |
location |
place name, or null |
date |
today, tomorrow, day_after_tomorrow, this_weekend, next_weekend, next_week, a weekday, or null |
metric |
temperature, rain, snow, wind, humidity, uv, cloud, general |
time_of_day |
morning, afternoon, evening, night, or null |
Results (300 held-out; base few-shot, ft/ft+grammar zero-shot; greedy)
| metric | base | ft | ft_grammar |
|---|---|---|---|
| in-scope slot exact-match | 52.0% | 99.6% | 100.0% |
| OOS recall (rejects unanswerable) | 80.0% | 100.0% | 100.0% |
| FAR β false-accept / confabulation | 20.0% | 0.0% | 0.0% |
| FRR β false-reject | 20.9% | 0.4% | 0.0% |
| out-of-vocab errors | 30 | 1 | 0 |
The honest read: reject-training does the heavy lifting (ft alone drops false-accept to 0); grammar is the last-mile guarantee that drives out-of-vocab to a structural 0. The gate turns structure into a cheap-over-escalation routing decision. Grammar guarantees validity, not correctness β the constraint tax is measured, not assumed.
Training
- QLoRA (4-bit NF4, double-quant, bf16), LoRA r=16 / alpha=32 on all linear layers
- 1 epoch, effective batch 16, LR 0.0002 cosine, assistant-only loss
- 1500 examples (25% out-of-scope, 4 balanced strata); single RTX 3050 Ti (4 GB)
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = "Qwen/Qwen2.5-1.5B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base, device_map="auto"), "Nicholas55555/qwen2.5-1.5b-weather-intent-v2")
sys = "You extract structured intent from user messages about weather. Return ONLY a JSON object..."
msgs = [{"role": "system", "content": sys}, {"role": "user", "content": "how's the pollen in Austin?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(tok.decode(model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=64)[0], skip_special_tokens=True))
# -> {"intent": "out_of_scope", ...} (pollen is not a supported metric)
For guaranteed-valid output, pair with XGrammar constrained decoding against the schema.
- Downloads last month
- 18