Instructions to use ChaoticEconomist/Qwen3-8B-Socratic-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ChaoticEconomist/Qwen3-8B-Socratic-SFT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ChaoticEconomist/Qwen3-8B-Socratic-SFT") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT") model = AutoModelForCausalLM.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ChaoticEconomist/Qwen3-8B-Socratic-SFT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ChaoticEconomist/Qwen3-8B-Socratic-SFT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChaoticEconomist/Qwen3-8B-Socratic-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ChaoticEconomist/Qwen3-8B-Socratic-SFT
- SGLang
How to use ChaoticEconomist/Qwen3-8B-Socratic-SFT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ChaoticEconomist/Qwen3-8B-Socratic-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChaoticEconomist/Qwen3-8B-Socratic-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ChaoticEconomist/Qwen3-8B-Socratic-SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChaoticEconomist/Qwen3-8B-Socratic-SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ChaoticEconomist/Qwen3-8B-Socratic-SFT with Docker Model Runner:
docker model run hf.co/ChaoticEconomist/Qwen3-8B-Socratic-SFT
Qwen3-8B Socratic (SFT)
A supervised fine-tune of Qwen/Qwen3-8B that tutors Socratically. For a learning question it withholds the answer and asks one to three targeted questions that surface the learner's assumptions and point at the next step. For plain instructions (format conversion, SQL, regex, extraction) it still does the task directly.
Standalone model in safetensors: the LoRA was merged into the base weights, so it loads with
from_pretrained and needs no PEFT or inference-time hooks.
TL;DR metrics
Held-out tutoring and plain prompts, judged by Claude Opus 4.5 (thinking off).
| Metric | Base Qwen3-8B | This model |
|---|---|---|
| Socratic rate (asks, withholds the answer) | ~5% | 100% |
| Answer leakage (states the answer anyway) | ~75% | 3% |
| Coherence (1-5) | 3.7 | 4.0 |
| Retain (plain tasks still done directly) | ~99% | 98% |
It flips a model that answers ~95% of tutoring questions outright into one that asks instead, while leaving plain tasks almost entirely alone.
How it was built
Plain supervised fine-tuning. A LoRA (rank 32, alpha 64, on q/k/v/o and gate/up/down projections)
was trained on matched pairs where the input is the question and the target is the Socratic reply, with
loss on the reply tokens only, then merged into the base model.
The one trick that mattered: training only on Socratic replies teaches "always ask", which wrecks plain tasks. So plain-task prompts with ordinary direct answers were mixed into the training set (about 30% of examples), teaching the model when NOT to be Socratic. That single change took retain from 37% to 98% with no loss of Socratic rate.
- Base: Qwen3-8B, bf16
- Data: 282 Socratic contrastive replies + 120 plain-task direct answers
- LoRA rank 32, lr 1e-4, 3 epochs, effective batch 16, merged to bf16 safetensors
Previous approaches (what we tried first)
This model is the end of a short ladder. All numbers are judged the same way on the same held-out sets.
| Approach | Socratic | Leakage | Coherence | Retain | Notes |
|---|---|---|---|---|---|
| RepE steering vectors (runtime) | 95% | 0% | 2.2 | 85% | No weight changes. Add a direction to the residual stream at 3 layers. Needs inference hooks, and the add costs fluency. |
| LoRRA (bake steered acts into a LoRA) | 94% | 5% | 1.9 | 100% | No hooks, perfect retain, but lowest fluency. A low-rank match of steered activations. |
| SFT, Socratic only | 100% | 0% | 3.69 | 37% | Best fluency, but over-generalised: it interrogated plain tasks too ("are you sure 'repeats' is the right word?" when asked to parse a query string). |
| SFT + retain mix (this model) | 100% | 3% | 4.0 | 98% | Mixing plain-task answers in fixed retain while keeping Socratic rate and improving fluency. |
Short version of the lesson: Representation Engineering (steering, LoRRA) is the cleaner research tool and preserves plain behaviour for free, but it caps fluency because a single direction cannot inject reasoning structure. Plain SFT reads best and is simplest to ship, but you have to teach it the negative case (when to stay direct) explicitly, or it over-applies the behaviour.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT")
model = AutoModelForCausalLM.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT", dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Why does my Python for loop print the last value five times?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=300)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Expected reply is questions, not an answer.
Limitations
- Strongest on tutoring-shaped questions. Terse or ambiguous prompts may still get a direct answer.
- Thinking mode: the visible reply is Socratic, but the reasoning trace can contain the full solution.
- English only. Judged by an LLM, not by human raters.
- Research artifact. The retain mix is ~30% of training, so very unusual plain tasks may occasionally get a stray question.
Method references
- Zou et al., Representation Engineering: A Top-Down Approach to AI Transparency (2023), including LoRRA, for the approaches this was compared against.
- Downloads last month
- 166