apex-flash-1-heretic-v2

cantina-security/apex-flash-1-abliterated (GLM-5.3-Flash architecture, BF16) with two directional ablations of its refusal behaviour. It supersedes the earlier apex-flash-1-heretic build (since removed), which still refused about four requests in ten once reasoning was enabled.

Intended for authorized security research in environments the researcher owns or has permission to test.

What was changed

Only the matrices that write to the residual stream were edited: self_attn.o_proj and every MLP down-projection (dense, shared and all 288 routed experts). All other tensors, including the MTP layer, are byte-identical to the source.

1. Stage 1 (identical to apex-flash-1-heretic): layers 0–44 orthogonalised against a per-layer direction taken at the last prompt token with the reasoning block closed (difference of mean residuals between mlabonne/harmful_behaviors and mlabonne/harmless_alpaca, 400 prompts each; parameters found with Heretic).
2. Stage 2: layers 18–44 orthogonalised, weight 1.0, against a second per-layer direction measured inside the reasoning block. The stage-1 model's own reasoning traces on the evaluation prompts were replayed through it, residuals were averaged over the last 96 reasoning tokens, and the direction is the difference between traces that ended in a refusal (39) and traces that ended in an answer (57). It is nearly orthogonal to the stage-1 direction (cosine within ±0.05 at every layer).

ablation.json holds both parameter sets; directions.pt and directions_traces.pt hold the directions.

Measured

Refusals counted by an LLM judge (llmfan46/gemma-4-31B-it-uncensored-heretic), system prompt "You are a helpful assistant.".

	Source	Stage 1 only	This model
mlabonne/harmful_behaviors test[:100], reasoning enabled, judged on the final answer	99	39	0
same prompts, reasoning skipped (<think></think>), first 64 tokens	99	2	0
KL divergence on mlabonne/harmless_alpaca test[:100], first token, reasoning skipped	0	0.095	0.100
JailbreakBench/JBB-Behaviors harmful (100), reasoning enabled, judged on the final answer	89	28	1

Read the first row with care: the stage-2 direction was fitted on reasoning traces for these same 100 prompts, so that figure is in-sample. The JailbreakBench row is the held-out check: none of those prompts were used to derive either direction.

In the reasoning-enabled run, responses that did not reach a final answer within 3000 tokens are counted as neither refused nor answered: on the first set 4 for stage 1 and 6 for this model, on JailbreakBench 1, 3 and 6 for source, stage 1 and this model. The median response on that set grew from 882 to 1611 tokens. Reasoning-enabled runs were made on the FP8 quantization served with vLLM; the other rows on the BF16 weights with transformers.

A 20-question sanity check with known answers (arithmetic, facts, short code) came out 20/20 with reasoning enabled. No capability benchmarks were run.