orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF

DeepSeek-V4.1-Flash-1017-abliterated GGUF

🛡️ New from Orca — OrcaCyber Zero 1.5 Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.01% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API.

🛡️ New from Orca — OrcaCyber Flash 1.0 No 236 GB box? Run DeepSeek V4.1 Flash Abliterated as a hosted, OpenAI-compatible API on OrcaRouter: no download, no GPUs, one API key.

The 763B uncensored DeepSeek-V4.1-Flash-1017, on one box.

236 GB in memory · 197B-parameter Engram streamed from your SSD · llama.cpp

OrcaRouter AI Gateway · X · Discord · GitHub · All Models

---

The full DeepSeek-V4.1-Flash-1017, run locally.

These are GGUF builds of the abliterated (refusal-removed) DeepSeek-V4.1-Flash-1017. The model is a 763B-parameter MoE with 384 routed experts (top-6), a 197B-parameter Engram n-gram memory, compressed sparse attention and a 1M-token context. The routed experts are the only part the bits were spent on: every layer's experts got their own precision from measured KL divergence, not a hand-set rule.

The Engram table is a lookup that reads 48 rows per token. llama.cpp maps it from disk and reads it on demand, so it lives on your SSD, not in RAM. What has to fit in memory is everything else.

---

⚠️ Disclaimer — read before use

This model is built on an abliterated base whose safety alignment has been substantially reduced: the refusal direction is orthogonalized out of every matrix that writes the residual stream. It will comply with many harmful, unethical, offensive, or illegal requests that the original DeepSeek-V4.1-Flash-1017 would refuse.

• It is released strictly for legitimate research: interpretability, AI-safety and refusal-mechanism study, red-teaming, and robustness evaluation.
• You assume full responsibility for how you use it and for everything it generates. Add your own safety, moderation, and abuse-prevention layers before any deployment.
• Use must comply with the MIT License inherited from the base model and with all applicable law.
• The authors accept no liability for misuse. Its outputs do not reflect the views of the uploaders or of DeepSeek.

---

Pick a build

Build	In memory	Download	Experts	KLD ↓	Top-1 ↑	PPL vs. original
Mix-256G	235.8 GB	346.5 GB	3.36 bpw	0.051	93.7%	+2.4%
Mix-128G	114.0 GB	225.0 GB	1.59 bpw	0.520	78.5%	+54.7%

Mix-256G is the build to use. It needs a box with 256 GB of RAM, unified memory, or VRAM. At 3.36 bits per expert weight, the model's next-token choice matches the unquantized model on 93.7% of held-out tokens.

Mix-128G fits 128 GB, but it is not a substitute. All experts at llama.cpp's smallest type (IQ1_S) already take 111 GB, so a 128 GB box is at the floor: perplexity is 55% higher and one token in five differs from the original. Use it to try the model, not to rely on it.

In memory is everything except the Engram table. Download adds the 110.6 GB Engram table, which stays on disk.

---

Evaluation

Both builds were scored against the lossless GGUF of the same checkpoint (MXFP4 experts, BF16 dense, MXFP8 Engram, converted bit-exact) on held-out text: wikitext-2 test, characters [0, 300000), 32 windows of 2048 tokens, full-vocabulary KLD. The precision search never saw this text. Each build is compared with llama.cpp's own mixture at a similar size, quantized from the same weights with the same importance matrix:

	In memory	KLD ↓	KLD p99	Top-1 ↑	PPL (original 2.259)
Mix-256G (this repo)	235.8 GB	0.0507	0.64	93.74%	2.313 (+2.4%)
llama.cpp IQ3_XXS	212.2 GB	0.2704	3.16	85.12%	2.841 (+25.8%)
Mix-128G (this repo)	114.0 GB	0.5202	4.95	78.50%	3.494 (+54.7%)
llama.cpp IQ1_S	111.1 GB	1.7307	9.02	54.80%	11.858 (×5.2)

Measured with the Engram table at MXFP8. The released files carry it at IQ4_NL (110.6 GB instead of 202.8 GB). The swap was not rescored. Per unsloth's measurements on this model, the Engram precision barely moves top-1.

Method note. These are full-vocabulary KLD and next-token agreement on held-out text at 2k context, not a downstream benchmark.

Refusal and capability of the abliterated model

Measured on the unquantized abliterated checkpoint (orcarouter/DeepSeek-V4.1-Flash-1017-abliterated), not re-measured on these GGUF builds. They are copied here from that model's card. The KLD above is how far each build sits from that checkpoint.

Setup.

• Hardware and runtime: 8×H200, the checkpoint's own reference implementation under tensor parallelism.
• Decoding: greedy, thinking_mode="chat", 64 new tokens.
• Scoring: refusal is judged by a rule-based opening-phrase classifier. caveat means the model answered but wrapped the answer in a disclaimer. Treat the numbers as indicative, not publication-grade.

Safety: harmful-prompt refusal (lower = more uncensored). Weighted mean over all 500 prompts: 0.074, against 0.874 for the base checkpoint.

Benchmark	n	Base	Abliterated	Δ	Caveat
JBB-harmful	100	0.950	0.030	−97%	0.360
MaliciousInstruct	100	0.930	0.020	−98%	0.610
ForbiddenQuestions	150	0.667	0.073	−89%	0.447
SimpleSafetyTests	50	1.000	0.120	−88%	0.560
AdvBench	100	0.990	0.150	−85%	0.550

Over-refusal: benign prompts wrongly refused (lower = better). Weighted mean over all 350 prompts: 0.006, against 0.069 for the base checkpoint.

Benchmark	n	Base	Abliterated	Δ	Caveat
XSTest-safe	250	0.052	0.008	−85%	0.028
JBB-benign	100	0.110	0.000	−100%	0.140

Capability retention.

Benchmark	n	Base	Abliterated
MMLU (zero-shot, answer letter read from the next token)	300	0.910	0.920

MMLU is unchanged; the one-point difference is within the sampling error of 300 questions. It is the only capability measured.

---

How the precision was chosen

Each tier was searched separately, with the budget set on the bytes that have to be in memory:

1. Measure. For every group of tensors, the search measures the real KLD of moving it one precision level up and one level down. A group is one layer's experts, with gate+up and down kept apart, or one kind of dense matrix. Each measurement runs llama.cpp's own kernels.
2. Choose. A knapsack picks the combination of moves that fits the budget exactly.
3. Accept. Only steps whose candidate KLD was actually measured lower are kept. The search repeats until it converges.

The searched 256G build puts its bits where they count:

Layers	Experts
24–30 and 32	entirely at DeepSeek's own MXFP4
15, 20, 23, 31, 33, 37, 39	ffn_down at MXFP4
0–3, 6, 7	gate/up at IQ2_S
everything else	IQ3_XXS / IQ3_S

In the same budget, a uniform allocation scores KLD 0.146 and the searched one 0.045 (in-sample). Starting from the uniform allocation, upgrading any single one of the middle layers to MXFP4 removes ~0.04 KLD. Their errors compound downstream rather than add up.

In the 128G build there is almost nothing left to place. Experts are IQ1_S everywhere except a few later layers at IQ1_M or IQ2_XXS.

The other parts are set as follows:

• Attention and shared experts are Q5_K–Q8_0 in 256G and Q4_K–Q6_K in 128G.
• The router, norms and hyper-connection mixers are left as shipped.
• The importance matrix comes from 614k tokens of English/Chinese chat and Wikipedia.

---

What's in this repo

Path	Size	Contents
Mix-256G/…-00001-of-00007.gguf … 00007	346.5 GB	recommended build; 00002 is the Engram table (110.6 GB)
Mix-128G/…-00001-of-00005.gguf … 00005	225.0 GB	128 GB build; 00002 is the Engram table (110.6 GB)
mmproj-DeepSeek-V4.1-Flash-1017-abliterated-BF16.gguf	0.97 GB	vision projector for image input, lossless BF16
mmproj-DeepSeek-V4.1-Flash-1017-abliterated-F16.gguf	0.97 GB	the same in F16, for backends without BF16
imatrix.gguf	1.5 GB	the importance matrix both builds were quantized with

The vision projector works with either build. It was converted from the abliterated checkpoint, whose image entry point (aligner.w2) and image markers carry the refusal edit, so do not swap in the mmproj of the original model. The DeepSeek-V4.1 chat template is embedded in each build. The DSpark speculative drafter is not included.

---

Serving (llama.cpp)

DeepSeek-V4.1 is not in llama.cpp master yet. Build the PR that adds it, ggml-org/llama.cpp#30817. These files were made and tested at commit b72050d03:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/30817/head:pr-30817 && git checkout b72050d03
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-cli

hf download orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF \
  --include "Mix-256G/*" "mmproj-*-BF16.gguf" --local-dir ./dsv41-gguf

# GPUs with ~250 GB of VRAM in total (tested on 2 x H200)
llama-server -m dsv41-gguf/Mix-256G/DeepSeek-V4.1-Flash-1017-abliterated-Mix-256G-00001-of-00007.gguf \
  --mmproj dsv41-gguf/mmproj-DeepSeek-V4.1-Flash-1017-abliterated-BF16.gguf \
  -ngl 99 -c 32768 -np 1 --no-reasoning-preserve

# CPU only, 256 GB+ RAM (tested): weight repacking must be off, see below
llama-server -m dsv41-gguf/Mix-256G/DeepSeek-V4.1-Flash-1017-abliterated-Mix-256G-00001-of-00007.gguf \
  --mmproj dsv41-gguf/mmproj-DeepSeek-V4.1-Flash-1017-abliterated-BF16.gguf \
  -ngl 0 --no-repack --fit off -c 8192 -np 1 --no-reasoning-preserve

• Keep the files on an NVMe SSD and leave mmap on (the default). The Engram table is always kept off the GPU and read lazily. --mlock or --no-mmap would pull all 110 GB into RAM.
• CPU only: pass --no-repack. With the PR at b72050d03, the CPU backend's weight repacking leaves an operation no backend will take. Startup then aborts with GGML_ASSERT(*cur_backend_id != -1). --no-repack --fit off runs.
• Images: pass --mmproj and send OpenAI-style image_url content parts (base64 data URLs work).
• 256 GB of system RAM and one GPU: add --cpu-moe to keep the routed experts in RAM and run attention on the GPU. This setup was not measured here.
• --no-reasoning-preserve drops earlier turns' thinking from the prompt, which is what DeepSeek's reference encoder does. With it, llama.cpp's rendered prompts match encoding.py token for token on DeepSeek's test cases.
• Thinking is on by default. Pass "chat_template_kwargs": {"thinking_mode": "chat"} for a direct answer, or "reasoning_effort": "low" | "medium" | "high" | "max" (or 1–100) to set its depth. The reasoning comes back in message.reasoning_content.
• Set a system prompt. Without one, the model sometimes misnames itself in thinking mode.

Measured with short prompts, one request at a time:

Build	Hardware	Prompt	Generation
Mix-256G	2 x H200 (CUDA)	~110 tok/s	~70 tok/s
Mix-128G	1 x H200 (CUDA)	~110 tok/s	~84 tok/s
Mix-256G + mmproj	CPU only, 64 threads of a 192-core server	~52 tok/s	~10 tok/s

The CPU run shared the machine with an unrelated GPU training job, so it is a floor rather than a peak. Apple Metal and partial-offload (--cpu-moe) speeds were not measured.

Image input was checked on CPU with DeepSeek's own example images. Asked what each food is and which part is eaten, the model answered correctly:

• carrots: the taproot
• corn: the kernels on the cob

---

Model details

Base model	orcarouter/DeepSeek-V4.1-Flash-1017-abliterated, the abliterated build of deepseek-ai/DeepSeek-V4.1-Flash-1017
Architecture	deepseek41 in llama.cpp. 40 layers, hidden 5120, compressed sparse attention, LoRA-factorised output projection (8 groups), 384 routed experts top-6 + 1 shared expert, Engram n-gram lookup at layers 1 and 14, 4-wide Hyper-Connections
Weights in these files	748.5B: 543.6B routed experts, 196.6B Engram, 8.2B attention / shared experts / embeddings / head. ~6 of 384 experts active per token
Quantization	Per-layer mixed precision found by measured-KLD search. Experts are IQ2_S–MXFP4 (256G) or IQ1_S–IQ2_XXS (128G). Dense parts are Q5_K–Q8_0 (256G) or Q4_K–Q6_K (128G). Engram is IQ4_NL. Uses an importance matrix.
Context	1,048,576 tokens (inherited; evaluated at 2k) · Vocabulary 129,280
Runtime	llama.cpp with PR #30817; text and images in (with --mmproj), text out; tool calls in DeepSeek's DSML format

---

Bias, risks, and limitations

• Safety guardrails are substantially reduced (inherited from the abliterated base); see the disclaimer. The refusal and MMLU numbers above come from the unquantized checkpoint. Mix-128G in particular sits far enough from it (KLD 0.52) that its behaviour may differ.
• Mix-128G is heavily degraded. This is the limit of 1.6-bit post-training quantization on this model, not a tuning issue.
• The IQ4_NL Engram in the released files was not rescored. The table above was measured with MXFP8.
• Vision is only lightly checked (two example images, on CPU). The refusal edit also touches the image entry point, and no image-conditioned refusal or quality evaluation has been run. There is no DSpark drafter.
• Inherits the biases and limitations of DeepSeek-V4.1-Flash-1017. Long-context behaviour was not tested: all measurements are at 2k context or on short chats.
• Requires a llama.cpp build with PR #30817. Stock llama.cpp, Ollama and LM Studio will not load these files until it is merged.

License

MIT, inherited from the base model deepseek-ai/DeepSeek-V4.1-Flash-1017. The embedded chat template is unsloth's port of DeepSeek's encoder (unsloth/DeepSeek-V4.1-Flash-1017-GGUF, MIT).

---

One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X