Instructions to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
# Run inference directly in the terminal:
./llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Use Docker

docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

LM Studio
Jan

vLLM

How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Ollama
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with Ollama:
```
ollama run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
```
Unsloth Desktop
Docker Model Runner
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with Docker Model Runner:
```
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS
```

Lemonade

How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF:IQ2_XXS

Run and chat with the model

lemonade run user.DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF-IQ2_XXS

List all available models

lemonade list

Atomic Chat

DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF

Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.

Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).

Model Summary

Foundation Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (28 Layers, 28:4 GQA, SwiGLU FFN, Test-Time Compute Reasoning)
Quantization Precision: ~2.06 bpw (2.12 GiB)
Methodology: Extreme sub-2.1-bit frontier reasoning checkpoint compressed to 2.1 GiB. Enables deployment of a 7.6B test-time compute reasoning model on lightweight edge devices and mobile hardware.
Continuous Holdout Perplexity (131k tokens): 4.1951
GSM8K Native CoT Proofs: 22/25 (88.0%)
Average Thinking Trace: 443.6 tokens (</think> Closure Rate: 88.0%)
Olympiad Competition Math: 3/10 (30.0%)
Python Code AST Execution: 0/10 (0.0%)
Inference Decode Throughput: 194.2 t/s on NVIDIA GeForce RTX 4080 Super (32GB VRAM)

Theoretical Methodology

Extended chain-of-thought models rely on dynamic test-time compute, generating hundreds of intermediate deliberation tokens inside <think>...</think> scratchpads. Discretization noise from standard PTQ accumulates multiplicatively across intermediate reasoning steps, leading to proof derailment and infinite repetitive cycles.

DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: $\Omega_i = \frac{1}{d} \left( \| \tilde{H}_{i,:} \|_2^2 - \tilde{H}_{ii}^2 \right)$ and projecting parameter updates strictly into the contractive Lyapunov half-space: $\operatorname{Re}(\lambda(S)) \le -\delta$ G-TAP dampens back-reaction noise and preserves reasoning attractor basins across extended thinking sequences.

Usage Instructions

Native Prompt Template

DeepSeek-R1 models require the native prompt format to activate internal chain-of-thought:

<｜begin of sentence｜><｜User｜>{prompt}<｜Assistant｜><think>

Running with llama-cli

llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF -p "<｜begin of sentence｜><｜User｜>Solve for all real roots: x^4 - 4x^3 + 6x^2 - 4x + 1 = 0.<｜Assistant｜><think>\n" -n 2048 -c 8192 --temp 0.6 --top-p 0.95

Serving with llama-server

llama-server -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF --port 8080 -c 8192 -ngl 99 -fa on

DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed

Downloads last month: 234

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

2-bit

Model tree for DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_XXS-GGUF

Base model

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Quantized

(203)

this model