UraionLabs/uraion-forge-2b-gguf

⚡ Uraion Forge 2B (GGUF & Dynamic Quants)

High-Throughput Quantitative Reasoning & Agentic Tool Inference at the Edge

Official Edge Release by Uraion Labs

---

Executive Summary

Uraion Forge 2B GGUF is the official edge-optimized distribution of Uraion Forge 2B, engineered by Uraion Labs for sovereign, sub-millisecond local execution. Packaged natively in the universal GGUF format, this repository delivers comprehensive coverage across all standard k-quants (F16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M, Q2_K) as well as cutting-edge Unsloth Dynamic Quantization 2.0 (dynamic_q4_k_xl, dynamic_q3_k_l, dynamic_q2_k_l).

Forge 2B transforms a 2-billion parameter compact foundation (MiniCPM5-2B / Llama-based architecture) into a deterministic quantitative reasoning and multi-turn agentic tool execution engine. It runs on consumer laptops, Raspberry Pi / embedded Linux devices, edge servers, Apple Silicon (Metal), and NVIDIA workstations via llama.cpp, Ollama, LM Studio, and Jan.ai.

---

🔬 Deep Dive: Unsloth Dynamic Quantization 2.0

Small reasoning models (sub-3B parameters) are notoriously vulnerable to quantization-induced degradation. When standard uniform low-bit quantization is applied, models suffer a catastrophic drop-off in capability known as the "Perplexity Cliff".

Standard Uniform Quantization:
All Layers (Embeddings, Attention Heads, MLP Layers, Output Heads) crushed to same bit-depth
                      │
                      ▼
     ┌────────────────────────────────────────────────────────┐
     │  Severe Perplexity Explosion at 2-Bit & 3-Bit Scale    │
     │  Broken JSON formatting, hallucinated parameters,       │
     │  and reasoning degradation ("Perplexity Cliff")        │
     └────────────────────────────────────────────────────────┘

Unsloth Dynamic Quantization 2.0 (Importance Matrix Guided):
     ┌────────────────────────────────────────────────────────┐
     │ Sensitive Anchors (Embeddings & Output Head): Q8_0/FP16│
     ├────────────────────────────────────────────────────────┤
     │ Attention Routing (Q, K, V, Output Projections): Q5_K  │
     ├────────────────────────────────────────────────────────┤
     │ Redundant MLP Weights (Gate, Up, Down Projections): Q2 │
     └────────────────────────────────────────────────────────┘
                      │
                      ▼
     ┌────────────────────────────────────────────────────────┐
     │  Smooth Perplexity Curve & Sub-1GB Footprint           │
     │  Preserves 90.5% Tool Reliability & Multi-Turn State   │
     └────────────────────────────────────────────────────────┘

1. The Perplexity Cliff in Sub-3B Models

In conventional quantization (such as naive Q2_K or Q3_K), every weight matrix in the transformer stack is forced into the identical low bit-budget. However, in small models:

• Token Embeddings (token_embd.weight) and the Language Model Output Head (output.weight) maintain critical coordinate representations for the vocabulary (73,440 tokens). Truncating them to 2 or 3 bits corrupts token lookup probabilities, resulting in repetitive token loops, syntax syntax errors, and malformed JSON schemas.
• Attention Projections (attn_q, attn_k, attn_v, attn_output) route information across tokens. When quantized uniformly below 4 bits, key-value matching degrades, destroying needle retrieval over long context windows (up to 16,384 tokens).
• Feed-Forward Networks (ffn_gate, ffn_up, ffn_down), which comprise ~65% of the total parameter count, possess substantial parameter redundancy and can tolerate aggressive low-bit compression.

2. The Dynamic Quant 2.0 Mechanism

Unsloth Dynamic Quantization 2.0 employs heterogeneous layer precision driven by an empirical Importance Matrix (imatrix):

1. Calibration Sweep: A curated multi-turn agentic and quantitative dataset is processed through the unquantized F16 model. For each parameter tensor $W$, the activation covariance matrix $\mathbf{H} \approx \mathbb{E}[X X^T]$ is computed to track the gradient sensitivity of each weight channel.
2. Selective Precision Anchoring:
   • Token Embeddings & LM Head: Locked at Q8_0 or FP16. This guarantees near-zero drift in vocabulary embeddings and output logit distribution.
   • Self-Attention Projections: Preserved at Q5_K / Q6_K in XL/L dynamic configurations. This protects multi-head attention routing, GQA (Grouped-Query Attention 8:1), and context retention.
   • FFN SwiGLU Projections: Quantized down to Q4_K, Q3_K, or Q2_K, extracting maximum memory reduction where the network has sufficient dimensional redundancy.
3. Outcome: A Dynamic dynamic_q2_k_l or dynamic_q3_k_l model delivers the reasoning fidelity and JSON tool-use accuracy of a model 1.5×–2× its size, completely bypassing the perplexity cliff while keeping the disk footprint under 1 GB.

---

📊 Quantization Matrix & VRAM Sizing Table

The table below outlines all available GGUF variants for Uraion Forge 2B, including memory footprints, relative perplexity deltas, and recommended deployment scenarios.

Artifact Filename	Precision / Method	File Size	VRAM (4k Context)	VRAM (16k Context)	Quality / Perplexity Retention	Target Hardware & Deployment Tier
uraion-forge-2b-f16.gguf	F16 (16-bit Float)	~4.8 GB	~5.8 GB	~8.4 GB	100.0% (Golden Baseline)	Reference benchmarking, GPU clusters with $\ge$8 GB VRAM
uraion-forge-2b-q8_0.gguf	Q8_0 (8-bit Quant)	~2.6 GB	~3.4 GB	~5.2 GB	Near-Lossless ($< 0.005$ PPL)	Production workstations, high-fidelity agentic serving
uraion-forge-2b-q6_k.gguf	Q6_K (6-bit K-Quant)	~2.0 GB	~2.7 GB	~4.4 GB	99.7% Quality Retention	High-throughput local servers with 4 GB–6 GB VRAM
uraion-forge-2b-q5_k_m.gguf	Q5_K_M (5-bit K-Quant)	~1.7 GB	~2.3 GB	~3.9 GB	99.2% Quality Retention	Optimal for 4 GB GPUs, M-series Macs, local inference
uraion-forge-2b-dynamic_q4_k_xl.gguf	Dynamic Q4_K_XL (Unsloth)	~1.5 GB	~2.1 GB	~3.7 GB	99.5% Quality Retention	Recommended Edge Standard: Q8_0 head + protected attention
uraion-forge-2b-q4_k_m.gguf	Q4_K_M (4-bit Standard)	~1.4 GB	~1.9 GB	~3.5 GB	98.6% Quality Retention	Default Ollama deployment, broad consumer hardware
uraion-forge-2b-dynamic_q3_k_l.gguf	Dynamic Q3_K_L (Unsloth)	~1.2 GB	~1.6 GB	~3.1 GB	97.8% Quality Retention	Mobile devices, low-RAM laptops, protected tool calling
uraion-forge-2b-q3_k_m.gguf	Q3_K_M (3-bit Standard)	~1.1 GB	~1.5 GB	~3.0 GB	95.1% Quality Retention	Memory-constrained systems, micro-instances
uraion-forge-2b-dynamic_q2_k_l.gguf	Dynamic Q2_K_L (Unsloth)	~0.95 GB	~1.3 GB	~2.7 GB	94.2% Quality Retention	Sub-1GB Sovereign Edge: Preserved core reasoning
uraion-forge-2b-q2_k.gguf	Q2_K (2-bit Standard)	~0.90 GB	~1.2 GB	~2.6 GB	88.4% (Perplexity Drop)	Experimental extreme low-memory environments

Production Recommendation: For production edge agents, use dynamic_q4_k_xl.gguf or q4_k_m.gguf. If VRAM is constrained strictly under 2 GB total system memory, use dynamic_q2_k_l.gguf rather than standard q2_k.gguf to prevent tool schema degradation.

---

🚀 Deployment Guides

1. Ollama Deployment

Uraion Forge 2B GGUF includes a pre-configured Modelfile with ChatML formatting, stop tokens, context windows, and agentic system instructions.

Step 1: Clone the Repository or Download GGUF

git lfs install
git clone https://huggingface.co/uraionlabs/uraion-forge-2b-gguf
cd uraion-forge-2b-gguf

Step 2: Build the Ollama Model

# Build custom model image from Modelfile
ollama create uraion-forge -f Modelfile

Step 3: Run Interactive Chat

ollama run uraion-forge "Write a thread-safe exponential moving average filter in Python."

Step 4: Python Ollama Client Integration

import ollama

client = ollama.Client()

response = client.chat(
    model="uraion-forge",
    messages=[
        {
            "role": "system",
            "content": "You are Uraion Forge, an autonomous agent capable of utilizing external tools."
        },
        {
            "role": "user",
            "content": "Fetch the 60-minute volume_vwap metric for asset 'URAI-ALPHA'."
        }
    ],
    options={
        "temperature": 0.2,
        "num_ctx": 16384
    }
)

print(response["message"]["content"])

---

2. llama.cpp CLI & Server Deployment

llama.cpp Direct CLI Execution

# Clone and build llama.cpp (with Metal or CUDA acceleration)
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build --config Release -j

# Execute single-turn quantitative reasoning
./build/bin/llama-cli \
  -m ./uraion-forge-2b-q4_k_m.gguf \
  -p "<|im_start|>system\nYou are Uraion Forge, an advanced autonomous reasoning engine developed by Uraion Labs.<|im_end|>\n<|im_start|>user\nWrite an efficient Python function to calculate streaming order-book VWAP.<|im_end|>\n<|im_start|>assistant\n" \
  -n 1024 \
  --temp 0.2 \
  --top-p 0.9 \
  -ngl 99 \
  -co

llama-server OpenAI-Compatible HTTP Endpoint

Launch high-throughput local serving compatible with any OpenAI API SDK client:

./build/bin/llama-server \
  -m ./uraion-forge-2b-q4_k_m.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 16384 \
  -ngl 99 \
  --threads 8

Querying llama-server via Python OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed"
)

completion = client.chat.completions.create(
    model="uraion-forge",
    messages=[
        {"role": "system", "content": "You are Uraion Forge, an advanced autonomous reasoning assistant."},
        {"role": "user", "content": "Implement an asynchronous ring buffer with lock-free atomic pointers."}
    ],
    temperature=0.2,
    max_tokens=1024
)

print(completion.choices[0].message.content)

---

3. LM Studio & Jan.ai Desktop Setup

LM Studio:

1. Open LM Studio.
2. Navigate to My Models (folder icon in sidebar).
3. Drag and drop any downloaded uraion-forge-2b-*.gguf file into the ~/.cache/lm-studio/models/uraionlabs/uraion-forge-2b-gguf/ directory.
4. Load the model and configure preset parameters:
   • Preset: Custom ChatML
   • System Prompt: You are Uraion Forge, an advanced autonomous reasoning and coding assistant developed by Uraion Labs (https://uraionlabs.com).
   • Stop Strings: <|im_end|>, [END], <|im_start|>
   • Context Length: 16384
   • GPU Offload: Maximize GPU Layers (-ngl 99 on Apple Metal or NVIDIA CUDA).

Jan.ai:

1. Open Jan.ai $\to$ Settings $\to$ Local Models.
2. Click Import Model and select uraion-forge-2b-dynamic_q4_k_xl.gguf.
3. Set prompt format to ChatML and temperature to 0.2.

---

🛠️ Agentic Tool Use & Prompt Specifications

Uraion Forge 2B is trained natively on ChatML syntax with explicit protocols for structured JSON tool invocation and active parameter disambiguation.

Prompt Template (ChatML)

<|im_start|>system
You are Uraion Forge, an autonomous agent capable of utilizing external tools and APIs.
When invoking a tool, emit a JSON code block. If a required parameter is missing, ask for clarification with a '?' rather than hallucinating.<|im_end|>
<|im_start|>user
{user_query}<|im_end|>
<|im_start|>assistant

1. Structured Function Call Example

Input:

<|im_start|>user
Fetch the 60-minute volume_vwap metric for asset 'URAI-ALPHA'.<|im_end|>
<|im_start|>assistant

Model Output:

{
  "name": "query_timeseries_metric",
  "arguments": {
    "asset_id": "URAI-ALPHA",
    "metric_name": "volume_vwap",
    "window_minutes": 60
  }
}

2. Active Disambiguation (Anti-Hallucination Safeguard)

When required arguments are omitted, Forge 2B does not hallucinate arbitrary values: Input:

<|im_start|>user
Deploy the low-latency worker profile to the 'equinix-ny4' cluster.<|im_end|>
<|im_start|>assistant

Model Output:

The timeout_seconds parameter is required to initiate deployment to cluster 'equinix-ny4'. What timeout value (1-120 seconds) should be configured?

3. Reasoning Allocation (Primary vs. Secondary Mode)

• Secondary Mode (concise / direct): For direct code generation, mathematical derivations, or low-latency programmatic responses. Bypasses internal reasoning loops for maximum tokens/sec throughput.
• Primary Mode (structured thinking): For multi-turn stateful tool workflows requiring optimistic concurrency resolution (CONFLICT $\to$ re-read $\to$ re-acquire locks), retry handling, and complex state management (90.5% benchmark reliability).

---

🛡️ Rigorous Privacy & Safeguards Notice

In strict accordance with Uraion Labs open-source standards:

• Zero Proprietary Alpha: Uraion Forge 2B contains zero proprietary trading alpha, proprietary execution logic, internal pricing algorithms, or private exchange gateway code.
• Public Domain Rigor: All training signals and quantitative tasks were synthesized using public mathematical principles, open-source distributed algorithms, and standard computational schemas.
• Local Sovereignty: Running Forge 2B locally via GGUF guarantees that zero telemetry, system logs, tool calls, or user queries leave your local machine.

---

📜 Technical Specifications

Parameter	Specification	Notes
Base Architecture	MiniCPM5-2B (LlamaForCausalLM)	Grouped-Query Attention (GQA 8:1)
Active Parameters	2,048,286,720 (2.05B)	100% active during inference
Transformer Layers	42	Decoder layers
Hidden Dimension	2048	Model width
Intermediate Dimension	6144	SwiGLU projection
Attention Heads	16 Query Heads / 2 Key-Value Heads	Memory-efficient KV cache
Context Length	16,384 tokens native (131k RoPE ceiling)	Needle retrieval verified to 32k
Vocabulary Size	73,440 tokens	BPE / SentencePiece
Supported Quantizations	F16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M, Q2_K, Dynamic Quants	Full GGUF v3 specification
License	Apache 2.0	Commercial and open-source permissive

---

📖 Citation & Contact

If you integrate Uraion Forge 2B GGUF into your edge systems, applications, or research, please cite:

@misc{uraionlabs2026forge2bgguf,
  title={Uraion Forge 2B GGUF: High-Throughput Quantitative Reasoning and Agentic Tool Inference at the Edge with Dynamic Quantization},
  author={{Uraion Labs Technical Team}},
  year={2026},
  howpublished={\url{https://huggingface.co/uraionlabs/uraion-forge-2b-gguf}},
  note={Uraion Labs Open Source Release}
}

For enterprise edge deployments, technical inquiries, and ecosystem integrations, visit Uraion Labs.

Fabric workflow example

Watch a checked maintenance change in Uraion Fabric: an isolated Agent Workspace, executable checks, human Review and Apply.

This synthetic source-build demonstration uses qwen3.8:27b-nvfp4 through local Ollama, not Forge. It is not evidence of this model's performance, a customer outcome, or a signed public beta. The product page explains current limitations, local/cloud data flow and the proposed paid-pilot scope; pricing remains a hypothesis and inference is separate.