Ornith-1.5-122B-MTP-GGUF

Ornith-1.5-122B is a 122-billion-parameter Mixture-of-Experts (MoE) large language model developed by Ornith AI, released as an open-weight model under the MIT license and designed to deliver strong performance while remaining efficient enough to run on a single 80GB GPU (such as a DGX node), with an additional quantized "Mobile" variant for edge deployment. It functions as a reasoning model, generating an internal chain-of-thought within tags before producing its final answer, and supports agentic workflows through native tool/function calling compatible with OpenAI-style APIs. The model supports a native context window of up to 262,144 tokens, extendable to roughly 1M tokens via YaRN-based RoPE scaling, and it achieves state-of-the-art results among open-source models of comparable size across coding (e.g., SWE-bench Verified/Pro/Multilingual, TerminalBench, CyberGym, NL2Repo), reasoning (HLE, SciCode), and agentic benchmarks (WideSearch, BrowseComp, MCP-Atlas, Toolathlon, ClawEval), often matching or surpassing larger models like DeepSeek-V4-Flash and Beam. It integrates with modern serving frameworks (vLLM, SGLang, Transformers), GGUF-based runtimes (llama.cpp, Ollama), and popular agent/coding tools (OpenCode, Hermes Agent, OpenClaw, Unsloth), making it well-suited for coding assistance, complex task decomposition, and real-world agentic applications.

MTP — Speculative Decoding: Yes

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

File Name	Quant Type	File Size	File Link	Description
Ornith-1.5-122B.BF16-00001-of-00006.gguf	BF16 (Part 1)	48.6 GB	Link	Full BF16 weights (Part 1 of 6). Highest quality.
Ornith-1.5-122B.BF16-00002-of-00006.gguf	BF16 (Part 2)	48.4 GB	Link	Full BF16 weights (Part 2 of 6). Highest quality.
Ornith-1.5-122B.BF16-00003-of-00006.gguf	BF16 (Part 3)	48.6 GB	Link	Full BF16 weights (Part 3 of 6). Highest quality.
Ornith-1.5-122B.BF16-00004-of-00006.gguf	BF16 (Part 4)	48.6 GB	Link	Full BF16 weights (Part 4 of 6). Highest quality.
Ornith-1.5-122B.BF16-00005-of-00006.gguf	BF16 (Part 5)	48.6 GB	Link	Full BF16 weights (Part 5 of 6). Highest quality.
Ornith-1.5-122B.BF16-00006-of-00006.gguf	BF16 (Part 6)	6.4 GB	Link	Full BF16 weights (Part 6 of 6). Highest quality.
Ornith-1.5-122B.Q3_K_L.gguf	Q3_K_L	64.7 GB	Link	Lower quality but usable, good for low RAM availability.
Ornith-1.5-122B.Q4_K_M.gguf	Q4_K_M	75.8 GB	Link	Good quality, default size for most use cases, recommended.
Ornith-1.5-122B.Q5_K_M.gguf	Q5_K_M	88.7 GB	Link	High quality, recommended.
Ornith-1.5-122B.mmproj-bf16.gguf	mmproj-bf16	912 MB	Link	Multimodal projection file in BF16 format. Used for vision/language models.

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Running Models with llama.cpp (llama-cli)

This guide outlines how to compile the llama-cli binary and run inference using Multi-Token Prediction (MTP) speculative decoding and KV cache quantization.

1. Build llama-cli

Navigate to your llama.cpp repository and compile the llama-cli target in Release mode:

cd /path/to/llama.cpp
cmake -B build
cmake --build build --config Release -j$(nproc) --target llama-cli

2. Run Inference

Execute the llama-cli binary with your desired GGUF model and runtime flags:

./build/bin/llama-cli \
  -m /path/to/models/your-model.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  -ngl 99 \
  -c 8192 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 512 \
  -st \
  -p "Your prompt here"

---

Quantized using llama.cpp by prithivMLmods