TurinTech

TurinTech models, built with Artemis

Google's Gemma 4 12B for chat, tool calls and images on one GPU, compressed with Artemis from the QAT checkpoint

Artemis Other TurinTech models on Hugging Face Run it with Ollama Run it in LM Studio TurinTech on Discord TurinTech on LinkedIn
Download GPU memory Tool calls right, BFCL live
6.72 GB 7.5 GiB (original 23.6 GiB) 210 of 258 (original 210)

Run it

llama-server -hf turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0 --port 8080 -c 8192 --jinja

Ollama: ollama run turintech/gemma-4-12b-it-artemis  Β·  LM Studio: turintech/gemma-4-12b-it-artemis

Needs llama.cpp with Gemma 4 support (the Results table was measured with b11617; the projector and the MTP drafter were converted, checked and measured with b11889). -hf also downloads the vision projector and loads it; --no-mmproj skips that 175 MB download for text only. The chat template turns thinking on by default: the server returns it in message.reasoning_content and the answer in message.content, so allow enough output tokens (about 1,500), or send "chat_template_kwargs": {"enable_thinking": false} to turn it off, which is how every result below was measured.

Request example, images, the MTP drafter, Ollama and a local download
curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "In two sentences, why is the sky blue?"}],
  "max_tokens": 1500
}'

-c 8192 sets an 8,192-token context and --jinja uses the chat template stored in the file. Leave the server running and send the request from a second terminal. Tool calls use the same endpoint with a tools list and come back in choices[0].message.tool_calls. To run a downloaded file (pip install -U huggingface_hub for the hf command):

hf download turintech/gemma-4-12B-it-Artemis-GGUF gemma-4-12B-it-Artemis-Q4_0.gguf --local-dir .
llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf --port 8080 -c 8192 --jinja

Images: the vision projector is a separate file, converted from the original weights; mmproj-F16.gguf is the same projector in F16. Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 13 right (12 with the F16 one), against 13 for Unsloth's UD-Q4_K_XL with Unsloth's projector (results page, Image questions). The projector also holds the audio input layers, which were not tested.

hf download turintech/gemma-4-12B-it-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Faster generation: mtp-gemma-4-12B-it-Artemis.gguf is Google's drafter for this model (gemma-4-12B-it-qat-q4_0-unquantized-assistant, revision 1893406), converted to GGUF and quantized to Q4_0. It drafts the next tokens for the main file to check and does not run on its own. With --spec-type draft-mtp, -hf downloads it as well; the same file is in the MTP/ folder as MTP/mtp-gemma-4-12B-it-Artemis-Q4_0.gguf.

llama-server -hf turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0 --spec-type draft-mtp --port 8080 -c 8192 --jinja
hf download turintech/gemma-4-12B-it-Artemis-GGUF mtp-gemma-4-12B-it-Artemis.gguf --local-dir .
llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf -md mtp-gemma-4-12B-it-Artemis.gguf --spec-type draft-mtp --port 8080 -c 8192 --jinja

On an RTX 5090 it raised generation from 160.1 to 334.6 tokens per second on 30 fixed prompts (results page, MTP). Greedy replies are not always word for word the same with it on: all 10 tool-call replies matched, and 1 of the 20 chat and code replies did. --mmproj and -md can be combined.

Ollama: the package turintech/gemma-4-12b-it-artemis (tag q4_0, also 6.7gb and latest) holds the main file only. For images, Ollama 0.40.11 or newer can pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0

With Ollama 0.40.11 on 26 October 2026, the same kind of pull from Unsloth's repository of this checkpoint (hf.co/unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL) brought its projector along and ollama show listed vision; the pull from this repository was not run. A local package of our Q4_0 file and mmproj-F16.gguf (two FROM lines, PARAMETER num_ctx 8192) listed vision, answered 13 of the 20 chart questions and returned get_weather with Paris and celsius for a tool request.

Results

Tool calls right on 258 BFCL live_simple tasks for our file, the original and three public files of the same model, with download sizes.

File Size Tool calls (of 258) GSM8K (of 250) MMLU-Pro (of 280) IFEval (of 539)
TurinTech Q4_0 6.72 GB 210 243 114 475
Unsloth UD-Q4_K_XL 6.72 GB 210 244 115 485
Google Q4_0 6.98 GB 211 242 112 477
bartowski Q4_K_M 7.66 GB 209 241 125 477
BF16 original 23.83 GB 210 242 115 478

The record of how the file was chosen does not mention these item sets (How it was tested); bold marks the best in each column. After correcting for 16 planned comparisons, no test separated our file from the original, Unsloth's UD-Q4_K_XL, Google's Q4_0 or bartowski's Q4_K_M; the closest case is MMLU-Pro against bartowski's file (114 against 125, p 0.019 before the correction). bartowski's file is made from google/gemma-4-12B-it rather than the QAT checkpoint, so its tasks compare fairly and its closeness to this original does not. MMLU-Pro allowed 8 output tokens for the answer letter; a reply that began working the problem instead was cut off and counts as wrong, 102 times for ours and 102 for the original. mradermacher's i1-IQ3_XXS (4.85 GB) produced one repeated token under this llama.cpp build and is on the results page only. Full results, memory and method.

Files

File Size Peak GPU memory Use it for
gemma-4-12B-it-Artemis-Q4_0.gguf 6.72 GB 7.5 GiB Default, Ollama tag q4_0 (latest)
mmproj-BF16.gguf 175 MB Image input, with --mmproj
mmproj-F16.gguf 175 MB Image input, with --mmproj
MTP/mtp-gemma-4-12B-it-Artemis-Q4_0.gguf 323 MB Faster generation (MTP companion)
mtp-gemma-4-12B-it-Artemis.gguf 323 MB Faster generation (MTP companion)

The vision projector and the MTP drafter are optional. The drafter is there twice, in MTP/ and at the top level, where -hf finds it. Peak GPU memory measured on NVIDIA RTX 5090, llama.cpp b11617, all layers on the GPU, 8,192-token context, one slot, prompt cache off (--cache-ram 0), first 25 items of each task, without the projector or the MTP drafter; host RAM includes the parts of the memory-mapped model file that were read.

Pick this model

  • Good for chat, tool calls and maths with thinking off on one GPU, with image input through the projector, in a 6.72 GB download.
  • Pick another file when closeness to the original on Wikipedia text matters most: Unsloth's UD-Q4_K_XL (6.72 GB) is slightly closer there (KL .1316 against our .1616), while ours is closer on chat text (.1618 against .1863). On IFEval Unsloth's file followed 485 of 539 prompts against our 475, a gap the tests did not separate.

How it was made

Built with Artemis, TurinTech's optimisation platform, from Google's quantisation-aware (QAT) checkpoint google/gemma-4-12B-it-qat-q4_0-unquantized, which Google trained to hold up well at low precision. The file carries the standard Q4_0 label and averages 4.51 bits per weight over the whole file, 3.7% smaller than Google's own Q4_0 GGUF of the same checkpoint and the size of Unsloth's UD-Q4_K_XL, so stock llama.cpp, Ollama and LM Studio run it. The vision and audio input layers are converted to the separate projector files, and Google's drafter model to the MTP file; neither is changed otherwise.

How it was tested

Every file in the Results table ran on one NVIDIA RTX 5090 with llama.cpp b11617, the same chat template, greedy decoding and thinking off. The first card records that this file was chosen on closeness to the original on two texts (WikiText-2 and OpenAssistant chat), with a final check on BFCL simple tool calls (items 200 to 399) and GSM8K (test items 250 to 499) that could have kept the previous candidate. The Results table uses other item sets: BFCL live_simple, GSM8K test items 1069 to 1318, a new MMLU-Pro sample and IFEval. That record does not mention them, so it neither shows nor rules out a part for them in the choice. IFEval is scored on 539 of its 541 prompts because two ask for a count of a punctuation mark that the upstream checker scores at random. The results page has the method, the full tables with mradermacher's file, the A100 run the card was first published with, and the image, MTP and memory checks of the companion files.

Closeness to the original

The closeness numbers from the tables on the results page, in one fixed layout with the protocol behind each table.

Protocols and standard tables
  • Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
  • 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
  • Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
  • PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-27

One NVIDIA RTX 5090, 27 October 2026.

  • Reference: BF16 GGUF converted with llama.cpp b11617 from the text weights of google/gemma-4-12B-it-qat-q4_0-unquantized revision b6ed86275a6a5735884e208bfed95b445a684ca2 (23,832,066,176 bytes, SHA-256 308b0d914e682705a28ff97a6f6c293610b63634fbaad36047f2db1407820d98); its saved log-probabilities for each text are the reference for every file
  • Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
  • Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
  • Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines
  • llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
  • Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 16 -ngl 99; the reference logs were written with the same command without --kl-divergence
  • Hardware: one NVIDIA RTX 5090 (AMD EPYC 7V13 host)
  • Settings: 16 threads, all layers on the GPU, beginning-of-sequence token on for every file as in the original; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings in the results page are for the task runs)
  • KL unit: nats (natural log)
  • Measured by: TurinTech downloaded the Unsloth, Google, bartowski and mradermacher files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine. bartowski's file is made from google/gemma-4-12B-it, not from the QAT checkpoint, so its values are against a reference it was not made from; mradermacher's file produced one repeated token under this build and its values stand as measured
  • Selection: the two texts were used while developing this file, so these columns are not independent tests
  • Source, on the results page: the table columns "KL wiki / chat", "Same next token, wiki / chat"
  • Comparison files: mradermacher: mradermacher/gemma-4-12B-it-qat-q4_0-unquantized-i1-GGUF at 2b88f4b, gemma-4-12B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 cfe0e8286e4b734335fe7aed099157e1217356fd19dc9e90f4da1ce66547e183; Unsloth: unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8, gemma-4-12B-it-qat-UD-Q4_K_XL.gguf, SHA-256 90fd44e29e0d7cffeb0fd00dc73cfdab9ed0b0e95306ecf7821ea634c940c370; Google: google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436, gemma-4-12b-it-qat-q4_0.gguf, SHA-256 93567e57a8fe10b23569b9d9ec38cd005deedf71e29477c421a4b83f418a538b; bartowski: bartowski/gemma-4-12B-it-GGUF at 2ae7d41, gemma-4-12B-it-Q4_K_M.gguf, SHA-256 3962624dcd25b947d889dc9ae1bf275b61db6cd4dbe694057f34fffef1671509

Protocol rtx5090-2026-10-27, text wiki (WikiText-2 test):

File Repository Measured by Size (GB) Size (bytes) Mean KLD (nats) Same top p (top-1 agreement, %)
gemma-4-12B-it-qat-bf16.gguf google/gemma-4-12B-it-qat-q4_0-unquantized at b6ed86275a6a reference 23.83 23832066176 reference reference
gemma-4-12B-it-Artemis-Q4_0.gguf turintech/gemma-4-12B-it-Artemis-GGUF at 036a2ce7f884 TurinTech 6.72 6716356736 0.1616 87.2
gemma-4-12B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf mradermacher/gemma-4-12B-it-qat-q4_0-unquantized-i1-GGUF at 2b88f4b TurinTech 4.85 4849194816 10.57 6.1
gemma-4-12B-it-qat-UD-Q4_K_XL.gguf unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8 TurinTech 6.72 6716356800 0.1316 88.2
gemma-4-12b-it-qat-q4_0.gguf google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436 TurinTech 6.98 6975879296 0.4921 73.5
gemma-4-12B-it-Q4_K_M.gguf bartowski/gemma-4-12B-it-GGUF at 2ae7d41 TurinTech 7.66 7662533088 3.30 39.9

Protocol rtx5090-2026-10-27, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

File Repository Measured by Size (GB) Size (bytes) Mean KLD (nats) Same top p (top-1 agreement, %)
gemma-4-12B-it-qat-bf16.gguf google/gemma-4-12B-it-qat-q4_0-unquantized at b6ed86275a6a reference 23.83 23832066176 reference reference
gemma-4-12B-it-Artemis-Q4_0.gguf turintech/gemma-4-12B-it-Artemis-GGUF at 036a2ce7f884 TurinTech 6.72 6716356736 0.1618 90.6
gemma-4-12B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf mradermacher/gemma-4-12B-it-qat-q4_0-unquantized-i1-GGUF at 2b88f4b TurinTech 4.85 4849194816 10.25 6.2
gemma-4-12B-it-qat-UD-Q4_K_XL.gguf unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8 TurinTech 6.72 6716356800 0.1863 89.6
gemma-4-12b-it-qat-q4_0.gguf google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436 TurinTech 6.98 6975879296 0.5607 78.0
gemma-4-12B-it-Q4_K_M.gguf bartowski/gemma-4-12B-it-GGUF at 2ae7d41 TurinTech 7.66 7662533088 2.71 53.2

Protocol a100-2026-10-24

One NVIDIA A100 80 GB, 24 October 2026 (the run the card was first published with).

  • Reference: BF16 GGUF conversion of google/gemma-4-12B-it-qat-q4_0-unquantized revision b6ed86275a6a5735884e208bfed95b445a684ca2 (text weights, 667 tensors, 23,832,066,176 bytes)
  • Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
  • Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
  • Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines
  • llama.cpp build: b11617 (commit 14310190c)
  • Command: llama-perplexity -m <file.gguf> --override-kv tokenizer.ggml.add_bos_token=bool:true --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -ngl 99 -t 16; the reference logs were written with -f <text> and the same flags without --kl-divergence
  • Hardware: one NVIDIA A100 80 GB, all layers on the GPU
  • Settings: beginning-of-sequence token on for every file, as in the original; flash attention and KV cache type not set on the command line
  • KL unit: nats (natural log)
  • Measured by: TurinTech downloaded the Google and Unsloth files at the revisions below and measured every file in this table with the same frozen scoring script, reference logs, binary and GPU
  • Selection: these texts were used while developing this file, so they are not an independent test
  • Source, on the results page: the table columns "KL wiki / chat", "Same next token, wiki / chat"
  • Comparison files: Unsloth: unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8, gemma-4-12B-it-qat-UD-Q4_K_XL.gguf, SHA-256 90fd44e29e0d7cffeb0fd00dc73cfdab9ed0b0e95306ecf7821ea634c940c370; Google: google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436, gemma-4-12b-it-qat-q4_0.gguf, SHA-256 93567e57a8fe10b23569b9d9ec38cd005deedf71e29477c421a4b83f418a538b

Protocol a100-2026-10-24, text wiki (WikiText-2 test):

File Repository Measured by Size (GB) Size (bytes) Mean KLD (nats) Same top p (top-1 agreement, %)
BF16 original google/gemma-4-12B-it-qat-q4_0-unquantized at b6ed86275a6a reference 23.83 23832066176 reference reference
gemma-4-12B-it-Artemis-Q4_0.gguf turintech/gemma-4-12B-it-Artemis-GGUF at 7c6b51e TurinTech 6.72 6716356736 0.156 86.8
gemma-4-12B-it-qat-UD-Q4_K_XL.gguf unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8 TurinTech 6.72 6716356800 0.147 87.2
gemma-4-12b-it-qat-q4_0.gguf google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436 TurinTech 6.98 6975879296 0.485 73.9

Protocol a100-2026-10-24, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

File Repository Measured by Size (GB) Size (bytes) Mean KLD (nats) Same top p (top-1 agreement, %)
BF16 original google/gemma-4-12B-it-qat-q4_0-unquantized at b6ed86275a6a reference 23.83 23832066176 reference reference
gemma-4-12B-it-Artemis-Q4_0.gguf turintech/gemma-4-12B-it-Artemis-GGUF at 7c6b51e TurinTech 6.72 6716356736 0.153 90.3
gemma-4-12B-it-qat-UD-Q4_K_XL.gguf unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8 TurinTech 6.72 6716356800 0.169 89.8
gemma-4-12b-it-qat-q4_0.gguf google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436 TurinTech 6.98 6975879296 0.567 77.8
File checksums
File Bytes sha256
gemma-4-12B-it-Artemis-Q4_0.gguf 6,716,356,736 25b9be6a385bcc2816a14dc4913ced4352f4720786b28bef1f5936f52255ae63
mmproj-BF16.gguf 175,115,616 57bcb483665d53c0ac402adafd4f7b0c16fe1ad25c28d2dbb6739f526d78ed3e
mmproj-F16.gguf 175,115,616 61fda9811b5ac5d04eeb3f5c0b341264425798b754c151d0b303cbcc3cb40863
MTP/mtp-gemma-4-12B-it-Artemis-Q4_0.gguf 322,914,912 10de2f2cb74c08cc10a2e11b151c2da334882851e5010aeaf47149d1ee839d97
mtp-gemma-4-12B-it-Artemis.gguf 322,914,912 10de2f2cb74c08cc10a2e11b151c2da334882851e5010aeaf47149d1ee839d97

Artemis is TurinTech's optimisation platform: teams use it to make their own models smaller and faster, or to optimise their code.

All our models Β· LinkedIn Β· Questions and requests

Apache-2.0, as released by Google (licence); see LICENSE and NOTICE. An independent TurinTech release, not affiliated with or endorsed by Google. When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.

Downloads last month
3,147
GGUF
Model size
12B params
Architecture
gemma4
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for turintech/gemma-4-12B-it-Artemis-GGUF

Quantized
(92)
this model

Collection including turintech/gemma-4-12B-it-Artemis-GGUF

Evaluation results

  • Tool calls (of 258) on BFCL live
    TurinTech
    210.000
  • Correct (of 250) on GSM8K, last 250
    TurinTech
    243.000
  • Correct (of 280) on MMLU-Pro sample
    TurinTech
    114.000
  • Followed (of 539) on IFEval
    TurinTech
    475.000