Swift ยท Qwen3.8 ยท 27b

EXL3  ยท  4.0 bpw  ยท  16.4 GB  ยท  Dense


format bpw size codebook arch

base model quantized by collection


An ExLlamaV3 build of ukisai/Swift-Qwen3.8-27b at 4.0 bits per weight. See Quants for sibling repos at other bitโ€‘widths or browse the collection.

Quants

BPW     Size     median KL
wiki2  
  Status
4.0 16.4 GB 0.0080 this repo

Inference

Loader Use it for
TabbyAPI OpenAIโ€‘compatible HTTP server. Dropโ€‘in for OpenAI clients.
textโ€‘generationโ€‘webui Local chat UI. Pick the ExLlamaV3 loader from the model dropdown.
ExLlamaV3 Direct Python API for embedding the model in your own code or pipeline.

Download

pip install -U huggingface_hub

hf download \
  Honkware/Swift-Qwen3.8-27b-exl3-4.0bpw \
  --local-dir ./Swift-Qwen3.8-27b-exl3-4.0bpw
Quantization recipe  (advanced, embedded in quantization_config.json)
Setting Value
Format EXL3
Bits per weight 4.0
Head bits 5
Calibration rows 250
Calibration data exllamav3 bundled mix (c4, code, multilingual, technical, tiny, wiki)
Codebook mul1
Outโ€‘scales always
Parallel mode enabled

License & use

Use and license follow the base model. Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.


Quantized with BlockQuant  ยท  convention {org}/{model}-exl3-{bpw}bpw
Downloads last month
82
Safetensors
Model size
8B params
Tensor type
BF16
ยท
F16
ยท
I16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Honkware/Swift-Qwen3.8-27b-exl3-4.0bpw

Base model

Qwen/Qwen3.8-27B
Quantized
(67)
this model

Collection including Honkware/Swift-Qwen3.8-27b-exl3-4.0bpw