Instructions to use mykor/mmBERT-base-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mykor/mmBERT-base-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf mykor/mmBERT-base-GGUF:Q4_K_M --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf mykor/mmBERT-base-GGUF:Q4_K_M --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf mykor/mmBERT-base-GGUF:Q4_K_M --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use mykor/mmBERT-base-GGUF with Ollama:
ollama run hf.co/mykor/mmBERT-base-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use mykor/mmBERT-base-GGUF with Docker Model Runner:
docker model run hf.co/mykor/mmBERT-base-GGUF:Q4_K_M
- Lemonade
How to use mykor/mmBERT-base-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mykor/mmBERT-base-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.mmBERT-base-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Add files using upload-large-folder tool
Browse files- mmBERT-Base-307M-BF16.gguf +2 -2
- mmBERT-Base-307M-F16.gguf +2 -2
- mmBERT-Base-307M-F32.gguf +2 -2
- mmBERT-Base-307M-Q2_K.gguf +2 -2
- mmBERT-Base-307M-Q3_K_M.gguf +2 -2
- mmBERT-Base-307M-Q4_K_M.gguf +2 -2
- mmBERT-Base-307M-Q5_K_M.gguf +2 -2
- mmBERT-Base-307M-Q6_K.gguf +2 -2
- mmBERT-Base-307M-Q8_0.gguf +2 -2
mmBERT-Base-307M-BF16.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e57eab45f390397e115ecff2b44052ec492fd213ddede2216909fc8421b4046
|
| 3 |
+
size 619985344
|
mmBERT-Base-307M-F16.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ef50730aa5bbde0157fc72fb9e884bcc0dcf6205d4051ef77475b6edecb5e9e
|
| 3 |
+
size 619985344
|
mmBERT-Base-307M-F32.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d0e28550077e71bfcc5e3a598bcd5ad5ae2edf07e6206ce6d56b089d92d76d2e
|
| 3 |
+
size 1233795520
|
mmBERT-Base-307M-Q2_K.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d3e08d7d8eb8b5cbdb11a25708e1a7ec8d44f4725427b17ba2712be759514e7c
|
| 3 |
+
size 209526208
|
mmBERT-Base-307M-Q3_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6f38dc4df17c4e17fd30f90176bc0205cca7dcde15f1382215899561e56a99de
|
| 3 |
+
size 226815424
|
mmBERT-Base-307M-Q4_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:59c31ebd7f838f5837e855b9f727e6b71f79eebe2e11bd3cd2dfcaa16d8fec8d
|
| 3 |
+
size 240114112
|
mmBERT-Base-307M-Q5_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a8140c9ee0b4d59923dc115a04d541e6d29f7e1fbc7563ee5337ef23516718d4
|
| 3 |
+
size 252435904
|
mmBERT-Base-307M-Q6_K.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f069238fc5f61663f0a7aa7fdd8da79cf4a386f52cd6206819ee76a5d3534bd9
|
| 3 |
+
size 262647232
|
mmBERT-Base-307M-Q8_0.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ad87b6c5cb2ef011b5d5a8d89074a9c02350ca81d6f9f3ef1f5428450c76da07
|
| 3 |
+
size 332261824
|