Instructions to use chakibmed/whispbook-functiongemma-270m-speaker-attribution-mlx-lora-v2-no-hint-b8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use chakibmed/whispbook-functiongemma-270m-speaker-attribution-mlx-lora-v2-no-hint-b8 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download chakibmed/whispbook-functiongemma-270m-speaker-attribution-mlx-lora-v2-no-hint-b8 --local-dir whispbook-functiongemma-270m-speaker-attribution-mlx-lora-v2-no-hint-b8
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
FunctionGemma MLX LoRA v2 no-hint fallback b8
Experimental local MLX-LM LoRA run for Whispbook book-dialogue speaker attribution.
This run tests a not_in_candidates fallback label without adding a direct fallback hint to procedural examples. The result is useful as a documented experiment, not yet as the best production adapter: fallback behavior is learnable, but the model overuses the fallback once training progresses.
Run Config
- Base model:
mlx-community/functiongemma-270m-it-4bit - Source model family:
unsloth/functiongemma-270m-it - Dataset:
chakibmed/whispbook-functiongemma-speaker-attribution-sft - Local data source:
datasets/sft/functiongemma - Train examples:
172,189 - Eval examples:
7,951 - Batch size:
8 - Learning rate:
5e-6 - Max sequence length:
2048 - LoRA rank/scale/dropout:
16 / 16 / 0.05 - Target modules:
q,k,v,oattention projections plusgate,up,downMLP projections - Final training checkpoint:
2200 - Final test loss:
0.080 - Final test perplexity:
1.083
Checkpoint Metrics
| checkpoint | eval rows | total accuracy | known accuracy | fallback accuracy | false fallback | forced pick |
|---|---|---|---|---|---|---|
| 400 | 500 | 0.1440 | 0.0939 | 0.6905 | 4 | 13 |
| 600 | 100 | 0.4400 | 0.0345 | 1.0000 | 22 | 0 |
| 800 | 100 | 0.4400 | 0.0345 | 1.0000 | 44 | 0 |
| 1000 | 100 | 0.4200 | 0.0000 | 1.0000 | 38 | 0 |
| 1400 | 100 | 0.4400 | 0.0345 | 1.0000 | 16 | 0 |
| 1800 | 100 | 0.4200 | 0.0000 | 1.0000 | 47 | 0 |
| 2200 | 100 | 0.4200 | 0.0000 | 1.0000 | 20 | 0 |
Notes:
known accuracymeasures rows where the gold speaker is a named candidate.fallback accuracymeasures rows where the correct label isnot_in_candidates.false fallbackcounts known-speaker rows predicted asnot_in_candidates.forced pickcounts fallback rows predicted as a named candidate.- Checkpoint
400has a full 500-row local generation evaluation; later checkpoints currently use the balanced 100-row smoke slice.
Files
adapters/: MLX-LM LoRA adapter config and saved checkpoints.functiongemma_lora_config.yaml: MLX-LM training config.run_mlx_lm_lora.sh: local launch command used for the run.eval/: generation-eval summaries and predictions.report/training_report.html: Plotly dashboard for README/research documentation.report/checkpoint_metrics.json: normalized checkpoint metric rows.
Interpretation
Removing the fallback hint was the right data-quality fix. This run shows that the model can learn not_in_candidates, but with the current fallback proportion it begins using fallback as an uncertainty escape hatch. The next run should reduce the fallback training proportion and evaluate early checkpoints.
Quantized
Model tree for chakibmed/whispbook-functiongemma-270m-speaker-attribution-mlx-lora-v2-no-hint-b8
Base model
google/functiongemma-270m-it