Instructions to use geoHeil/Nemotron-3.5-Parse-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use geoHeil/Nemotron-3.5-Parse-MLX-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("geoHeil/Nemotron-3.5-Parse-MLX-8bit") config = load_config("geoHeil/Nemotron-3.5-Parse-MLX-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Nemotron 3.5 Parse MLX 8bit
Native MLX conversion of NVIDIA Nemotron 3.5 Parse. Source revision: e7ea79242a87ea0ad3919b672bfedb429a683206. Architecture: RadioMBartPrefixForCausalLM, with a fully causal visual prefix. This is separate from the Parse 2.0 encoder-decoder architecture.
Runtime: native nemotron_parse_v3_5 port in this exact MLX-VLM commit. The current PyPI MLX-VLM release does not include the port. Install the exact commit:
pip install 'mlx-vlm @ git+https://github.com/geoHeil/mlx-vlm.git@393c69d7ae9db566f5a0385db3e257b68288a6f9'
mlx_vlm.generate --model geoHeil/Nemotron-3.5-Parse-MLX-8bit --image page.png --temperature 0 --max-tokens 8744 --repetition-penalty 1.1 --repetition-context-size 0 --prompt '</s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>'
Conversion: 8-bit affine quantization, group size 64, for language-model linear layers and tied embeddings; the vision tower remains bfloat16. Plain autoregressive decoding is supported; MTP acceleration is omitted. The processor accepts one page at a time. It preserves the raw six-token prompt. Whole-prefix prefill keeps those prompt IDs in the full repetition history.
Local fidelity
The converted weights were uploaded first, then downloaded by immutable revision and measured locally against the pinned Transformers reference. See PARITY.md for complete settings, versions, input hashes, source/weight hashes, numerical measures, full output receipts, and reproducible commands.
Matched bfloat16 reference: greedy, the raw six-token prompt, repetition penalty 1.1 over full history, Transformers on MPS, and native MLX. The bf16 and 8-bit synthetic text/table page preserve text and classes; coordinates differ. The bf16 scientific page preserves classes with 99.914% text similarity after removal of geometry. The low-resolution bill has text/class differences; exact reduced-precision parity across all pages is not claimed. The 8-bit and 4-bit results are reported separately. The 4-bit model has additional text/geometry loss.
The float32 architecture control uses the original source weights in both runtimes. Its maximum absolute teacher-forced logit error stays below 0.001 across the three pages, and teacher argmax agrees at all 38 tested positions on each page. Full output is exact on all three pages, including text, class sequences, and coordinates.
These are three-page local fidelity measurements, not a broad quality benchmark. The source file stores float32 weights. Apple Silicon and source weights are required for reproduction; this is not a CI check. Use bf16 when fidelity is the priority.
License and provenance
The source model weights and NVIDIA model materials use OpenMDW-1.1. The tokenizer uses MIT with NVIDIA modifications. This repository retains the source LICENSE, THIRD_PARTY_NOTICES.md, and TOKENIZER_ATTRIBUTION.md. No training or calibration was performed.
- Downloads last month
- 344
8-bit
Model tree for geoHeil/Nemotron-3.5-Parse-MLX-8bit
Base model
nvidia/Nemotron-3.5-Parse