Nemotron 3.5 Parse MLX 8bit

Native MLX conversion of NVIDIA Nemotron 3.5 Parse. Source revision: e7ea79242a87ea0ad3919b672bfedb429a683206. Architecture: RadioMBartPrefixForCausalLM, with a fully causal visual prefix. This is separate from the Parse 2.0 encoder-decoder architecture.

Runtime: native nemotron_parse_v3_5 port in this exact MLX-VLM commit. The current PyPI MLX-VLM release does not include the port. Install the exact commit:

pip install 'mlx-vlm @ git+https://github.com/geoHeil/mlx-vlm.git@393c69d7ae9db566f5a0385db3e257b68288a6f9'
mlx_vlm.generate --model geoHeil/Nemotron-3.5-Parse-MLX-8bit --image page.png --temperature 0 --max-tokens 8744 --repetition-penalty 1.1 --repetition-context-size 0 --prompt '</s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>'

Conversion: 8-bit affine quantization, group size 64, for language-model linear layers and tied embeddings; the vision tower remains bfloat16. Plain autoregressive decoding is supported; MTP acceleration is omitted. The processor accepts one page at a time. It preserves the raw six-token prompt. Whole-prefix prefill keeps those prompt IDs in the full repetition history.

Local fidelity

The converted weights were uploaded first, then downloaded by immutable revision and measured locally against the pinned Transformers reference. See PARITY.md for complete settings, versions, input hashes, source/weight hashes, numerical measures, full output receipts, and reproducible commands.

Matched bfloat16 reference: greedy, the raw six-token prompt, repetition penalty 1.1 over full history, Transformers on MPS, and native MLX. The bf16 and 8-bit synthetic text/table page preserve text and classes; coordinates differ. The bf16 scientific page preserves classes with 99.914% text similarity after removal of geometry. The low-resolution bill has text/class differences; exact reduced-precision parity across all pages is not claimed. The 8-bit and 4-bit results are reported separately. The 4-bit model has additional text/geometry loss.

The float32 architecture control uses the original source weights in both runtimes. Its maximum absolute teacher-forced logit error stays below 0.001 across the three pages, and teacher argmax agrees at all 38 tested positions on each page. Full output is exact on all three pages, including text, class sequences, and coordinates.

These are three-page local fidelity measurements, not a broad quality benchmark. The source file stores float32 weights. Apple Silicon and source weights are required for reproduction; this is not a CI check. Use bf16 when fidelity is the priority.

License and provenance

The source model weights and NVIDIA model materials use OpenMDW-1.1. The tokenizer uses MIT with NVIDIA modifications. This repository retains the source LICENSE, THIRD_PARTY_NOTICES.md, and TOKENIZER_ATTRIBUTION.md. No training or calibration was performed.

Downloads last month
344
Safetensors
Model size
0.6B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for geoHeil/Nemotron-3.5-Parse-MLX-8bit

Quantized
(8)
this model