Sketchpad: certified exact arithmetic with a small vision-language model
Abstract
A language model with a low-rank adapter and bounded external memory can be certified exact for arbitrary-length arithmetic by checking finitely many views, and it performs multiple exact arithmetic procedures on a laptop browser.
Language models are accurate on most long-arithmetic problems, but their accuracy is estimated on samples and carries no guarantee for an unseen input; tool use is exact but moves the computation outside the network. We show that exactness at all input lengths becomes a finite, checkable property of a network when each of its steps reads only a bounded view of an external working memory. In Sketchpad, Qwen3.5-0.8B with a low-rank adapter executes arithmetic procedures on a rendered sheet. Each forward pass reads a three-cell view and emits one step: a digit, a carry or borrow, and the next operation. A fixed runtime moves the view and writes no digit of a result. We prove in Lean that the model is then exact on all inputs of any length if and only if it is exact on the finite set of views the runtime presents, that no finite test suffices when steps also read earlier steps, and that the guarantee tolerates bounded numerical perturbation and extends to compositions of procedures. All 2,446 views pass on the deployment device, a laptop GPU running the model in a web browser. The certified model performs signed addition and subtraction, long multiplication and division, square roots and greatest common divisors at arbitrary length. It solved 200 of 200 pre-registered problems in the browser, and 220 of 220 sums of up to 1,000 digits and products of up to 20 digits, of which GPT-6.1 Sol, the strongest of ten other models, solved 217. A symbolic runtime built on these procedures solves, expands and factors polynomials with every number computed by the model, and a multiplicative gate on the adapter recovers the base model exactly when the sheet is not in use.
Community
Author here. Can a small model be proven exact at arithmetic on inputs of every length?
🧮 The theory
If each step of a model reads only a bounded view of an external memory, exactness at all lengths reduces to exactness on a finite set of views. Proved in Lean, along with:
- Necessity: once steps can also read earlier steps, no finite test suffices.
- Robustness: the guarantee survives bounded numerical error.
- Composition: certified procedures stay certified when combined.
⚙️ In practice
Qwen3.5-0.8B + a small adapter passes all 2,446 views on a laptop GPU, in the browser. It is certified for + − × ÷ √ gcd at any length.
| 220 pre-registered problems | Score |
|---|---|
| Sketchpad (0.8B) | 220 |
| GPT-6.1 Sol · Claude Opus 5.5 · Gemini 4 Argon | 217 · 216 · 207 |
| Five open-weight models | 149–199 |
| Qwen3.5-27B · Qwen3.5-0.8B (base) | 121 · 23 |
On 1,000-digit sums: Sketchpad 20/20; frontier models 19, 17, 9; every open-weight model ≤ 2.
Proving exactness in Lean for a 0.8B model reading only a three-cell view is a lovely idea. Does the guarantee still hold if the sheet is rendered a bit differently, say another font or noisy pixels, or does it depend on a clean render?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model (2026)
- Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication (2026)
- Shallow Encoding Emits Deep Memories (2026)
- Algorithmic Scratchpads and Curriculum Staging for Arithmetic Reasoning in Tiny Transformers (2026)
- What Sets the Serving Speed of Looped Language Models? Measurements and Optimizations Graded by Exactness (2026)
- PACT: Certifying What Is Written in Parallel Decoding of Diffusion Language Models (2026)
- Integer-Exact Microscaling: Running MXFP4 Language Models on INT8 Matrix Units (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.04434 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 2
mohsenvand/sketchpad-0.8b-runs
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper