int-llm model.mgw — an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers

This repo hosts the committed reference weights of int-llm: a tiny character-level GPT trained and sampled entirely in Q16.48 fixed-point integer arithmetic — no float, no double, no libm anywhere in the compute path — with byte-reproducible results across every tested platform and compiler.

Note: this is a custom C artifact in the project's own .mgw container — not a Transformers/safetensors checkpoint. It cannot be loaded with AutoModel or run on hosted inference; it is consumed by the C programs in the GitHub repo below. It is also not the TinyLlama-derived inference weights discussed in the int-llm write-up — this repository publishes only the small character-level reference GPT used throughout the project.

Background and motivation: the int-llm blog post.

The file

File	model.mgw — 115,576 bytes
sha256	466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb
Model	character-level GPT: N_EMBD=32, N_HEAD=4, N_LAYER=1, 14,272 parameters (9 weight tensors)
Weights	MGW v1 indexed container; 9 int64_t Q16.48 weight tensors plus tokenizer alphabet and RNG state
Training	5000 steps on the public makemore names dataset, integer-only (./gpt_int --save model.mgw)
Math contract	determinism-gate golden hash c0d933ea340452ec — reproduced by every backend and every validated target below
License	Apache-2.0

The .mgw format (MGW v1) is a small indexed container: a 64-byte header (magic, version, endianness rejection tag), a 64-byte config block, a tensor index, the nine weight matrices as raw int64_t Q16.48 values, and two bookkeeping tensors — the tokenizer alphabet and the sampler's RNG state (11 stored tensors in all: 9 learned + 2 bookkeeping). There is nothing to dequantize: the model was trained directly in Q16.48, so floating-point weights never existed for this model. It can be loaded from a file (--load model.mgw) or zero-copy from memory (mgpt_load_mem()), which is how microcontrollers run it straight out of memory-mapped flash without ever copying the weights into RAM.

Why host 115 KB of weights?

First, because the size is the point: at 115 KB the entire model — not a distilled or re-quantized derivative — bakes into the flash of most mainstream MCUs alongside the code that runs it. Even a 256 KB-flash part holds both with room to spare (measured firmware sizes below), and on XIP-capable chips the weights are read in place, occupying zero RAM.

Second, because the file itself is a claim. This exact byte sequence is independently reproduced by running the full 5000-step training on four different hosts — arm64 macOS (clang), x86-64 AMD (gcc), x86-64 Intel (gcc), and a 2014 Raspberry Pi 1 B+ (32-bit ARMv6, a CPU with no __int128) — and every validated inference run on the targets below reproduces the training host's 20 sampled names byte-for-byte, PRNG stream included. Most model files are "weights we happened to save"; this one is a fixed point (pun intended) you can re-derive from source.

That makes it useful as an oracle for anyone working on quantization, numerical drift, regression testing, or deterministic inference: any deviation from it is an implementation bug, never rounding ambiguity — within the documented arithmetic and file-format contract, there is no floating-point tolerance to hide behind.

Validated on real hardware

The train-big/run-small loop closes on microcontrollers: train on a laptop, run inference-only from this weight file on a $5 board. Every target below passed both checks (determinism-grid golden hash + byte-identical 20-sample inference); raw provenance-stamped transcripts live in validation/cpu/.

target	ISA	determinism	20 samples
XIAO RP2040 (Cortex-M0+ @ 133 MHz)	Armv6-M	10.9 s	6.7 s
Raspberry Pi Pico 2 (RP2350, ARM mode, Cortex-M33)	Armv8-M	4.1 s	3.1 s
Raspberry Pi Pico 2 (RP2350, RISC-V mode, Hazard3)	rv32imac	5.4 s	3.8 s
ESP32-C6	rv32imac	5.6 s	2.0 s
Heltec V3 (ESP32-S3, LX7 @ 240 MHz)	Xtensa	3.9 s	1.1 s
LILYGO T-Beam (ESP32, LX6 @ 240 MHz)	Xtensa	4.2 s	2.6 s
XIAO nRF52840 (Cortex-M4F @ 64 MHz)	Armv7E-M	13.9 s	3.4 s
Raspberry Pi 1 B+ (ARMv6, 32-bit Linux)	ARMv6	0.7 s	0.2 s
AMD Ryzen 7 7700 (Linux, gcc)	x86-64	native + portable	byte-identical
Intel Core i7-7700 (Linux, gcc)	x86-64	native + portable	byte-identical

Same die, two ISAs: the Pico 2 reproduces the identical output in both its ARM and RISC-V boot modes. The Linux rows additionally rerun the full training and reproduce this repo's model.mgw byte-for-byte.

Footprint (XIAO RP2040, the smallest target): the complete firmware is 187,932 B of flash — including the whole 115 KB weight file baked into rodata — and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights are read in place over XIP flash; RAM holds only KV cache, activations, and the USB stack. Code + weights comfortably fit 256 KB-flash-class parts.

Run it

git clone https://github.com/nmicic/int-llm
cd int-llm
make gpt_int
./gpt_int --load model.mgw     # 20 names, byte-identical on every validated target

The identical file is committed in the GitHub repo; to fetch just the weights from here instead:

hf download nmicic/int-llm model.mgw --local-dir .
# for strict reproducibility, pin a revision once published:
#   hf download nmicic/int-llm model.mgw --revision <commit-or-tag> --local-dir .

To retrain and verify the reproducibility claim yourself:

make input                      # fetch the names dataset
./gpt_int --save model2.mgw     # ~2 s on a desktop, ~10 min on a Pi 1
cmp model2.mgw model.mgw        # exit 0

The MCU harnesses (PlatformIO, one folder per board, flash + serial-capture scripts) are in validation/cpu/.

Intended use

This artifact is intended for: validating integer-only or fixed-point implementations against an exact reference, regression testing, deterministic inference research, and embedded/TinyML experiments. It is not intended for production language generation, hosted inference, use with the Transformers library, or capability comparisons against modern LLMs.

Limitations & provenance

• "Integer-only" refers to the model compute path: training, inference, sampling, and serialization are all Q16.48 integer arithmetic. Build tooling, logging, and timing on the host are outside that claim.
• This is a toy by design: a character-level name generator, not a general text LLM. BLOCK_SIZE is 8, so generated names are at most 8 characters. Its value is the methodology — exact integer arithmetic and cross-platform byte-reproducibility — not the language modeling.
• Training data: names.txt from Andrej Karpathy's makemore (MIT license), fetched by make input; sha256 0a30b5557f192f32ab962680889aac5f6fda0f4cecf40a6d0b5694f58ea8cc4d.
• MGW v1 is a host-native-endian format with an endianness rejection tag in the header; this published file is little-endian (as are all validated targets).
• The loader is a research-grade parser, not hardened against adversarial inputs — verify the sha256 above and load only trusted .mgw files.

Links

• Source, math library, and validation records: https://github.com/nmicic/int-llm
• Write-up: https://huggingface.co/blog/nmicic/int-llm
• The portable Q16.48 dual-backend math header matured in the sibling project astro-nav-int