AltSlate Labs

Efficient, honest AI — small models trained from scratch, with the full recipe published.

We build compact language models that punch above their weight, and we release everything needed to trust and reproduce them: code, weights, evaluations, and reports.

🪰 The Jugnu family

Tiny language models pretrained from scratch (jugnu — जुगनू — means "firefly": small, but it glows). All under 150M parameters, all trained on 4× Blackwell GPUs, all Apache-2.0.

JugnuLM-110M's 81.3% BLiMP ≈ GPT-X2-125M at ~12% fewer parameters and ~9× fewer training tokens.

The ablation ladder

Starting from an honest 110M baseline (R0), we add one lever at a time and keep only what beats the previous rung on the metric that matters:

• R1 — value residuals (ResFormer): ARC-Easy +2.2 → kept.
• R2 — Muon optimizer: ARC-Easy +1.4, lower perplexity → kept.
• R3 — diverse data blend (+DCLM +FineMath): best BLiMP & perplexity, but ARC-Easy −2.5 → dropped. An honest negative — at this scale, FineWeb-Edu's educational filtering feeds ARC-Easy more than raw diversity does.

Training code & full recipe: github.com/AltSlate-Labs/jugnu

What we care about

• Efficiency per parameter — the sub-150M regime: on-device, low-latency, cheap-to-serve models.
• Reproducibility — public code, open weights, and one-command evals.
• Honest reporting — clean baselines and measured ablations over unvalidated clever tricks. We publish what didn't work (see R3), not just what did.

Links

• 💻 Code — https://github.com/AltSlate-Labs
• 🌐 Web — https://altslate.com
• ✉️ hello@altslate.com