⚠️ THIS IS A PREVIEW VARIANT OF LUAUDEV! DO NOT USE THIS MODEL FOR PRODUCTION. FOR PRODUCTION WAIT FOR THE OFFICIAL RELEASE
When luaudev officially releases, its readme will be more full with information. this is after one stage of sft and pre-training. The base model will be released along side the official instruct model.
This is the preview instruct variant.
pre-training info:
- Luaudev was pre-traiend on 21 billion unique tokens, at 4096 sequence length with packing enabled, with a knowledge cutoff date of march 2026 for code and web and for math may 2025.
- the model was pre-trained with fp8 matmuls with bf16 weights using the adamwfp8 optimizer and an a effective batch size of 288. All training was done on a single Nvidia RTX 4080 (overclocked)
post-training info:
- this luaudev variant was post-trained with sft on about 1.5 billion tokens of mixed luau specific code, math, q&a and so on. This model supports 4 reasoning efforts: disabled(no reasoning), low, medium, high. Note the model may loop with high reasoning more than normal. Luaudev also supports tool calling and agentic output and inputs yet for this preview the performance is not ideal.
- the model was post trained with an effective batch size of 32, and with a custom home-made ademamix based 2bit fp8 optimizer for real memory savings.
- post training was done with a sequence length of 32k, and thus that is the supported max sequence length by the model.
PLB benchmark results:
text:
| Model | Publisher | Elo |
|---|---|---|
| GPT OSS 20B ** | OpenAI | 75.4 🏆 |
| LuauDev 177M Preview * | Pinkstack | 56.1 |
| Qwen3.5 2B | Qwen | 17.3 |
| LFM2.5 350M | Liquid AI | 11.0 |
| Gemma 3 1B | 9.5 |
.* luaudev supports reasoning efforts — set to disabled (reasoning off) for a fair comparison.
.** gpt-oss-20b supports reasoning efforts — lowest available is low, so it was set to low.
GGUF
https://huggingface.co/Pinkstack/Luaudev-177M-Preview-inst-GGUF
License:
Luaudev is released under openmdw 1.1
- Downloads last month
- -