** THIS MODEL IS BEING FINISHED AND POLISHED YET **

🧠 MiniAxion1-0.9M

MiniAxion1-0.9M is a Nano Reasoning Model (NRM) with ~920K parameters designed to explore the emergence of structured reasoning in extremely small neural networks.

Despite its minimal size, the model demonstrates strong consistency in reasoning format and step-based thinking using explicit <THINK> and <STEP> tokens.

🚀 Overview

Model Type: Nano Reasoning Model (NRM)
Parameters: ~920,833
Architecture: Transformer (6 layers: 2 entry + 2 shared + 2 exit)
d_model: 256
Heads: 8
FFN size: 512
LoRA Rank: 16
Vocabulary Size: 2048
Training Time: ~80 minutes (CPU)

🧠 Key Capabilities

✅ Structured Reasoning

The model reliably produces structured reasoning traces:

<THINK>
<STEP> ...
<STEP> ...
</THINK>
<ANS>...</ANS>

100% usage of reasoning tokens
Consistent multi-step formatting
Stable output structure across tasks

⚡ Ultra-Lightweight

Runs efficiently on CPU
Designed for experimentation and rapid iteration
Suitable for embedded or game-like environments

🧪 Research-Oriented Design

MiniAxion1 is not intended to compete with large-scale models. Instead, it is built to:

Study reasoning emergence in small models
Explore structure vs correctness trade-offs
Enable fast iteration cycles for AI research

📊 Evaluation Results

Task	Accuracy
Arithmetic	3.3%
Two-Step Arithmetic	10.0%
Even/Odd	100.0%
Comparison	5.0%
Pattern Completion	0.0%
Word Problems	0.0%
Sorting	0.0%
Chain-of-Thought Format	100.0%

Average Accuracy: 16.9%

🔍 Observations

The model learns reasoning structure before reasoning correctness
Chain-of-thought formatting is highly reliable
Arithmetic and symbolic reasoning remain limited at this scale
Evidence of partial decoupling between reasoning steps and final answers

⚠️ Limitations

Weak performance on arithmetic and multi-step reasoning tasks
Susceptible to incorrect intermediate reasoning steps
Limited generalization beyond trained patterns
Not suitable for production use in critical systems
Due to 920k parameters, low results on evaluation is expected

🎯 Intended Use Cases

🧪 AI research and experimentation
🎮 Game AI / NPC reasoning simulation
📚 Educational demonstrations of reasoning structure
⚙️ Lightweight reasoning prototypes

Quick start


import torch
from model import NRMModel
from tokenizer import Tokenizer

# load
model = NRMModel.from_config("config.json")
model.load_state_dict(torch.load("model.pt"))
model.eval()

tokenizer = Tokenizer.load("tokenizer.json")

def generate(prompt):
    tokens = tokenizer.encode(prompt)
    output = model.generate(tokens)
    return tokenizer.decode(output)

print(generate("<INST>What is 2 + 2?</INST>"))

🧠 Philosophy

MiniAxion1 explores a key question:

Can structured reasoning emerge in extremely small models?

This model provides early evidence that:

Reasoning format can be learned efficiently
Structure and correctness are separable capabilities
Useful behavior can emerge even at sub-1M scale

🔮 Future Directions

Improved dataset alignment for arithmetic reasoning
Scaling parameters (1M → 10M range)
Better coupling between reasoning and answers
Task-specific specialization (e.g., math-only variants)
distillation knowledge on bigger models

🤝 Acknowledgments

This model was developed as part of ongoing experimentation in nano-scale reasoning systems. the main question was: "How low could a model think(or mimic it)?

📎 Model

👉 https://huggingface.co/AxionLab-Co/MiniAxion1-0.9M

🧪 Disclaimer

This is an experimental research model. Outputs may be incorrect even when reasoning appears structured or convincing.

Downloads last month: 23