stas122/stentor_python_30m

Model Description

Stentor Python 30M is a compact language model specifically fine-tuned for Python code generation and autocompletion tasks. Based on the Stentor-30M architecture, this model contains 30 million parameters and is designed to run efficiently on resource-constrained devices including mobile phones and embedded systems.

Model Details

• Developed by: Experimental fine-tuning project
• Model type: Causal language model (LlamaForCausalLM)
• Language: Python code, English instructions
• Parameters: 30,419,712
• Context length: 512 tokens
• Model size: 60 MB (FP16), 30 MB (INT8)
• License: Apache 2.0

Training Data

The model was fine-tuned on a curated dataset of 872 Python examples, including:

• Basic algorithms (factorial, prime numbers, list operations)
• Class implementations (Stack, BankAccount, Rectangle, Circle)
• Recursive functions (quicksort, Fibonacci)
• String manipulation (palindrome, anagram, vowel counting)
• MBPP (Mostly Basic Python Problems) dataset tasks

All examples follow a consistent format with "### Task:" instruction and "### Solution:" code block.

Training Process

The fine-tuning process involved multiple stages:

1. Base model: Stentor-30M
2. Initial fine-tuning on 50k examples
3. Multiple correction rounds with progressively lower learning rates
4. Final detoxification training with learning rate 3e-7 to remove undesirable patterns

Evaluation Results

The model was evaluated on several test categories:

Category	Pass Rate	Notes
Basic functions	80%	Factorial, prime check, etc.
Classes from training set	100%	Stack, BankAccount, Rectangle
New complex classes	33%	Graph, Queue, inheritance
Function signatures (MBPP)	100%	Correctly generates def statements

Capabilities

• Generates Python functions from natural language descriptions
• Implements basic algorithms (factorial, prime check, palindrome)
• Creates class definitions with methods (Stack, BankAccount, Rectangle)
• Handles recursive functions (quicksort, Fibonacci)
• Produces syntactically correct function signatures

Limitations

• May produce repeated or redundant code after the main solution
• Struggles with complex data structures (graphs, trees, queues)
• Does not reliably handle class inheritance patterns
• Can generate incorrect list indexing operations
• May continue generating text beyond the intended solution
• Limited to 512 token context window
• Not suitable for production use without output post-processing

Recommended Use Cases

• Code autocompletion in lightweight IDEs
• Educational tool for Python beginners
• Rapid prototyping of simple functions
• Embedded systems with limited computational resources
• Offline code assistance on mobile devices

Not Recommended For

• Complex algorithm implementation
• Production code generation without human review
• Tasks requiring deep contextual understanding
• Generating large codebases
• Security-critical applications

Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM

model_path = "stas122/stentor_python_30m"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)

prompt = "### Task: Write a function that checks if a number is even\n\n### Solution:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.2)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Ethical Considerations

This model is intended for educational and development assistance purposes. Users should verify all generated code before deployment, particularly for security-sensitive applications. The model may occasionally produce incorrect or inefficient code and should not be relied upon as the sole source of truth for programming tasks.

Contact

For questions or feedback about this model, please open an issue on the Hugging Face repository.