Improve model card with detailed usage, training, and limits

README.md

@@ -16,60 +16,159 @@ tags:
 - k-12
 - code-generation
 - animation
+- synthetic-data
 datasets:
 - custom
+model-index:
+- name: qwen3-p5js-physics-lora
+  results:
+  - task:
+      type: text-generation
+      name: Causal Language Modeling
+    dataset:
+      type: custom
+      name: Internal synthetic p5.js science dataset
+    metrics:
+    - type: loss
+      name: Final logged train loss (step 90)
+      value: 0.4950
+    - type: loss
+      name: Train run average loss
+      value: 0.5917
+    - type: loss
+      name: Eval loss (step 50)
+      value: 0.6164
 ---
 
-# Qwen3-0.6B LoRA for p5.js Physics Animations
+# qwen3-p5js-physics-lora
 
-A LoRA adapter for [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) fine-tuned to generate interactive [p5.js](https://p5js.org/) animations that teach K-12 students physics and science concepts.
+LoRA adapter for [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B), tuned for generating educational p5.js animations from natural-language prompts.
 
-Given a natural language prompt like *"Show me how gravity affects falling objects"*, the model outputs complete, runnable p5.js code with animations, labels, and interactivity.
+Primary objective: produce runnable, classroom-friendly JavaScript sketches (`setup()` + `draw()`) that explain K-12 science concepts visually.
 
-## Key Results
+## Model Details
 
-| Metric | Value |
-|---|---|
-| Training examples | 1,036 |
-| Unique topics | 124 |
-| Training time | 2.9 min (4x A100-80GB) |
-| Train loss | 0.909 → 0.495 |
-| Eval loss | 0.616 |
-| Token accuracy | 85.6% |
-| Trainable params | 40.4M / 792M (5.1%) |
+- Base model: `Qwen/Qwen3-0.6B`
+- Fine-tuning method: LoRA + supervised fine-tuning (SFT)
+- Domain: p5.js animation code generation for science education
+- Intended language: English prompts and code comments
+- Adapter repo: `mr-dee/qwen3-p5js-physics-lora`
+- Source project: https://github.com/dylanler/qwen3-p5js-physics
 
-## How to Use
+## Intended Use
 
-### With PEFT + Transformers
+Direct use:
+- Generate p5.js teaching demos for topics like gravity, circuits, optics, astronomy, and earth science.
+- Bootstrap lesson visuals that teachers/students can edit locally.
+
+Downstream use:
+- Interactive educational apps, coding workshops, and science visualization demos.
+
+Out of scope:
+- Safety-critical software.
+- Scientific simulation requiring high-precision numerical correctness.
+- Unreviewed classroom deployment without human validation.
+
+## Training Data
+
+Dataset summary:
+- 1,036 instruction/code examples.
+- 124 unique K-12 science topics.
+- Synthetic dataset generated via parallel agent workflows and validated into JSONL format.
+
+Each example contains:
+- `instruction`
+- `topic`
+- `grade_level`
+- `p5js_code` (full runnable sketch)
+
+Data style constraints emphasized:
+- 600x400 canvas
+- `setup()` and `draw()` structure
+- labels/annotations for explanation
+- interactive and animated behavior
+
+## Training Procedure
+
+Hardware and runtime:
+- 4x NVIDIA A100-SXM4-80GB
+- ~171.9 seconds total training runtime
+- multi-GPU training via `accelerate`
+
+LoRA config:
+- `r=64`
+- `lora_alpha=128`
+- `lora_dropout=0.05`
+- target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
+
+Optimization config:
+- epochs: 3
+- per-device batch size: 4
+- gradient accumulation: 2
+- effective batch size: 32 (4 GPUs)
+- learning rate: `2e-4`
+- scheduler: cosine, warmup ratio 0.05
+- weight decay: 0.01
+- max sequence length: 2048
+- precision: bf16
+- optimizer: AdamW
+
+## Training/Eval Snapshot
+
+- Train loss (logged): `0.9090 -> 0.4950` (step 10 to step 90)
+- Train run average loss (`train_metrics.json`): `0.5917`
+- Eval loss (step 50): `0.6164`
+- Reported token accuracy during run: up to ~85.6%
+
+These are training-time indicators, not full benchmark performance across external test sets.
+
+## Quick Start
+
+### Transformers + PEFT
 
 ```python
+import torch
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
-import torch
 
-base_model = AutoModelForCausalLM.from_pretrained(
-    "Qwen/Qwen3-0.6B",
+BASE = "Qwen/Qwen3-0.6B"
+ADAPTER = "mr-dee/qwen3-p5js-physics-lora"
+
+tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    BASE,
     torch_dtype=torch.bfloat16,
     device_map="auto",
+    trust_remote_code=True,
 )
-model = PeftModel.from_pretrained(base_model, "mr-dee/qwen3-p5js-physics-lora")
-tokenizer = AutoTokenizer.from_pretrained("mr-dee/qwen3-p5js-physics-lora")
-
-prompt = """<|im_start|>system
-You are a p5.js animation expert for K-12 physics education. Generate complete, working p5.js code that creates educational animations.
-<|im_end|>
-<|im_start|>user
-Create a p5.js animation showing projectile motion with adjustable launch angle
-<|im_end|>
-<|im_start|>assistant
-"""
-
+model = PeftModel.from_pretrained(model, ADAPTER)
+
+messages = [
+    {
+        "role": "system",
+        "content": (
+            "You are a p5.js animation expert for K-12 physics education. "
+            "Return complete, runnable JavaScript."
+        ),
+    },
+    {
+        "role": "user",
+        "content": "Create an interactive p5.js sketch that teaches projectile motion.",
+    },
+]
+
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.7, top_p=0.9)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=1024,
+    temperature=0.4,
+    top_p=0.9,
+)
 print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
-### With vLLM (recommended for serving)
+### vLLM Serving (LoRA)
 
 ```bash
 vllm serve Qwen/Qwen3-0.6B \
@@ -79,93 +178,56 @@ vllm serve Qwen/Qwen3-0.6B \
   --max-model-len 2048
 ```
 
-Then query via the OpenAI-compatible API:
+Then query:
 
 ```bash
-curl http://localhost:8000/v1/chat/completions \
+curl http://127.0.0.1:8000/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
     "model": "p5js",
     "messages": [
-      {"role": "system", "content": "You are a p5.js animation expert for K-12 physics education."},
-      {"role": "user", "content": "Create an animation showing wave interference patterns"}
+      {"role":"system","content":"You are a p5.js animation expert for K-12 physics education."},
+      {"role":"user","content":"Create an animation showing wave interference with labeled nodes and antinodes."}
     ],
-    "max_tokens": 2048,
-    "temperature": 0.7
+    "max_tokens": 800,
+    "temperature": 0.4
   }'
 ```
 
-## Training Details
-
-### Dataset
-
-1,036 synthetic instruction-code pairs generated using parallel Claude agents across 124 K-12 science topics:
-
-- **Physics**: gravity, Newton's laws, projectile motion, pendulums, momentum, friction, centripetal force
-- **Waves & Optics**: wave interference, double slit experiment, reflection, refraction, Doppler effect
-- **Electricity & Magnetism**: circuit diagrams, electromagnetic induction, Faraday's law
-- **Astronomy**: orbital mechanics, Kepler's laws, stellar lifecycle, tidal forces
-- **Chemistry**: atomic structure, gas laws, chemical bonding, phase transitions
-- **Biology**: photosynthesis, cell division, DNA replication, ecosystem dynamics
-- **Earth Science**: plate tectonics, volcano lifecycle, water cycle, rock cycle
-
-Each example contains a natural language instruction and complete p5.js code using a 600x400 canvas with `setup()`/`draw()`, text labels, and smooth animations.
+Important:
+- keep `prompt_tokens + max_tokens <= max_model_len`
+- otherwise vLLM returns HTTP 400 validation errors
 
-### LoRA Configuration
+## Limitations and Risks
 
-| Parameter | Value |
-|---|---|
-| Rank (r) | 64 |
-| Alpha | 128 |
-| Dropout | 0.05 |
-| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
-| Trainable parameters | 40.4M (5.1% of 792M) |
+- Output is not guaranteed bug-free JavaScript; review before use.
+- Physical explanations may be simplified or occasionally incorrect.
+- Performance drops for domains outside training distribution.
+- Small base model can struggle with very long or highly complex simulations.
 
-### Hyperparameters
+## Responsible Use
 
-| Parameter | Value |
-|---|---|
-| Epochs | 3 |
-| Effective batch size | 32 (4/device x 2 grad_accum x 4 GPUs) |
-| Learning rate | 2e-4 |
-| LR scheduler | Cosine with 5% warmup |
-| Optimizer | AdamW (weight decay 0.01) |
-| Max sequence length | 2048 |
-| Precision | bf16 |
-| Gradient checkpointing | Enabled |
+- Use with human review in educational settings.
+- Validate scientific correctness before presenting to students.
+- Sandbox or lint generated code before running in production applications.
 
-### Training Loss Progression
-
-| Step | Loss | Accuracy |
-|---|---|---|
-| 10 | 0.909 | 77.0% |
-| 30 | 0.621 | 82.3% |
-| 50 | 0.549 | 84.0% |
-| 70 | 0.510 | 84.9% |
-| 93 | 0.495 | 85.6% |
-
-## Hardware
-
-- **GPUs**: 4x NVIDIA A100-SXM4-80GB
-- **Training time**: 171.9 seconds (~2.9 minutes)
-- **Throughput**: 17.2 samples/sec, 0.54 steps/sec
-- **Adapter size**: 155 MB
-
-## Source Code
-
-Full training pipeline, dataset generation scripts, and serving code: [github.com/dylanler/qwen3-p5js-physics](https://github.com/dylanler/qwen3-p5js-physics)
+## Framework Versions
 
-## Limitations
+- PEFT: 0.18.1
+- Transformers: 4.57.6
+- TRL: 0.27.2
+- PyTorch: 2.9.1
+- Accelerate: 1.12+
 
-- Optimized for p5.js code generation only; not a general-purpose code model
-- Best results with physics/science animation prompts matching the training distribution
-- Generated code may occasionally have minor bugs requiring manual fixes
-- Small base model (0.6B) limits complexity of generated animations compared to larger models
+## Citation
 
-## Framework Versions
+If you use this adapter, cite:
 
-- **PEFT**: 0.18.1
-- **Transformers**: 4.57.6
-- **TRL**: 0.27.2
-- **PyTorch**: 2.9.1
-- **Accelerate**: 1.12+
+```bibtex
+@misc{qwen3_p5js_physics_lora_2026,
+  title  = {qwen3-p5js-physics-lora},
+  author = {mr-dee},
+  year   = {2026},
+  url    = {https://huggingface.co/mr-dee/qwen3-p5js-physics-lora}
+}
+```