hanemay/smartsightCoach

SmartSight Coach LoRA fine-tunes

LoRA fine-tunes for the SmartSight AI coach — an on-device fitness and nutrition coach that runs entirely on the phone, no account and no server round-trip. Converted to .litertlm for Android/iOS inference. → smartsight.app/ai-coach

Built and maintained by Niclas Bade.

Base model changed at v42 (2026-08-12). v27–v38 were fine-tuned from google/gemma-4-E2B-it. Everything from v42 on is fine-tuned from Google's quantization-aware-trained checkpoint google/gemma-4-E2B-it-qat-q4_0-unquantized, whose weights are conditioned to survive 4-bit rounding. The frontmatter above reflects the CURRENT parent, so the model tree matches what actually shipped. This repo keeps the current best checkpoints plus earlier superseded milestones for history — not every candidate tested each round. Rounds that lose are documented internally with their real failure cases; they are never uploaded here.

More coming. The vision tower is being trained surface by surface — body fat, meal photos and nutrition-label reading — each measured against held-out data before anything ships. Progress and honest limitations for every round are written up below and at smartsight.app/ai-coach.

smart-coach-vision.litertlm — CURRENT CHAMPION (v85)

Promoted 2026-08-23. Lineage: v27 → v31 → v34 → v38 → v42 → v44 → v46 → v47 → v48 → v51 → v54 → v55 → v69 → v85. sha256 f9f62c5b9b7c3ee6…, 2,795,655,584 bytes.

Measured against v69 on the same box, same session, same backend, on a 29-row photo set that now includes ten DEXA-measured subjects. Text and follow-ups were run on two eval seeds each.

What improved

surface	v69	v85
body fat, 17 real photographs	MAE 1.99, slope 0.815	MAE 1.69, slope 1.043
body fat, high end (21 / 24 / 26%)	−5.0 / −3.0 / −5.0 pp	−5.0 / 0.0 / −2.0 pp
body fat, read bias	−1.28	−0.05
coach questions (128 cases, two seeds)	123 / 121	123 / 122
conversational follow-ups (28 cases, two seeds)	17 / 16	16 / 18

The slope is the number that matters. v69 compressed the range — it read high body fat as lower than it was, calling a 24% physique 21 and a 26% physique 21. v85 tracks measured values roughly one-for-one. Coach text and follow-ups are level within noise; nothing was traded away for this.

⚠️ What did NOT change — the lean end

photograph (DEXA truth)	v69	v85
5.9%	8	8
8.8%	8	8
10.0%	8	8
12.0%	12	12
13.0%	12	12

Every lean photograph reads identically to the previous model. Three different measured truths — 5.9, 8.8 and 10.0 — all get the same answer, in both models. v69's apparent strength at the lean end was never accuracy; it answers 8 to most lean physiques whatever the true value, and v85 inherits that unchanged. Fixing it needs more DEXA-measured lean subjects, not another training round.

⚠️ Honest limits of this result

• The gain rests on three photographs. Of 17 real photographs, 13 read identically to v69, 3 improved and 1 got worse. The improvement is concentrated at the high end.
• Not seed-confirmed — a weaker standard than v69. v69 trained three seeds on one config and won on two independent eval seeds. v85 is a single training seed. In the same round, two runs at effectively the same corpus ratio scored 3.28 and 1.69 — a 1.6-point spread, larger than this model's 0.30-point edge. The artifact measurably scores better; the recipe is not shown to reproduce it.
• Generated-image holdout is slightly worse, within noise on error but not on bias: MAE 1.97 → 2.03, bias −0.21 → +1.06. Watch this if it grows.
• Still answers a confident percentage for things that are not bodies (a grey rectangle reads 16–20%). Unchanged from v69 — a known open defect, not a regression.

smart-coach-vision.litertlm — previous champion (v69)

Promoted 2026-08-20, superseded by v85 on 2026-08-23. sha256 b15b50a58bd74699…, 2,795,655,584 bytes. Kept here because it is the model v85 was measured against. Three seeds on an identical config, winner confirmed on two independent eval seeds — a stricter standard than v85 met.

surface	v55	v69
coach questions (128 cases, two seeds)	118 / 118	123 / 121
"what did I train yesterday?"	24 / 32	32 / 32 and 30 / 32
outside the 7-day window	30 / 32	32 / 32
conversational follow-ups (28 cases)	14 / 28	17 / 28
body fat, lean tail, 37 unseen photos	MAE 2.88, bias +2.88	MAE 1.88, bias +1.82
body fat, most common single answer	43% of photos	19%

⚠️ v69 introduced one regression — asked "I trained on Wednesday, does that count?" about a past day inside the window it sometimes said no, when the session is logged and does count (32/32 → 27/32). v85 partly repairs it (29/32 and 28/32). Full detail in the champion history below.

---

Champion history — what each round actually fixed

Every entry here was a promoted champion, verified against the previous one on a held-out suite with a full read of every answer. Rejected rounds (v28-v30, v32, v33, v35-v37, v39, v40, v41) are not listed — they are documented internally with their real failure cases.

Round	Beat	What it fixed	Base
v27	baseline	first winning fine-tune — shorter, equally accurate general advice	gemma-4-E2B-it
v31	v27	exercise-form answers reached the untouched baseline's zero-error record (v27 was wrong on 4/5 direct form questions); no more garbled or duplicated plan output. The unlock was the export recipe, not data — three consecutive data-only rounds failed first, then Hadamard-rotation int4 fixed it with zero retraining	gemma-4-E2B-it
v31 export update	—	same weights, AlgorithmName.HADAMARD_ROTATION custom op instead of the decomposed variant: 88% faster decode, ~85 MB smaller, quality statistically unchanged (5.8% vs 6.4% defect rate over 312 generations)	—
v34	v31	dietary-rule compliance (a "vegetarian, no nuts" request had served peanut butter); first correct full examples for face pull and plank; squat-vs-hip-thrust contrast cases	gemma-4-E2B-it
v38	v34	AI challenges: unrealistic targets for low-frequency goals (150 progress photos in six weeks from zero) and non-English requests answering in English. Won over its own alternate seed on a Danish grammar defect — plural "jeres/jer" where the app addresses one person	gemma-4-E2B-it
v42	v38	see above — QAT base, blockwise export, and the fenced-JSON data fix. Won over its own alternate seed, which produced a confident wrong number (subtracting a 78 kg bodyweight goal from a 106.7 kg squat 1RM)	gemma-4-E2B-it-qat-q4_0-unquantized
v44	v42	Danish macro vocabulary and challenge titles. Measured cause: of 7,288 corpus rows, 13 contained any Danish/Norwegian, zero taught "kulhydrat", and ten demonstrated exactly the wrong behaviour — English macro words inside Danish prose. Localising those ten rows fixed it; /nutrition emission went 5/7 → 7/7	gemma-4-E2B-it-qat-q4_0-unquantized
v46	v44	avoid-list violations eliminated (0/36 vs 5/36) by matching the corpus to the prompt format production actually sends	gemma-4-E2B-it-qat-q4_0-unquantized
v47	v46	no retraining — the vision export ran at 140 soft tokens instead of the 280 the checkpoint specifies, so every shipped build saw 48.9% of the pixel area. The asymmetry proved the mechanism: re-exporting at 280 moved body-fat rank correlation in men +0.492 → +0.704 while women were unchanged, because what separates adjacent bands in men is fine texture only a few pixels tall and downscaling is a low-pass filter	gemma-4-E2B-it-qat-q4_0-unquantized
v48	v47	conversational follow-ups: the coach dropped the app's own numbers as soon as a user pushed back. Failures 16/24 → 8/24	gemma-4-E2B-it-qat-q4_0-unquantized
v51	v48	stopped abandoning the app's numbers on a user contradiction, and stopped inventing plausible detail it could not see ("right in the middle of where it usually sits" → "I can't tell what your usual is")	gemma-4-E2B-it-qat-q4_0-unquantized
v54	v51	read the wrong cell of the 7-day history line: asked "what did I train yesterday?" it answered about TODAY, 18 times out of 32	gemma-4-E2B-it-qat-q4_0-unquantized
v55	v54	the body-fat read was a CONSTANT — 18% for all 42 test photos, and also for a grey rectangle, for pure noise, and for no image at all. Cause was the training data, not the vision tower: a text-only LoRA destroys vision the base model already has (sensitivity fell 0.1164 → 0.0405 across v51→v54 with ZERO vision tensors in the adapter — the language layers consume the image tokens). Mixing ~17% of the vision surface's rows back into the text corpus restored it: MAE 9.13 → 3.42, sensitivity 0.0405 → 0.2183, grey rectangle finally separating from real photos. ⚠️ It also lost the base model's striation detection while gaining the percentage read — capabilities traded, not accumulated, which is why every surface is probed at promotion now	gemma-4-E2B-it-qat-q4_0-unquantized
v69	v55	relative-date reads ("what did I train yesterday") 24/32 → 32/32. v55 had a consistent off-by-one — it treated the last day in the 7-day line as yesterday instead of today, then described the wrong row accurately: fluent, confident, wrong. Six earlier rounds tried to train it away and every one made it worse. What worked was teaching the model to STATE THE ANCHOR BEFORE USING IT ("the line ends with Sun, so today is Sun; one back is Sat"), turning an indexing problem into two lookups it could already do. Body fat on 37 unseen ordinary phone photos (bathroom mirrors, kitchens, garages — not studio imagery) 2.88 → 1.88 pp, and the most common single answer fell 43% → 19% of photos. ⚠️ It also introduced a regression, fixed by addition not retreat: 100% of the new rows ask the model to locate a day BEHIND today, so it over-generalised to "behind today = not current" and started saying a logged past session doesn't count (32/32 → 27/32, identical on all three seeds)	gemma-4-E2B-it-qat-q4_0-unquantized
v85	v69	body-fat reads on real photographs stopped compressing the range: slope 0.815 → 1.043, mean error 1.99 → 1.69 pp, and the high end went from reading 24% as 21 to reading it as 24. Coach text and follow-ups level on two seeds. ⚠️ the lean end did not move at all — 5.9%, 8.8% and 10.0% all still read 8, exactly as in v69	gemma-4-E2B-it-qat-q4_0-unquantized

Recipe constant across every round since v22c: LoRA r=16 / alpha=32 / dropout=0.1, 2 epochs, lr 6e-5, NEFTune noise_alpha=5, completion-only loss masking (train_on_responses_only), per-task validation shards, two seeds per round with the winner chosen on behaviour rather than validation loss. Only the training data, the base, and the export recipe have moved.

Honest current weaknesses

Documented rather than hidden, because they are the targets for the next rounds:

• Body fat at the lean end is a bucket, not a read. DEXA-measured 5.9%, 8.8% and 10.0% photographs all return 8 — three different truths, one answer, in v85 exactly as in v69. A good average error there is an artefact of the band being narrow, not evidence of accuracy. The fix is more DEXA-measured lean subjects; no amount of retraining on the current set moves it.

• Images that are not bodies still get a confident percentage. A plain grey rectangle reads 16–20%, as does pure noise. The model has no category for "this is not a person" — absence of visible definition and absence of a body produce the same answer. Prompting does not fix it: a variant instructing the model to reply NOT_A_BODY was ignored, returning byte-identical output.

• Meal-plan portions are sized by habit, not arithmetic. The model anchors on ~100 g portions rather than solving the prompt's stated calorie budget. Mostly invisible in the app, which rescales a day to its target (MealPlanScaler, clamped 0.6x–2.0x) — but one case per model is still off after that clamp, and the underlying arithmetic is unsolved.

• Avoid-lists are not always respected — foods explicitly banned for variety reappear (2/29 held-out cases on v44, v42 and v38 alike; this is the single defect class that has survived every round so far).

• Nordic follow-ups about an UNLOGGED past activity regressed in v44. Asked in Norwegian about a hike "yesterday" that is not in the app, v42 correctly said the gap was only a logging omission; v44 answers as though advising about today. 26 Nordic follow-up rows were added to fix exactly this and did not move it — the cause is not simply coverage.

• Occasional Danish/Norwegian word-formation slips in v44 — "vækning" (not a word) and the hybrid "fat-mål". The macro vocabulary itself is now correct, but morphology is not reliable.

• The /nutrition command line is improvised. The corpus contains ZERO examples of it; the format lives only in the prompt. v44 emits the line on 7/7 applicable cases (v42 omitted it entirely on 2/7) but drops the trailing fat value on 3/7, so the app applies three of the four fields. Harmless by design — NutritionCommandParser accepts any subset — but the fat target silently stays stale.

• Within-day duplicate or misplaced exercises still appear occasionally in generated routines (e.g. a chest isolation movement landing in a Legs day). Lower real-world impact than it sounds, since the app's own parser drops duplicate slots.

• Confabulation-despite-coverage on a few specific lifts (squat/hip-thrust blending, farmer's carry) that does not reliably respond to more training data.

• Superset pairing — avoiding two competing-muscle compounds back to back — is imperfect.

• Tight macro-budget precision (±8–10%) is unsolved across every model tested, including the untouched baseline.

Evaluation discipline

Every promote/reject decision reads every row of the eval suite, never a sample — an early round reported a verdict from ~12 of 52 rows and missed hard timeout loops entirely. Prose quality is judged as its own axis alongside structural correctness, because a structurally clean answer can still be flat, templated, or subtly wrong in a second language. Candidates are always compared against both the previous champion and the untouched baseline.

Files

Which file the app fetches

Two files are published. The app decides which one to fetch based on its build version; there is nothing to select and nothing to configure.

file	who gets it	size
smart-coach-vision.litertlm	⭐ current champion — text coaching AND vision (body-fat photos, meal photos, nutrition labels) in one model. Fetched by the next app release.	2.80 GB
smart-coach.litertlm	text-only, fetched by app builds already installed. Unchanged on purpose.	2.63 GB

Both carry the same v46 coach weights (v47 changes only the vision export, not the weights), so an older build is not stuck on an older coach. They exist side by side because installed builds also fetch a separate 3.66 GB vision model — handing them the larger consolidated file would raise their total download on devices already tight on RAM. See the current-champion section above for what the consolidation actually changes.

• smart-coach-vision.litertlm — ⭐ current champion, fetched by the next app release. The v46 coach WITH a working vision tower: one model for text coaching, body-fat photos, meal photos and nutrition labels (2.80 GB). The next app release points both the coach and vision URLs here and stops downloading the separate 3.66 GB vision model.
• smart-coach.litertlm — STABLE, text-only (2.63 GB). The file app builds already in the wild download at runtime on Android and iOS, and the one they keep using. Deliberately left unchanged: those builds also fetch the separate vision model, so giving them the larger consolidated file would raise their total download on devices that are already RAM-constrained. Same v46 coach weights, so nobody is stuck on an older coach for stability's sake.
• coach.litertlm — the original working int8 build from the v1 era (2.59 GB), kept as the historical starting point.
• coach-finetuned-int4.litertlm — the first int4 conversion (2.56 GB), kept because it is the artifact that exposed the LiteRT-LM chat-template .get() incompatibility.
• model.safetensors + config.json + tokenizer files — HF-format artifacts for reference.