tthhanh commited on
Commit
77edc70
·
1 Parent(s): 30fed09

feat: expand coach summary training pipeline

Browse files
configs/coach_summary_lora.default.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "base_model": "Qwen/Qwen2.5-7B-Instruct",
3
  "train_file": "/Users/h/Documents/Pozify/data/sft/coach_summary_train.jsonl",
4
  "eval_file": "/Users/h/Documents/Pozify/data/sft/coach_summary_eval.jsonl",
5
  "output_dir": "/Users/h/Documents/Pozify/models/coach_summary_lora",
@@ -11,4 +11,4 @@
11
  "per_device_train_batch_size": 1,
12
  "gradient_accumulation_steps": 8,
13
  "max_seq_length": 4096
14
- }
 
1
  {
2
+ "base_model": "Qwen/Qwen3-14B",
3
  "train_file": "/Users/h/Documents/Pozify/data/sft/coach_summary_train.jsonl",
4
  "eval_file": "/Users/h/Documents/Pozify/data/sft/coach_summary_eval.jsonl",
5
  "output_dir": "/Users/h/Documents/Pozify/models/coach_summary_lora",
 
11
  "per_device_train_batch_size": 1,
12
  "gradient_accumulation_steps": 8,
13
  "max_seq_length": 4096
14
+ }
docs/coach-model-data-plan.md ADDED
@@ -0,0 +1,654 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Pozify Coach Model Data Plan
2
+
3
+ ## Objective
4
+
5
+ Define a practical data and modeling plan for improving Pozify's coach summary and coach-style guidance using open datasets and the current codebase.
6
+
7
+ This document answers:
8
+
9
+ - Which public datasets are actually useful for Pozify's current problem.
10
+ - Whether to use RAG, SFT, or both.
11
+ - How those choices fit the current pipeline, artifacts, verifier, and UI.
12
+ - A concrete implementation roadmap that does not break the current app architecture.
13
+
14
+ ## Current Codebase Fit
15
+
16
+ Pozify today is not a generic fitness chatbot. It is a structured video-analysis pipeline that ends with a grounded coach summary:
17
+
18
+ - `video_qc` -> `pose_landmarker` -> `exercise_classifier` -> `rep_analysis` -> `variation` -> `issue_markers` -> `coach_summary` -> `verifier`
19
+ - The coach summary step already consumes structured evidence and retrieved knowledge cards.
20
+ - The verifier already rejects summaries that invent issues, overcorrect valid variations, or make unsafe claims.
21
+
22
+ Relevant modules:
23
+
24
+ - `src/pozify/pipeline.py`
25
+ - `src/pozify/steps/coach_summary.py`
26
+ - `src/pozify/slm/prompting.py`
27
+ - `src/pozify/knowledge_cards.py`
28
+ - `src/pozify/steps/verifier.py`
29
+
30
+ That means any dataset strategy must be evaluated against the real task:
31
+
32
+ > Generate safe, grounded, structured coaching language from Pose/Rep/Variation/Issue JSON artifacts.
33
+
34
+ This is **not** the same task as:
35
+
36
+ - answering open-ended fitness questions,
37
+ - free-form nutrition coaching,
38
+ - broad lifestyle coaching,
39
+ - or generic health chat.
40
+
41
+ ## Recommendation Summary
42
+
43
+ ### Best near-term strategy
44
+
45
+ Use a **hybrid approach**:
46
+
47
+ 1. **Expand knowledge retrieval first** using a curated exercise knowledge base.
48
+ 2. **Keep coach summary grounded on structured Pozify artifacts.**
49
+ 3. **Use lightweight SFT only after you have a Pozify-native training set** built from your own analysis artifacts and target summaries.
50
+
51
+ ### What not to do first
52
+
53
+ Do **not** directly fine-tune the current coach-summary model on broad fitness chat datasets and expect it to improve the grounded summary step.
54
+
55
+ That would likely:
56
+
57
+ - make the model more conversational,
58
+ - but also make it more likely to hallucinate,
59
+ - talk beyond the provided JSON,
60
+ - and fight against the verifier.
61
+
62
+ ### Strong recommendation
63
+
64
+ For the current codebase, the best order is:
65
+
66
+ 1. **RAG / retrieval upgrade**
67
+ 2. **Pozify-native supervised dataset creation**
68
+ 3. **LoRA / SFT on the Pozify-native task**
69
+ 4. **Optional separate chatbot mode** trained on public chat/Q&A datasets
70
+
71
+ ## Dataset-by-Dataset Assessment
72
+
73
+ ### 1. `onurSakar/GYM-Exercise`
74
+
75
+ Source notes:
76
+
77
+ - Hugging Face dataset page shows `1.66k` rows and a single text field formatted in Llama-style instruction format.
78
+ - The visible examples include fitness Q&A, but also broader wellness topics such as environmental stressors, work-life balance, mental clarity, and social well-being.
79
+
80
+ Fit for Pozify:
81
+
82
+ - Good for **general fitness coach tone**.
83
+ - Weak fit for **grounded rep-by-rep coach summary generation**.
84
+ - Weak fit for **variation-vs-issue reasoning** tied to structured pose artifacts.
85
+
86
+ Risks:
87
+
88
+ - Topic drift beyond movement coaching.
89
+ - May encourage generic answer patterns instead of artifact-conditioned reasoning.
90
+ - Could increase verifier failures because the model may mention things outside `issue_markers.json`.
91
+
92
+ Best use in Pozify:
93
+
94
+ - Use only as **auxiliary style data**.
95
+ - Filter to movement-technique and workout-guidance samples.
96
+ - Do not use raw as the main SFT dataset for `coach_summary`.
97
+
98
+ Verdict:
99
+
100
+ - **Useful later**
101
+ - **Not the primary dataset for the current summary pipeline**
102
+
103
+ ### 2. `HazSylvia/Fitness_Unformatted`
104
+
105
+ Source notes:
106
+
107
+ - Hugging Face page shows `928` rows.
108
+ - Fields appear as `Human` and `Assistant`.
109
+ - Examples are cleaner fitness Q&A than the dataset above and closer to gym onboarding and workout guidance.
110
+
111
+ Fit for Pozify:
112
+
113
+ - Better than `onurSakar/GYM-Exercise` for conversational coach tone.
114
+ - Still not tightly aligned with Pozify's structured JSON-to-summary task.
115
+
116
+ Risks:
117
+
118
+ - Still mostly free-form Q&A, not evidence-grounded explanation.
119
+ - Lacks explicit inputs like `rep_analysis`, `variation`, `issue_markers`.
120
+
121
+ Best use in Pozify:
122
+
123
+ - Use as **secondary SFT data for tone/voice**.
124
+ - Convert to JSONL and filter for exercise technique, beginner coaching, and workout-planning responses.
125
+ - Consider this dataset for a future `Ask Pozify` chat mode.
126
+
127
+ Verdict:
128
+
129
+ - **Good for coach voice**
130
+ - **Not enough by itself for the current pipeline**
131
+
132
+ ### 3. `chibbss/fitness-chat-prompt-completion-dataset`
133
+
134
+ Source notes:
135
+
136
+ - Hugging Face page shows `245` rows.
137
+ - Has `instruction` and `output` fields.
138
+ - Content appears to focus on healthy habits, routines, stress, sleep, and lifestyle guidance.
139
+
140
+ Fit for Pozify:
141
+
142
+ - Good format for instruction tuning.
143
+ - Too small and too broad to be the main fine-tuning source for the current task.
144
+
145
+ Risks:
146
+
147
+ - Small size.
148
+ - Lifestyle-heavy answers may not improve form-review summaries.
149
+ - Could push the model toward broad wellness advice instead of evidence-based movement coaching.
150
+
151
+ Best use in Pozify:
152
+
153
+ - Optional augmentation for tone.
154
+ - Better suited to a future “habit coach” or “ask a coach” feature than to movement-summary generation.
155
+
156
+ Verdict:
157
+
158
+ - **Supplementary only**
159
+
160
+ ### 4. `hasaneyldrm/exercises-dataset`
161
+
162
+ Based on your description, this is the most strategically valuable dataset for Pozify right now.
163
+
164
+ Fit for Pozify:
165
+
166
+ - High fit for retrieval and knowledge expansion.
167
+ - It maps well to Pozify's current `knowledge_cards` architecture.
168
+ - It improves factual consistency for exercise descriptions, target muscles, equipment, and execution cues.
169
+
170
+ Why it fits the codebase:
171
+
172
+ - Pozify already retrieves deterministic cards in `src/pozify/knowledge_cards.py`.
173
+ - The prompt builder already injects knowledge cards into the model input.
174
+ - The current gap is not “more chat data”; the gap is “richer, broader, more standardized exercise knowledge.”
175
+
176
+ Best use in Pozify:
177
+
178
+ - Use this as the basis for a **local knowledge base** first.
179
+ - Convert exercise rows into:
180
+ - exercise cards,
181
+ - variation cards,
182
+ - equipment-aware cues,
183
+ - and optional goal-specific cue overlays.
184
+ - Later, optionally index it in FAISS/Chroma for semantic retrieval in a separate chat flow.
185
+
186
+ Verdict:
187
+
188
+ - **Best immediate dataset to integrate**
189
+
190
+ ### 5. `strova-ai/fitness-tracker-dataset`
191
+
192
+ Source notes:
193
+
194
+ - Hugging Face page describes it as a synthetic wearable/activity dataset.
195
+ - It contains tabular metrics such as age, gender, height, weight, steps, heart rate, calories, distance, and activity.
196
+ - The page also shows dataset-viewer schema issues, which suggests ingestion may need cleanup.
197
+
198
+ Fit for Pozify:
199
+
200
+ - Useful for personalization experiments.
201
+ - Weak fit for current video-form summary generation.
202
+
203
+ Why it does not fit immediately:
204
+
205
+ - Pozify's current app does not reason from wearable timeseries.
206
+ - The current `UserProfile` contract is small and categorical.
207
+ - There is no current component that transforms tabular physiology into grounded motion advice.
208
+
209
+ Best use in Pozify:
210
+
211
+ - Optional later-stage personalization module.
212
+ - Useful for future “today's activity context” or “session recommendation” features.
213
+ - Not a priority for the current coach summary pipeline.
214
+
215
+ Verdict:
216
+
217
+ - **Future personalization dataset, not Phase 1**
218
+
219
+ ## What This Means For Pozify
220
+
221
+ ### The current task is not generic chat SFT
222
+
223
+ Pozify's core output is:
224
+
225
+ - bounded by `exercise_classification.json`
226
+ - bounded by `rep_analysis.json`
227
+ - bounded by `variation.json`
228
+ - bounded by `issue_markers.json`
229
+ - conditioned by knowledge cards
230
+ - checked by a verifier
231
+
232
+ So the most valuable training examples are not public generic fitness Q&A.
233
+
234
+ The most valuable examples look like this:
235
+
236
+ ```json
237
+ {
238
+ "analysis_json": {
239
+ "user_profile": {},
240
+ "exercise_classification": {},
241
+ "rep_analysis": {},
242
+ "variation": {},
243
+ "issue_markers": {}
244
+ },
245
+ "retrieved_knowledge_cards": [],
246
+ "ideal_coach_summary": {
247
+ "summary": "",
248
+ "what_you_did": [],
249
+ "what_looked_good": [],
250
+ "what_changed_across_reps": [],
251
+ "valid_variation_vs_issue": [],
252
+ "top_fixes": [],
253
+ "next_session_plan": [],
254
+ "confidence_notes": []
255
+ }
256
+ }
257
+ ```
258
+
259
+ This is already much closer to the current runtime prompt contract than any public chat dataset you listed.
260
+
261
+ ## Recommended Strategy
262
+
263
+ ## Option A: Retrieval-first upgrade
264
+
265
+ ### Why this is the best first move
266
+
267
+ It fits the current architecture with the least risk.
268
+
269
+ It improves:
270
+
271
+ - factual exercise guidance,
272
+ - coaching cue coverage,
273
+ - equipment-specific instructions,
274
+ - and future exercise expansion,
275
+
276
+ without retraining the model first.
277
+
278
+ ### How to use `hasaneyldrm/exercises-dataset`
279
+
280
+ Convert exercise rows into a normalized internal artifact such as:
281
+
282
+ ```json
283
+ {
284
+ "card_id": "exercise:barbell_bench_press",
285
+ "card_type": "exercise",
286
+ "labels": ["barbell_bench_press", "bench_press"],
287
+ "title": "Barbell Bench Press",
288
+ "summary": "Compound upper-body press.",
289
+ "target_muscles": ["chest"],
290
+ "secondary_muscles": ["triceps", "front_delts"],
291
+ "equipment": ["barbell", "bench"],
292
+ "instructions": [
293
+ "Set the shoulder blades before unracking.",
294
+ "Lower with control to a repeatable touch point.",
295
+ "Press while keeping wrists stacked."
296
+ ]
297
+ }
298
+ ```
299
+
300
+ ### Implementation in current codebase
301
+
302
+ 1. Add a data ingestion script under `scripts/`:
303
+ - `scripts/build_exercise_knowledge_base.py`
304
+ 2. Store normalized cards under:
305
+ - `data/knowledge/exercises.json`
306
+ 3. Update `src/pozify/knowledge_cards.py` to:
307
+ - load built-in cards,
308
+ - merge external exercise cards,
309
+ - keep deterministic retrieval for current exercise/variation/issue labels.
310
+ 4. Keep the current coach-summary flow intact.
311
+
312
+ ### Benefits
313
+
314
+ - Immediate product improvement.
315
+ - No retraining required.
316
+ - Low safety risk.
317
+ - Reuses the current prompt architecture.
318
+
319
+ ## Option B: Pozify-native SFT for coach summary
320
+
321
+ ### Why this is the right SFT target
322
+
323
+ If you do SFT, train on the task Pozify actually runs:
324
+
325
+ - structured evidence in
326
+ - structured grounded summary out
327
+
328
+ ### Recommended data source for SFT
329
+
330
+ Build your own dataset from:
331
+
332
+ - current artifact pipeline outputs in `runs/`
333
+ - synthetic or hand-authored expert summaries
334
+ - optional edits by the team
335
+
336
+ Then optionally augment with:
337
+
338
+ - filtered examples from `HazSylvia/Fitness_Unformatted`
339
+ - filtered examples from `onurSakar/GYM-Exercise`
340
+ - filtered examples from `chibbss/fitness-chat-prompt-completion-dataset`
341
+
342
+ But use public datasets only for:
343
+
344
+ - tone,
345
+ - phrasing,
346
+ - concise coaching style,
347
+ - and generic structure.
348
+
349
+ Do not use them to replace the grounded target format.
350
+
351
+ ### SFT training objective
352
+
353
+ Input:
354
+
355
+ - current prompt evidence object
356
+ - retrieved cards
357
+
358
+ Output:
359
+
360
+ - the exact `CoachSummary` JSON contract
361
+
362
+ ### Recommended model
363
+
364
+ Use the current runtime-aligned model family first:
365
+
366
+ - `Qwen/Qwen3-14B`
367
+
368
+ Because the app already uses this model family successfully through Hugging Face Inference.
369
+
370
+ ### Recommended training tools
371
+
372
+ - TRL if you want closer Hugging Face-native control
373
+ - Unsloth if you want fast LoRA iteration on a smaller GPU budget
374
+ - Axolotl if you want a cleaner config-driven fine-tuning pipeline
375
+
376
+ ### Best fine-tuning style
377
+
378
+ - LoRA / QLoRA
379
+ - JSON-only target format
380
+ - deterministic eval checks against verifier outcomes
381
+
382
+ ### Why not train directly on public chat sets first
383
+
384
+ Because the task mismatch is too large:
385
+
386
+ - public chat sets teach “fitness assistant”
387
+ - Pozify needs “artifact-grounded explanation generator”
388
+
389
+ ## Option C: Separate chat / Ask-a-Coach feature
390
+
391
+ This is where the public datasets are actually strongest.
392
+
393
+ ### Best datasets for that feature
394
+
395
+ - `HazSylvia/Fitness_Unformatted`
396
+ - `onurSakar/GYM-Exercise`
397
+ - `chibbss/fitness-chat-prompt-completion-dataset`
398
+
399
+ ### Suitable user stories
400
+
401
+ - “I am new to the gym, where do I start?”
402
+ - “Can you give me a 3-day weekly plan?”
403
+ - “What should I eat after a workout?”
404
+ - “How do I stay consistent?”
405
+
406
+ ### Why keep this separate from coach summary
407
+
408
+ Because:
409
+
410
+ - the current summary step must stay grounded
411
+ - broad wellness chat should not weaken form-review safety
412
+ - the verifier logic is built for artifact-based outputs, not broad coaching conversations
413
+
414
+ ### Suggested architecture
415
+
416
+ - Keep `coach_summary` as today, evidence-first.
417
+ - Add a future `ask_coach` endpoint and UI tab.
418
+ - Back that feature with:
419
+ - a filtered fitness Q&A dataset,
420
+ - optional RAG over exercise knowledge,
421
+ - and a different safety policy.
422
+
423
+ ## Recommended Final Architecture
424
+
425
+ ### Phase 1
426
+
427
+ - Retrieval-only upgrade from exercise knowledge base
428
+ - No SFT required
429
+
430
+ ### Phase 2
431
+
432
+ - Build Pozify-native summary dataset
433
+ - Fine-tune a LoRA adapter for structured summary generation
434
+
435
+ ### Phase 3
436
+
437
+ - Add optional `Ask Pozify` chat mode
438
+ - Use public Q&A datasets there
439
+
440
+ ### Phase 4
441
+
442
+ - Add personalization features using wearable/tabular data
443
+ - Use `strova-ai/fitness-tracker-dataset` only if product scope expands into recommendations
444
+
445
+ ## Concrete Implementation Plan
446
+
447
+ ## Step 1: Add external exercise knowledge ingestion
448
+
449
+ Goal:
450
+
451
+ - turn exercise reference data into Pozify-ready knowledge cards
452
+
453
+ Work:
454
+
455
+ 1. Create `data/knowledge/` directory.
456
+ 2. Add `scripts/build_exercise_knowledge_base.py`.
457
+ 3. Normalize raw exercise dataset fields into a local JSON schema.
458
+ 4. Add tests for normalization and card loading.
459
+ 5. Update `src/pozify/knowledge_cards.py` to load external cards.
460
+
461
+ Deliverables:
462
+
463
+ - `data/knowledge/exercises.json`
464
+ - updated knowledge card loader
465
+ - tests
466
+
467
+ ## Step 2: Expand current retrieval interface
468
+
469
+ Goal:
470
+
471
+ - use richer exercise and equipment context without changing app UX
472
+
473
+ Work:
474
+
475
+ 1. Keep deterministic retrieval for current labels.
476
+ 2. Add equipment-aware retrieval using `UserProfile.equipment`.
477
+ 3. Add goal-aware retrieval overlays.
478
+ 4. Add per-exercise cue prioritization for the summary prompt.
479
+
480
+ Deliverables:
481
+
482
+ - richer `retrieve_cards(...)`
483
+ - no changes needed in the UI contract
484
+
485
+ ## Step 3: Create a Pozify-native SFT dataset builder
486
+
487
+ Goal:
488
+
489
+ - create training examples from the actual runtime task
490
+
491
+ Work:
492
+
493
+ 1. Add `scripts/build_coach_summary_sft_dataset.py`.
494
+ 2. Read from:
495
+ - `runs/*/exercise_classification.json`
496
+ - `runs/*/rep_analysis.json`
497
+ - `runs/*/variation.json`
498
+ - `runs/*/issue_markers.json`
499
+ - retrieved knowledge cards
500
+ 3. Export JSONL rows like:
501
+
502
+ ```json
503
+ {
504
+ "messages": [
505
+ {"role": "system", "content": "..."},
506
+ {"role": "user", "content": "...structured evidence..."},
507
+ {"role": "assistant", "content": "{...coach summary json...}"}
508
+ ]
509
+ }
510
+ ```
511
+
512
+ 4. Start with hand-authored gold summaries for 100-300 samples.
513
+ 5. Add validation scripts that reject malformed outputs.
514
+
515
+ Deliverables:
516
+
517
+ - `data/sft/coach_summary_train.jsonl`
518
+ - `data/sft/coach_summary_eval.jsonl`
519
+
520
+ ## Step 4: Use public datasets only as auxiliary style data
521
+
522
+ Goal:
523
+
524
+ - improve coach tone without damaging grounded reasoning
525
+
526
+ Work:
527
+
528
+ 1. Add `scripts/prepare_public_fitness_chat_data.py`.
529
+ 2. Ingest:
530
+ - `onurSakar/GYM-Exercise`
531
+ - `HazSylvia/Fitness_Unformatted`
532
+ - `chibbss/fitness-chat-prompt-completion-dataset`
533
+ 3. Filter out:
534
+ - broad medical claims
535
+ - unrelated wellness topics
536
+ - environment/toxin/social-life content
537
+ 4. Convert only the most relevant rows into a secondary style corpus.
538
+ 5. Use this corpus with lower sampling weight than Pozify-native data.
539
+
540
+ Deliverables:
541
+
542
+ - `data/sft/public_fitness_style.jsonl`
543
+
544
+ ## Step 5: Fine-tune with LoRA
545
+
546
+ Goal:
547
+
548
+ - make the model better at emitting Pozify's exact summary contract
549
+
550
+ Work:
551
+
552
+ 1. Choose one training stack:
553
+ - TRL
554
+ - Unsloth
555
+ - Axolotl
556
+ 2. Fine-tune `Qwen/Qwen3-14B`.
557
+ 3. Use:
558
+ - low temperature at inference
559
+ - JSON-only targets
560
+ - training/eval split with verifier-based metrics
561
+ 4. Track:
562
+ - JSON validity rate
563
+ - verifier pass rate
564
+ - unsupported-issue mention rate
565
+ - variation-overcorrection rate
566
+
567
+ Deliverables:
568
+
569
+ - LoRA adapter
570
+ - eval report
571
+ - deployment notes
572
+
573
+ ## Step 6: Add pluggable model loading for local or Hub adapters
574
+
575
+ Goal:
576
+
577
+ - swap between base model and fine-tuned adapter cleanly
578
+
579
+ Work:
580
+
581
+ 1. Extend `src/pozify/slm/providers.py`.
582
+ 2. Add env-driven model selection:
583
+ - base model
584
+ - fine-tuned adapter
585
+ - fallback model
586
+ 3. Surface model metadata in `final_report.artifacts`.
587
+
588
+ Deliverables:
589
+
590
+ - deployable adapter path
591
+ - clear model/version tracking in UI and artifacts
592
+
593
+ ## Step 7: Optional future RAG for free-form chat
594
+
595
+ Goal:
596
+
597
+ - add a broader coach chat feature without weakening grounded summaries
598
+
599
+ Work:
600
+
601
+ 1. Add `src/pozify/chat/` module.
602
+ 2. Index exercise knowledge into:
603
+ - FAISS first, or
604
+ - ChromaDB if persistence/filtering is preferred
605
+ 3. Use RAG only for:
606
+ - exercise explanations
607
+ - equipment substitutions
608
+ - general programming suggestions
609
+ 4. Keep this separate from `coach_summary`.
610
+
611
+ Deliverables:
612
+
613
+ - `Ask Pozify` feature
614
+ - isolated inference path
615
+
616
+ ## Decision Table
617
+
618
+ | Dataset | Best use in Pozify | Use now? | Notes |
619
+ |---|---|---:|---|
620
+ | `onurSakar/GYM-Exercise` | auxiliary style SFT | No | useful tone, but too broad for summary task |
621
+ | `HazSylvia/Fitness_Unformatted` | auxiliary style SFT / future chat | Later | cleaner coach Q&A, still task-mismatched |
622
+ | `chibbss/fitness-chat-prompt-completion-dataset` | auxiliary style SFT / future chat | Later | small, useful for tone only |
623
+ | `hasaneyldrm/exercises-dataset` | retrieval knowledge base | Yes | best immediate fit for current architecture |
624
+ | `strova-ai/fitness-tracker-dataset` | future personalization | No | not aligned with current app contracts |
625
+
626
+ ## Final Recommendation
627
+
628
+ If the goal is to improve the current Pozify product, do this:
629
+
630
+ 1. **Integrate `hasaneyldrm/exercises-dataset` as structured knowledge cards first.**
631
+ 2. **Build a Pozify-native coach-summary SFT dataset from real artifacts.**
632
+ 3. **Fine-tune Qwen with LoRA on the Pozify-native task.**
633
+ 4. **Use public fitness chat datasets only as secondary style data.**
634
+ 5. **Use RAG later for a separate chat feature, not to replace the current grounded summary path.**
635
+
636
+ This path gives the best balance of:
637
+
638
+ - safety,
639
+ - architecture fit,
640
+ - product usefulness,
641
+ - and implementation risk.
642
+
643
+ ## Sources
644
+
645
+ - Hugging Face dataset page for `onurSakar/GYM-Exercise`: https://huggingface.co/datasets/onurSakar/GYM-Exercise
646
+ - Hugging Face dataset page for `HazSylvia/Fitness_Unformatted`: https://huggingface.co/datasets/HazSylvia/Fitness_Unformatted
647
+ - Hugging Face dataset page for `chibbss/fitness-chat-prompt-completion-dataset`: https://huggingface.co/datasets/chibbss/fitness-chat-prompt-completion-dataset
648
+ - Hugging Face dataset page for `strova-ai/fitness-tracker-dataset`: https://huggingface.co/datasets/strova-ai/fitness-tracker-dataset
649
+ - Pozify runtime code:
650
+ - `src/pozify/pipeline.py`
651
+ - `src/pozify/steps/coach_summary.py`
652
+ - `src/pozify/slm/prompting.py`
653
+ - `src/pozify/knowledge_cards.py`
654
+ - `src/pozify/steps/verifier.py`
docs/coach-summary-modal-training.md CHANGED
@@ -13,7 +13,9 @@ It follows the same stage pattern as the exercise-router Modal pipeline:
13
  - `prepare-data`
14
  - `train`
15
  - `evaluate`
 
16
  - `publish`
 
17
  - `all`
18
 
19
  ## Requirements
@@ -29,9 +31,11 @@ modal setup
29
 
30
  - `HF_TOKEN`
31
  - optional `POZIFY_COACH_SUMMARY_HF_REPO_ID`
 
 
32
  - optional `POZIFY_COACH_SUMMARY_HF_PRIVATE=1`
33
 
34
- The script uses `modal.Secret.from_dotenv()`, so for local `modal run ...` usage you do not need to pre-create a hosted Modal secret just to run `prepare-data` or `train`.
35
 
36
  If you prefer a hosted secret for team/shared environments, you can still create one manually in Modal and adapt the script later.
37
 
@@ -72,6 +76,32 @@ Publish the adapter and metadata to Hugging Face:
72
  modal run scripts/coach_summary_modal.py --stage publish --repo-id build-small-hackathon/pozify-coach-summary
73
  ```
74
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  Run the full pipeline end to end:
76
 
77
  ```bash
@@ -83,22 +113,27 @@ modal run scripts/coach_summary_modal.py --stage all --epochs 2 --style-weight 0
83
  The Modal model volume stores:
84
 
85
  - `adapter/`
 
86
  - `training_config.json`
87
  - `training_summary.json`
88
  - `evaluation.json`
89
  - `hf_upload.json`
 
 
90
  - `README.md`
91
 
92
  ## Runtime Usage
93
 
94
- After publish, point Pozify runtime to the trained adapter by setting:
95
 
96
  ```bash
97
- export POZIFY_COACH_SUMMARY_ADAPTER_ID=build-small-hackathon/pozify-coach-summary
98
  ```
99
 
100
- If you download the adapter locally instead, use:
 
 
101
 
102
  ```bash
103
- export POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR=/path/to/adapter
104
  ```
 
13
  - `prepare-data`
14
  - `train`
15
  - `evaluate`
16
+ - `merge`
17
  - `publish`
18
+ - `publish-merged`
19
  - `all`
20
 
21
  ## Requirements
 
31
 
32
  - `HF_TOKEN`
33
  - optional `POZIFY_COACH_SUMMARY_HF_REPO_ID`
34
+ - optional `POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID`
35
+ - recommended `POZIFY_COACH_SUMMARY_MODEL`
36
  - optional `POZIFY_COACH_SUMMARY_HF_PRIVATE=1`
37
 
38
+ The script reads your local `.env` and injects those values into a Modal secret at runtime, so for local `modal run ...` usage you do not need to pre-create a hosted Modal secret just to run `prepare-data`, `train`, or `publish`.
39
 
40
  If you prefer a hosted secret for team/shared environments, you can still create one manually in Modal and adapt the script later.
41
 
 
76
  modal run scripts/coach_summary_modal.py --stage publish --repo-id build-small-hackathon/pozify-coach-summary
77
  ```
78
 
79
+ Merge the LoRA adapter into a full Transformers checkpoint:
80
+
81
+ ```bash
82
+ modal run scripts/coach_summary_modal.py --stage merge
83
+ ```
84
+
85
+ Publish the merged inference-ready model to Hugging Face:
86
+
87
+ ```bash
88
+ modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id build-small-hackathon/pozify-coach-summary
89
+ ```
90
+
91
+ If your Hugging Face token does not have permission to create or write under an organization
92
+ namespace, publish to a personal repo instead:
93
+
94
+ ```bash
95
+ modal run scripts/coach_summary_modal.py --stage publish-merged --repo-id <your-username>/pozify-coach-summary
96
+ ```
97
+
98
+ You can also set this once in `.env`:
99
+
100
+ ```bash
101
+ POZIFY_COACH_SUMMARY_MODEL=<your-username>/pozify-coach-summary
102
+ POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID=<your-username>/pozify-coach-summary
103
+ ```
104
+
105
  Run the full pipeline end to end:
106
 
107
  ```bash
 
113
  The Modal model volume stores:
114
 
115
  - `adapter/`
116
+ - `merged_model/`
117
  - `training_config.json`
118
  - `training_summary.json`
119
  - `evaluation.json`
120
  - `hf_upload.json`
121
+ - `merge_summary.json`
122
+ - `hf_merged_upload.json`
123
  - `README.md`
124
 
125
  ## Runtime Usage
126
 
127
+ For cloud inference in the current Pozify app, point runtime to the merged full model repo:
128
 
129
  ```bash
130
+ export POZIFY_COACH_SUMMARY_MODEL=build-small-hackathon/pozify-coach-summary
131
  ```
132
 
133
+ `POZIFY_COACH_SUMMARY_ADAPTER_ID` is no longer used as the remote inference target. Keep it only for adapter-centric workflows such as local merge/evaluation.
134
+
135
+ If you download the merged model locally instead, use:
136
 
137
  ```bash
138
+ export POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR=/path/to/merged_model
139
  ```
docs/demo-video-transcript.md ADDED
@@ -0,0 +1,247 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Pozify Demo Video Transcript
2
+
3
+ ## Demo Concept
4
+
5
+ `Pozify: The gym coach for people who would rather not be judged`
6
+
7
+ ## Characters
8
+
9
+ - `Linh`: wants to work out, but avoids the gym
10
+ - `Minh`: friend, funny but supportive
11
+ - Optional `Narrator`: can be one of the two, or a voiceover
12
+
13
+ ## Tone
14
+
15
+ - Light, relatable, slightly comedic
16
+ - Short lines, each line reveals the pain point
17
+ - Then pivot into a polished product demo
18
+
19
+ ## Total Length
20
+
21
+ - Around 2.5 to 3 minutes
22
+
23
+ ## Script
24
+
25
+ ### Scene 1: The Problem
26
+
27
+ `0:00 - 0:20`
28
+
29
+ Visual:
30
+ Linh stands in workout clothes at home, staring at a gym bag like it personally offended her.
31
+
32
+ Minh walks in.
33
+
34
+ Minh:
35
+ "You're not going?"
36
+
37
+ Linh:
38
+ "It's too far."
39
+
40
+ Beat.
41
+
42
+ Minh:
43
+ "That's not the real reason."
44
+
45
+ Linh:
46
+ "Fine. I don't want to be the girl doing squats wrong in front of twenty strangers and one guy filming his triceps."
47
+
48
+ Minh:
49
+ "Fair."
50
+
51
+ Linh:
52
+ "And a private trainer costs more than my groceries."
53
+
54
+ Small beat.
55
+
56
+ Minh:
57
+ "So your fitness plan is... emotional support and denial?"
58
+
59
+ Linh looks at the gym bag.
60
+
61
+ Linh:
62
+ "Currently, yes."
63
+
64
+ Why this works:
65
+
66
+ - `too far` = distance
67
+ - `doing squats wrong in front of strangers` = shame / fear of judgment
68
+ - `private trainer costs more than my groceries` = PT too expensive
69
+ - funny but very clear target customer
70
+
71
+ ### Scene 2: The Reframe
72
+
73
+ `0:20 - 0:35`
74
+
75
+ Visual:
76
+ Minh picks up Linh's phone.
77
+
78
+ Minh:
79
+ "What if you could train at home... and still get real feedback?"
80
+
81
+ Linh:
82
+ "From who? The front camera?"
83
+
84
+ Minh:
85
+ "From AI. But not the kind that just says 'great job' and disappears."
86
+
87
+ Cut to product UI.
88
+
89
+ Voiceover:
90
+ "That's exactly why we built Pozify."
91
+
92
+ ### Scene 3: Product Intro
93
+
94
+ `0:35 - 0:55`
95
+
96
+ Visual:
97
+ Screen recording of Pozify homepage and upload flow.
98
+
99
+ Voiceover:
100
+ "Pozify is an AI workout coach for people who want to train seriously, but don't want the cost, distance, or pressure of the gym. You upload a workout video, and Pozify turns it into a structured, explainable coaching report."
101
+
102
+ Visual:
103
+ Show progress steps:
104
+
105
+ - video quality
106
+ - pose tracking
107
+ - exercise classification
108
+ - rep counting
109
+ - issue detection
110
+ - coach summary
111
+
112
+ Voiceover:
113
+ "It doesn't just watch a video and guess. It checks video quality, tracks pose, identifies the exercise, counts reps, analyzes each rep, detects valid variations, flags real issues, and then generates a grounded coach summary."
114
+
115
+ ### Scene 4: The Magic
116
+
117
+ `0:55 - 1:30`
118
+
119
+ Visual:
120
+ Show real squat run in the app.
121
+ Open annotated video, issue clips, summary tab.
122
+
123
+ Voiceover:
124
+ "Here's the key difference: Pozify doesn't give generic advice like 'keep your chest up' and call it a day. It shows what exercise you did, how many reps you completed, which reps changed, what variation was valid, and what issue actually appeared."
125
+
126
+ Visual:
127
+ Highlight:
128
+
129
+ - `squat`
130
+ - rep count
131
+ - `wide_squat_stance`
132
+ - `shallow_depth`
133
+
134
+ Voiceover:
135
+ "In this example, Pozify recognizes a wide squat stance as a valid variation, not a mistake. But it still catches shallow depth on specific reps. So instead of overcorrecting everything, it separates style from actual form breakdown."
136
+
137
+ ### Scene 5: Why It's Trustworthy
138
+
139
+ `1:30 - 1:55`
140
+
141
+ Visual:
142
+ Show JSON tab or artifacts view briefly.
143
+ Show Coach tab with provider/model/source.
144
+
145
+ Voiceover:
146
+ "And we built guardrails into the system. The coach summary is generated from structured artifacts, not from vague intuition. Then a verifier checks whether the model mentioned issues outside the evidence, overcorrected a valid variation, or drifted into unsafe claims."
147
+
148
+ Visual:
149
+ Quick highlight:
150
+
151
+ - evidence artifacts
152
+ - verifier
153
+ - fallback logic
154
+
155
+ Voiceover:
156
+ "If the model gets too creative, Pozify falls back to a safer summary. So the user gets feedback they can actually trust."
157
+
158
+ ### Scene 6: Sponsor Tech
159
+
160
+ `1:55 - 2:15`
161
+
162
+ Visual:
163
+ Subtle overlays or callouts:
164
+
165
+ - Hugging Face Spaces
166
+ - ZeroGPU
167
+ - Hugging Face Inference
168
+ - Hugging Face Hub
169
+
170
+ Voiceover:
171
+ "We also leaned into the Hugging Face ecosystem to make this real. Pozify runs beautifully in Hugging Face Spaces, uses ZeroGPU for demo-ready compute, calls Hugging Face Inference for the coaching model, and loads our exercise router through Hugging Face Hub."
172
+
173
+ ### Scene 7: Return To Characters
174
+
175
+ `2:15 - 2:40`
176
+
177
+ Visual:
178
+ Back to Linh and Minh at home.
179
+ Linh finishes a set, checks the app, sees feedback.
180
+
181
+ Minh:
182
+ "So... no gym?"
183
+
184
+ Linh:
185
+ "No commute. No crowd. No fake confidence."
186
+
187
+ Minh:
188
+ "And no trainer bill?"
189
+
190
+ Linh:
191
+ "Exactly."
192
+
193
+ Beat.
194
+
195
+ Minh:
196
+ "So now the only thing judging you... is a grounded, evidence-based AI."
197
+
198
+ Linh:
199
+ "Which is somehow less terrifying than people."
200
+
201
+ They both laugh.
202
+
203
+ ### Scene 8: Closing Line
204
+
205
+ `2:40 - 2:55`
206
+
207
+ Visual:
208
+ Product hero shot, annotated video, summary, issue clips.
209
+
210
+ Voiceover:
211
+ "Pozify is for the people who want to get stronger, but not embarrassed. For the people who want real coaching, but not gym anxiety. For the people who don't need hype. They need feedback."
212
+
213
+ Final line on screen:
214
+ `Pozify: Your coach, without the crowd.`
215
+
216
+ ## Why This Version Is Strong
217
+
218
+ - The opening quickly locks onto the right customer:
219
+ - the gym is far away
220
+ - they feel uncomfortable in crowded spaces
221
+ - they are afraid of being judged for bad form
222
+ - one-on-one PT is too expensive
223
+ - The dialogue is short, punchy, and easy to perform
224
+ - It has a little humor without becoming silly
225
+ - It transitions naturally into the product demo
226
+ - It makes room to highlight sponsor tech
227
+ - It ends on user emotion, not just a feature list
228
+
229
+ ## Optional Funnier Variations
230
+
231
+ You can replace:
232
+
233
+ Minh:
234
+ "So your fitness plan is... emotional support and denial?"
235
+
236
+ with:
237
+
238
+ Minh:
239
+ "So your current workout split is... anxiety, excuses, and one YouTube warm-up?"
240
+
241
+ Or replace the ending with:
242
+
243
+ Minh:
244
+ "So now the only thing judging you is AI."
245
+
246
+ Linh:
247
+ "Perfect. AI doesn't smirk."
scripts/coach_summary_modal.py CHANGED
@@ -1,6 +1,7 @@
1
  from __future__ import annotations
2
 
3
  import json
 
4
  import os
5
  from pathlib import Path
6
  import shutil
@@ -11,9 +12,11 @@ import modal
11
 
12
 
13
  APP_NAME = "pozify-coach-summary"
14
- DEFAULT_HF_REPO_ID = "build-small-hackathon/pozify-coach-summary"
15
  HF_REPO_ID_ENV = "POZIFY_COACH_SUMMARY_HF_REPO_ID"
 
16
  HF_PRIVATE_ENV = "POZIFY_COACH_SUMMARY_HF_PRIVATE"
 
17
  DATA_ROOT = Path("/data")
18
  MODEL_ROOT = Path("/models")
19
  ROOT_DATA = Path("/root/data")
@@ -25,12 +28,17 @@ TRAINING_CONFIG_PATH = MODEL_ROOT / "training_config.json"
25
  TRAINING_SUMMARY_PATH = MODEL_ROOT / "training_summary.json"
26
  EVALUATION_PATH = MODEL_ROOT / "evaluation.json"
27
  HF_UPLOAD_PATH = MODEL_ROOT / "hf_upload.json"
 
 
28
  DEFAULT_ADAPTER_DIR = MODEL_ROOT / "adapter"
 
29
  HF_METADATA_FILENAMES = (
30
  "training_config.json",
31
  "training_summary.json",
32
  "evaluation.json",
33
  "hf_upload.json",
 
 
34
  )
35
  HF_DATA_FILENAMES = (
36
  "coach_summary_train.jsonl",
@@ -89,7 +97,13 @@ def _load_local_env_vars(filename: str = ".env") -> dict[str, str]:
89
  def _hf_secret() -> modal.Secret:
90
  env_values = _load_local_env_vars()
91
  secret_payload: dict[str, str] = {}
92
- for key in ("HF_TOKEN", HF_REPO_ID_ENV, HF_PRIVATE_ENV):
 
 
 
 
 
 
93
  value = os.getenv(key, env_values.get(key))
94
  if value is not None and str(value).strip():
95
  secret_payload[key] = str(value).strip()
@@ -135,6 +149,22 @@ def _env_truthy(value: str | None) -> bool:
135
  return value is not None and value.strip().lower() in {"1", "true", "yes", "on"}
136
 
137
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
  def _load_config() -> dict[str, Any]:
139
  config = _read_json(DEFAULT_CONFIG_PATH)
140
  if TRAINING_CONFIG_PATH.exists():
@@ -234,6 +264,29 @@ def _model_card_text(
234
  return "\n".join(lines)
235
 
236
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
237
  def _verifier_inputs_from_evidence(payload: dict[str, Any]) -> tuple[Any, Any, Any, Any, Any]:
238
  sys.path.insert(0, "/root/src")
239
  from pozify.contracts import (
@@ -448,27 +501,43 @@ def train(
448
  )
449
 
450
  adapter_dir = MODEL_ROOT / output_subdir
451
- trainer = SFTTrainer(
452
- model=model,
453
- tokenizer=tokenizer,
454
- train_dataset=train_dataset,
455
- eval_dataset=eval_dataset,
456
- peft_config=peft_config,
457
- dataset_text_field="text",
458
- max_seq_length=int(config["max_seq_length"]),
459
- args=SFTConfig(
460
- output_dir=str(adapter_dir),
461
- learning_rate=float(config["learning_rate"]),
462
- num_train_epochs=float(config["num_train_epochs"]),
463
- per_device_train_batch_size=int(config["per_device_train_batch_size"]),
464
- gradient_accumulation_steps=int(config["gradient_accumulation_steps"]),
465
- save_strategy="epoch",
466
- eval_strategy="epoch",
467
- logging_steps=10,
468
- bf16=True,
469
- report_to=[],
470
- ),
471
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
472
  train_result = trainer.train()
473
  trainer.save_model(str(adapter_dir))
474
  tokenizer.save_pretrained(str(adapter_dir))
@@ -640,6 +709,57 @@ def _upload_hf_file(
640
  }
641
 
642
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
643
  @app.function(
644
  volumes={str(MODEL_ROOT): model_volume, str(DATA_ROOT): data_volume},
645
  secrets=[_hf_secret()],
@@ -651,19 +771,21 @@ def publish_to_hf(
651
  adapter_subdir: str = "adapter",
652
  ) -> dict[str, Any]:
653
  from huggingface_hub import HfApi
 
654
 
655
- repo_id = repo_id or os.getenv(HF_REPO_ID_ENV) or DEFAULT_HF_REPO_ID
656
  private = _env_truthy(os.getenv(HF_PRIVATE_ENV)) if private is None else private
657
  if not os.getenv("HF_TOKEN"):
658
  return {
659
  "ok": False,
660
  "error": "HF_TOKEN is required in the Modal environment or local .env",
661
- "repo_id": repo_id,
662
  }
663
 
664
  config = _read_json(TRAINING_CONFIG_PATH) if TRAINING_CONFIG_PATH.exists() else {}
665
  training_summary = _read_json(TRAINING_SUMMARY_PATH) if TRAINING_SUMMARY_PATH.exists() else None
666
  evaluation = _read_json(EVALUATION_PATH) if EVALUATION_PATH.exists() else None
 
 
667
  MODEL_CARD_PATH.write_text(
668
  _model_card_text(
669
  repo_id=repo_id,
@@ -674,8 +796,23 @@ def publish_to_hf(
674
  encoding="utf-8",
675
  )
676
 
677
- api = HfApi()
678
- api.create_repo(repo_id=repo_id, repo_type="model", private=private, exist_ok=True)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
679
  uploads = [
680
  _upload_hf_file(
681
  api,
@@ -729,6 +866,132 @@ def publish_to_hf(
729
  return result
730
 
731
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
732
  @app.local_entrypoint()
733
  def main(
734
  stage: str = "evaluate",
@@ -744,14 +1007,19 @@ def main(
744
  print(train.remote(epochs=epochs, style_weight=style_weight))
745
  elif stage == "evaluate":
746
  print(evaluate.remote(limit=limit))
 
 
747
  elif stage == "publish":
748
  print(publish_to_hf.remote(repo_id=repo_id, private=private))
 
 
749
  elif stage == "all":
750
  print(prepare_data.remote())
751
  print(train.remote(epochs=epochs, style_weight=style_weight))
752
  print(evaluate.remote(limit=limit))
753
- print(publish_to_hf.remote(repo_id=repo_id, private=private))
 
754
  else:
755
  raise ValueError(
756
- "stage must be one of: prepare-data, train, evaluate, publish, all"
757
  )
 
1
  from __future__ import annotations
2
 
3
  import json
4
+ import inspect
5
  import os
6
  from pathlib import Path
7
  import shutil
 
12
 
13
 
14
  APP_NAME = "pozify-coach-summary"
15
+ DEFAULT_HF_REPO_NAME = "pozify-coach-summary"
16
  HF_REPO_ID_ENV = "POZIFY_COACH_SUMMARY_HF_REPO_ID"
17
+ HF_MERGED_REPO_ID_ENV = "POZIFY_COACH_SUMMARY_MERGED_HF_REPO_ID"
18
  HF_PRIVATE_ENV = "POZIFY_COACH_SUMMARY_HF_PRIVATE"
19
+ RUNTIME_MODEL_ENV = "POZIFY_COACH_SUMMARY_MODEL"
20
  DATA_ROOT = Path("/data")
21
  MODEL_ROOT = Path("/models")
22
  ROOT_DATA = Path("/root/data")
 
28
  TRAINING_SUMMARY_PATH = MODEL_ROOT / "training_summary.json"
29
  EVALUATION_PATH = MODEL_ROOT / "evaluation.json"
30
  HF_UPLOAD_PATH = MODEL_ROOT / "hf_upload.json"
31
+ MERGE_SUMMARY_PATH = MODEL_ROOT / "merge_summary.json"
32
+ HF_MERGED_UPLOAD_PATH = MODEL_ROOT / "hf_merged_upload.json"
33
  DEFAULT_ADAPTER_DIR = MODEL_ROOT / "adapter"
34
+ DEFAULT_MERGED_DIR = MODEL_ROOT / "merged_model"
35
  HF_METADATA_FILENAMES = (
36
  "training_config.json",
37
  "training_summary.json",
38
  "evaluation.json",
39
  "hf_upload.json",
40
+ "merge_summary.json",
41
+ "hf_merged_upload.json",
42
  )
43
  HF_DATA_FILENAMES = (
44
  "coach_summary_train.jsonl",
 
97
  def _hf_secret() -> modal.Secret:
98
  env_values = _load_local_env_vars()
99
  secret_payload: dict[str, str] = {}
100
+ for key in (
101
+ "HF_TOKEN",
102
+ HF_REPO_ID_ENV,
103
+ HF_MERGED_REPO_ID_ENV,
104
+ HF_PRIVATE_ENV,
105
+ RUNTIME_MODEL_ENV,
106
+ ):
107
  value = os.getenv(key, env_values.get(key))
108
  if value is not None and str(value).strip():
109
  secret_payload[key] = str(value).strip()
 
149
  return value is not None and value.strip().lower() in {"1", "true", "yes", "on"}
150
 
151
 
152
+ def _supports_kwarg(callable_obj: Any, name: str) -> bool:
153
+ try:
154
+ parameters = inspect.signature(callable_obj).parameters
155
+ except (TypeError, ValueError):
156
+ return False
157
+ return name in parameters
158
+
159
+
160
+ def _filtered_kwargs(callable_obj: Any, kwargs: dict[str, Any]) -> dict[str, Any]:
161
+ try:
162
+ parameters = inspect.signature(callable_obj).parameters
163
+ except (TypeError, ValueError):
164
+ return dict(kwargs)
165
+ return {key: value for key, value in kwargs.items() if key in parameters}
166
+
167
+
168
  def _load_config() -> dict[str, Any]:
169
  config = _read_json(DEFAULT_CONFIG_PATH)
170
  if TRAINING_CONFIG_PATH.exists():
 
264
  return "\n".join(lines)
265
 
266
 
267
+ def _resolve_repo_id(
268
+ api: Any,
269
+ repo_id: str | None,
270
+ *,
271
+ env_names: tuple[str, ...] = (HF_REPO_ID_ENV,),
272
+ ) -> str:
273
+ if repo_id:
274
+ return repo_id
275
+ for env_name in env_names:
276
+ configured = os.getenv(env_name)
277
+ if configured:
278
+ return configured
279
+ try:
280
+ whoami = api.whoami()
281
+ if isinstance(whoami, dict):
282
+ username = whoami.get("name") or whoami.get("fullname")
283
+ if isinstance(username, str) and username.strip():
284
+ return f"{username.strip()}/{DEFAULT_HF_REPO_NAME}"
285
+ except Exception:
286
+ pass
287
+ return DEFAULT_HF_REPO_NAME
288
+
289
+
290
  def _verifier_inputs_from_evidence(payload: dict[str, Any]) -> tuple[Any, Any, Any, Any, Any]:
291
  sys.path.insert(0, "/root/src")
292
  from pozify.contracts import (
 
501
  )
502
 
503
  adapter_dir = MODEL_ROOT / output_subdir
504
+ sft_config_kwargs = {
505
+ "output_dir": str(adapter_dir),
506
+ "learning_rate": float(config["learning_rate"]),
507
+ "num_train_epochs": float(config["num_train_epochs"]),
508
+ "per_device_train_batch_size": int(config["per_device_train_batch_size"]),
509
+ "gradient_accumulation_steps": int(config["gradient_accumulation_steps"]),
510
+ "save_strategy": "epoch",
511
+ "logging_steps": 10,
512
+ "bf16": True,
513
+ "report_to": [],
514
+ }
515
+ if _supports_kwarg(SFTConfig.__init__, "eval_strategy"):
516
+ sft_config_kwargs["eval_strategy"] = "epoch"
517
+ elif _supports_kwarg(SFTConfig.__init__, "evaluation_strategy"):
518
+ sft_config_kwargs["evaluation_strategy"] = "epoch"
519
+ if _supports_kwarg(SFTConfig.__init__, "max_seq_length"):
520
+ sft_config_kwargs["max_seq_length"] = int(config["max_seq_length"])
521
+
522
+ trainer_kwargs: dict[str, Any] = {
523
+ "model": model,
524
+ "train_dataset": train_dataset,
525
+ "eval_dataset": eval_dataset,
526
+ "peft_config": peft_config,
527
+ "args": SFTConfig(**_filtered_kwargs(SFTConfig.__init__, sft_config_kwargs)),
528
+ }
529
+ if _supports_kwarg(SFTTrainer.__init__, "processing_class"):
530
+ trainer_kwargs["processing_class"] = tokenizer
531
+ elif _supports_kwarg(SFTTrainer.__init__, "tokenizer"):
532
+ trainer_kwargs["tokenizer"] = tokenizer
533
+ if _supports_kwarg(SFTTrainer.__init__, "dataset_text_field"):
534
+ trainer_kwargs["dataset_text_field"] = "text"
535
+ elif _supports_kwarg(SFTTrainer.__init__, "formatting_func"):
536
+ trainer_kwargs["formatting_func"] = lambda example: example["text"]
537
+ if _supports_kwarg(SFTTrainer.__init__, "max_seq_length"):
538
+ trainer_kwargs["max_seq_length"] = int(config["max_seq_length"])
539
+
540
+ trainer = SFTTrainer(**_filtered_kwargs(SFTTrainer.__init__, trainer_kwargs))
541
  train_result = trainer.train()
542
  trainer.save_model(str(adapter_dir))
543
  tokenizer.save_pretrained(str(adapter_dir))
 
709
  }
710
 
711
 
712
+ @app.function(
713
+ gpu="A10G",
714
+ volumes={str(MODEL_ROOT): model_volume},
715
+ secrets=[_hf_secret()],
716
+ timeout=90 * 60,
717
+ )
718
+ def merge(
719
+ adapter_subdir: str = "adapter",
720
+ merged_subdir: str = "merged_model",
721
+ ) -> dict[str, Any]:
722
+ import torch
723
+ from peft import PeftModel
724
+ from transformers import AutoModelForCausalLM, AutoTokenizer
725
+
726
+ config = _load_config()
727
+ adapter_dir = MODEL_ROOT / adapter_subdir
728
+ merged_dir = MODEL_ROOT / merged_subdir
729
+ if not adapter_dir.exists():
730
+ result = {"ok": False, "error": f"Adapter dir not found: {adapter_dir}"}
731
+ _write_json(MERGE_SUMMARY_PATH, result)
732
+ model_volume.commit()
733
+ return result
734
+
735
+ if merged_dir.exists():
736
+ shutil.rmtree(merged_dir)
737
+ merged_dir.mkdir(parents=True, exist_ok=True)
738
+
739
+ tokenizer = AutoTokenizer.from_pretrained(str(adapter_dir))
740
+ base_model = AutoModelForCausalLM.from_pretrained(
741
+ str(config["base_model"]),
742
+ torch_dtype=torch.bfloat16,
743
+ device_map="auto",
744
+ low_cpu_mem_usage=True,
745
+ )
746
+ model = PeftModel.from_pretrained(base_model, str(adapter_dir))
747
+ merged_model = model.merge_and_unload()
748
+ merged_model.save_pretrained(str(merged_dir), safe_serialization=True)
749
+ tokenizer.save_pretrained(str(merged_dir))
750
+
751
+ result = {
752
+ "ok": True,
753
+ "base_model": config["base_model"],
754
+ "adapter_dir": str(adapter_dir),
755
+ "merged_dir": str(merged_dir),
756
+ "dtype": "bfloat16",
757
+ }
758
+ _write_json(MERGE_SUMMARY_PATH, result)
759
+ model_volume.commit()
760
+ return result
761
+
762
+
763
  @app.function(
764
  volumes={str(MODEL_ROOT): model_volume, str(DATA_ROOT): data_volume},
765
  secrets=[_hf_secret()],
 
771
  adapter_subdir: str = "adapter",
772
  ) -> dict[str, Any]:
773
  from huggingface_hub import HfApi
774
+ from huggingface_hub.errors import HfHubHTTPError
775
 
 
776
  private = _env_truthy(os.getenv(HF_PRIVATE_ENV)) if private is None else private
777
  if not os.getenv("HF_TOKEN"):
778
  return {
779
  "ok": False,
780
  "error": "HF_TOKEN is required in the Modal environment or local .env",
781
+ "repo_id": repo_id or os.getenv(HF_REPO_ID_ENV) or DEFAULT_HF_REPO_NAME,
782
  }
783
 
784
  config = _read_json(TRAINING_CONFIG_PATH) if TRAINING_CONFIG_PATH.exists() else {}
785
  training_summary = _read_json(TRAINING_SUMMARY_PATH) if TRAINING_SUMMARY_PATH.exists() else None
786
  evaluation = _read_json(EVALUATION_PATH) if EVALUATION_PATH.exists() else None
787
+ api = HfApi()
788
+ repo_id = _resolve_repo_id(api, repo_id, env_names=(HF_REPO_ID_ENV,))
789
  MODEL_CARD_PATH.write_text(
790
  _model_card_text(
791
  repo_id=repo_id,
 
796
  encoding="utf-8",
797
  )
798
 
799
+ try:
800
+ api.create_repo(repo_id=repo_id, repo_type="model", private=private, exist_ok=True)
801
+ except HfHubHTTPError as exc:
802
+ message = str(exc)
803
+ guidance = (
804
+ "Publish failed while creating or accessing the Hugging Face model repo. "
805
+ "If your token does not have org-level write access, publish to a personal repo id "
806
+ "such as `<your-username>/pozify-coach-summary`, or set "
807
+ f"`{HF_REPO_ID_ENV}` in `.env` to a repo you control."
808
+ )
809
+ return {
810
+ "ok": False,
811
+ "repo_id": repo_id,
812
+ "private": private,
813
+ "error": message,
814
+ "guidance": guidance,
815
+ }
816
  uploads = [
817
  _upload_hf_file(
818
  api,
 
866
  return result
867
 
868
 
869
+ @app.function(
870
+ volumes={str(MODEL_ROOT): model_volume, str(DATA_ROOT): data_volume},
871
+ secrets=[_hf_secret()],
872
+ timeout=60 * 60,
873
+ )
874
+ def publish_merged_to_hf(
875
+ repo_id: str | None = None,
876
+ private: bool | None = None,
877
+ merged_subdir: str = "merged_model",
878
+ ) -> dict[str, Any]:
879
+ from huggingface_hub import HfApi
880
+ from huggingface_hub.errors import HfHubHTTPError
881
+
882
+ private = _env_truthy(os.getenv(HF_PRIVATE_ENV)) if private is None else private
883
+ resolved_repo_hint = (
884
+ repo_id
885
+ or os.getenv(RUNTIME_MODEL_ENV)
886
+ or os.getenv(HF_MERGED_REPO_ID_ENV)
887
+ or DEFAULT_HF_REPO_NAME
888
+ )
889
+ if not os.getenv("HF_TOKEN"):
890
+ return {
891
+ "ok": False,
892
+ "error": "HF_TOKEN is required in the Modal environment or local .env",
893
+ "repo_id": resolved_repo_hint,
894
+ }
895
+
896
+ config = _read_json(TRAINING_CONFIG_PATH) if TRAINING_CONFIG_PATH.exists() else {}
897
+ training_summary = _read_json(TRAINING_SUMMARY_PATH) if TRAINING_SUMMARY_PATH.exists() else None
898
+ evaluation = _read_json(EVALUATION_PATH) if EVALUATION_PATH.exists() else None
899
+ merge_summary = _read_json(MERGE_SUMMARY_PATH) if MERGE_SUMMARY_PATH.exists() else None
900
+ merged_dir = MODEL_ROOT / merged_subdir
901
+ if not merged_dir.exists():
902
+ result = {
903
+ "ok": False,
904
+ "error": f"Merged model dir not found: {merged_dir}",
905
+ "repo_id": resolved_repo_hint,
906
+ }
907
+ _write_json(HF_MERGED_UPLOAD_PATH, result)
908
+ model_volume.commit()
909
+ return result
910
+
911
+ api = HfApi()
912
+ repo_id = _resolve_repo_id(
913
+ api,
914
+ repo_id,
915
+ env_names=(RUNTIME_MODEL_ENV, HF_MERGED_REPO_ID_ENV, HF_REPO_ID_ENV),
916
+ )
917
+
918
+ MODEL_CARD_PATH.write_text(
919
+ _model_card_text(
920
+ repo_id=repo_id,
921
+ config=config,
922
+ training_summary=training_summary,
923
+ evaluation=evaluation,
924
+ )
925
+ + "\n## Packaging\n\n- Published as a merged, inference-ready Transformers checkpoint.\n",
926
+ encoding="utf-8",
927
+ )
928
+
929
+ try:
930
+ api.create_repo(repo_id=repo_id, repo_type="model", private=private, exist_ok=True)
931
+ except HfHubHTTPError as exc:
932
+ message = str(exc)
933
+ guidance = (
934
+ "Publish failed while creating or accessing the merged Hugging Face model repo. "
935
+ "Set `POZIFY_COACH_SUMMARY_MODEL` or pass `--repo-id <your-username>/pozify-coach-summary` "
936
+ "to publish to a repo your token can write to."
937
+ )
938
+ result = {
939
+ "ok": False,
940
+ "repo_id": repo_id,
941
+ "private": private,
942
+ "error": message,
943
+ "guidance": guidance,
944
+ }
945
+ _write_json(HF_MERGED_UPLOAD_PATH, result)
946
+ model_volume.commit()
947
+ return result
948
+
949
+ api.upload_folder(
950
+ repo_id=repo_id,
951
+ repo_type="model",
952
+ folder_path=str(merged_dir),
953
+ )
954
+
955
+ uploads = [
956
+ {
957
+ "path": str(merged_dir),
958
+ "path_in_repo": "./",
959
+ "uploaded": True,
960
+ },
961
+ _upload_hf_file(
962
+ api,
963
+ repo_id=repo_id,
964
+ local_path=MODEL_CARD_PATH,
965
+ path_in_repo="README.md",
966
+ ),
967
+ ]
968
+ uploads.extend(
969
+ _upload_hf_file(
970
+ api,
971
+ repo_id=repo_id,
972
+ local_path=MODEL_ROOT / filename,
973
+ path_in_repo=filename,
974
+ )
975
+ for filename in (
976
+ "training_config.json",
977
+ "training_summary.json",
978
+ "evaluation.json",
979
+ "merge_summary.json",
980
+ )
981
+ )
982
+
983
+ result = {
984
+ "ok": True,
985
+ "repo_id": repo_id,
986
+ "private": private,
987
+ "merge_summary": merge_summary,
988
+ "uploads": uploads,
989
+ }
990
+ _write_json(HF_MERGED_UPLOAD_PATH, result)
991
+ model_volume.commit()
992
+ return result
993
+
994
+
995
  @app.local_entrypoint()
996
  def main(
997
  stage: str = "evaluate",
 
1007
  print(train.remote(epochs=epochs, style_weight=style_weight))
1008
  elif stage == "evaluate":
1009
  print(evaluate.remote(limit=limit))
1010
+ elif stage == "merge":
1011
+ print(merge.remote())
1012
  elif stage == "publish":
1013
  print(publish_to_hf.remote(repo_id=repo_id, private=private))
1014
+ elif stage == "publish-merged":
1015
+ print(publish_merged_to_hf.remote(repo_id=repo_id, private=private))
1016
  elif stage == "all":
1017
  print(prepare_data.remote())
1018
  print(train.remote(epochs=epochs, style_weight=style_weight))
1019
  print(evaluate.remote(limit=limit))
1020
+ print(merge.remote())
1021
+ print(publish_merged_to_hf.remote(repo_id=repo_id, private=private))
1022
  else:
1023
  raise ValueError(
1024
+ "stage must be one of: prepare-data, train, evaluate, merge, publish, publish-merged, all"
1025
  )
scripts/train_coach_summary_lora.py CHANGED
@@ -11,7 +11,7 @@ sys.path.insert(0, str(ROOT / "src"))
11
 
12
 
13
  DEFAULT_CONFIG = {
14
- "base_model": "Qwen/Qwen2.5-7B-Instruct",
15
  "train_file": str(ROOT / "data/sft/coach_summary_train.jsonl"),
16
  "eval_file": str(ROOT / "data/sft/coach_summary_eval.jsonl"),
17
  "output_dir": str(ROOT / "models/coach_summary_lora"),
 
11
 
12
 
13
  DEFAULT_CONFIG = {
14
+ "base_model": "Qwen/Qwen3-14B",
15
  "train_file": str(ROOT / "data/sft/coach_summary_train.jsonl"),
16
  "eval_file": str(ROOT / "data/sft/coach_summary_eval.jsonl"),
17
  "output_dir": str(ROOT / "models/coach_summary_lora"),
tests/test_coach_summary.py CHANGED
@@ -56,7 +56,7 @@ class _GoodModel:
56
  '"confidence_notes":["Confidence is limited."]}'
57
  ),
58
  provider="hf_inference",
59
- model="Qwen/Qwen2.5-7B-Instruct",
60
  )
61
 
62
 
@@ -159,7 +159,7 @@ class CoachSummaryTests(unittest.TestCase):
159
  with tempfile.TemporaryDirectory() as temp_dir:
160
  env_path = Path(temp_dir) / ".env"
161
  env_path.write_text(
162
- "HF_TOKEN=test-token\nPOZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen2.5-7B-Instruct\n",
163
  encoding="utf-8",
164
  )
165
  with patch.dict(os.environ, {}, clear=True):
@@ -173,7 +173,7 @@ class CoachSummaryTests(unittest.TestCase):
173
  self.assertEqual(os.getenv("HF_TOKEN"), "test-token")
174
  self.assertEqual(
175
  os.getenv("POZIFY_COACH_SUMMARY_MODEL"),
176
- "Qwen/Qwen2.5-7B-Instruct",
177
  )
178
 
179
  def test_get_coach_summary_model_can_use_local_transformers_provider(self) -> None:
@@ -414,7 +414,7 @@ class CoachSummaryTests(unittest.TestCase):
414
  )
415
 
416
  self.assertEqual(result.provider, "hf_inference")
417
- self.assertEqual(result.model, "Qwen/Qwen2.5-7B-Instruct")
418
  self.assertEqual(result.source, "model_or_local")
419
 
420
  def test_verifier_rejects_issue_not_in_json(self) -> None:
 
56
  '"confidence_notes":["Confidence is limited."]}'
57
  ),
58
  provider="hf_inference",
59
+ model="Qwen/Qwen3-14B",
60
  )
61
 
62
 
 
159
  with tempfile.TemporaryDirectory() as temp_dir:
160
  env_path = Path(temp_dir) / ".env"
161
  env_path.write_text(
162
+ "HF_TOKEN=test-token\nPOZIFY_COACH_SUMMARY_MODEL=Qwen/Qwen3-14B\n",
163
  encoding="utf-8",
164
  )
165
  with patch.dict(os.environ, {}, clear=True):
 
173
  self.assertEqual(os.getenv("HF_TOKEN"), "test-token")
174
  self.assertEqual(
175
  os.getenv("POZIFY_COACH_SUMMARY_MODEL"),
176
+ "Qwen/Qwen3-14B",
177
  )
178
 
179
  def test_get_coach_summary_model_can_use_local_transformers_provider(self) -> None:
 
414
  )
415
 
416
  self.assertEqual(result.provider, "hf_inference")
417
+ self.assertEqual(result.model, "Qwen/Qwen3-14B")
418
  self.assertEqual(result.source, "model_or_local")
419
 
420
  def test_verifier_rejects_issue_not_in_json(self) -> None:
tests/test_pipeline_contracts.py CHANGED
@@ -359,7 +359,7 @@ class PipelineContractTests(unittest.TestCase):
359
  return_value=CoachSummaryResult(
360
  summary=model_summary,
361
  provider="hf_inference",
362
- model="Qwen/Qwen2.5-7B-Instruct",
363
  source="model_or_local",
364
  ),
365
  ),
@@ -389,7 +389,7 @@ class PipelineContractTests(unittest.TestCase):
389
  self.assertEqual(report["artifacts"]["coach_summary_provider"], "hf_inference")
390
  self.assertEqual(
391
  report["artifacts"]["coach_summary_model"],
392
- "Qwen/Qwen2.5-7B-Instruct",
393
  )
394
  self.assertTrue(report["artifacts"]["coach_summary_verifier_bypassed"])
395
  self.assertFalse(report["verification"]["passed"])
 
359
  return_value=CoachSummaryResult(
360
  summary=model_summary,
361
  provider="hf_inference",
362
+ model="Qwen/Qwen3-14B",
363
  source="model_or_local",
364
  ),
365
  ),
 
389
  self.assertEqual(report["artifacts"]["coach_summary_provider"], "hf_inference")
390
  self.assertEqual(
391
  report["artifacts"]["coach_summary_model"],
392
+ "Qwen/Qwen3-14B",
393
  )
394
  self.assertTrue(report["artifacts"]["coach_summary_verifier_bypassed"])
395
  self.assertFalse(report["verification"]["passed"])
tests/test_slm_providers.py CHANGED
@@ -21,7 +21,7 @@ class SlmProviderTests(unittest.TestCase):
21
  os.environ,
22
  {
23
  "POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR": "/tmp/local-model",
24
- "POZIFY_COACH_SUMMARY_BASE_MODEL": "Qwen/Qwen2.5-7B-Instruct",
25
  "POZIFY_COACH_SUMMARY_ADAPTER_ID": "pozify/coach-summary-lora",
26
  },
27
  clear=True,
@@ -34,7 +34,7 @@ class SlmProviderTests(unittest.TestCase):
34
  with patch.dict(
35
  os.environ,
36
  {
37
- "POZIFY_COACH_SUMMARY_MODEL": "Qwen/Qwen2.5-7B-Instruct",
38
  },
39
  clear=True,
40
  ):
@@ -42,10 +42,11 @@ class SlmProviderTests(unittest.TestCase):
42
 
43
  self.assertIsInstance(model, HFInferenceCoachSummaryModel)
44
 
45
- def test_remote_model_prefers_adapter_repo_when_present(self) -> None:
46
  with patch.dict(
47
  os.environ,
48
  {
 
49
  "POZIFY_COACH_SUMMARY_ADAPTER_ID": "pozify/coach-summary-lora",
50
  },
51
  clear=True,
@@ -53,7 +54,20 @@ class SlmProviderTests(unittest.TestCase):
53
  model = get_coach_summary_model()
54
 
55
  self.assertIsInstance(model, HFInferenceCoachSummaryModel)
56
- self.assertEqual(model.model, "pozify/coach-summary-lora")
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
 
59
  if __name__ == "__main__":
 
21
  os.environ,
22
  {
23
  "POZIFY_COACH_SUMMARY_LOCAL_MODEL_DIR": "/tmp/local-model",
24
+ "POZIFY_COACH_SUMMARY_BASE_MODEL": "Qwen/Qwen3-14B",
25
  "POZIFY_COACH_SUMMARY_ADAPTER_ID": "pozify/coach-summary-lora",
26
  },
27
  clear=True,
 
34
  with patch.dict(
35
  os.environ,
36
  {
37
+ "POZIFY_COACH_SUMMARY_MODEL": "Qwen/Qwen3-14B",
38
  },
39
  clear=True,
40
  ):
 
42
 
43
  self.assertIsInstance(model, HFInferenceCoachSummaryModel)
44
 
45
+ def test_remote_model_uses_runtime_model_not_adapter_repo(self) -> None:
46
  with patch.dict(
47
  os.environ,
48
  {
49
+ "POZIFY_COACH_SUMMARY_MODEL": "build-small-hackathon/pozify-coach-summary",
50
  "POZIFY_COACH_SUMMARY_ADAPTER_ID": "pozify/coach-summary-lora",
51
  },
52
  clear=True,
 
54
  model = get_coach_summary_model()
55
 
56
  self.assertIsInstance(model, HFInferenceCoachSummaryModel)
57
+ self.assertEqual(model.model, "build-small-hackathon/pozify-coach-summary")
58
+
59
+ def test_remote_model_falls_back_to_default_when_only_adapter_repo_is_set(self) -> None:
60
+ with patch.dict(
61
+ os.environ,
62
+ {
63
+ "POZIFY_COACH_SUMMARY_ADAPTER_ID": "pozify/coach-summary-lora",
64
+ },
65
+ clear=True,
66
+ ):
67
+ model = get_coach_summary_model()
68
+
69
+ self.assertIsInstance(model, HFInferenceCoachSummaryModel)
70
+ self.assertEqual(model.model, "Qwen/Qwen3-14B")
71
 
72
 
73
  if __name__ == "__main__":