DOPER_HOPE — RTMPose keypoint models for HOPE objects

Trained on 40k synthetic images per object (Blender EEVEE, uniform SO(3) pose, 5–8 distractors). Evaluated on the real HOPE BOP val split using PnP pose estimation.

Pipeline

1. Build .blend from BOP PLY (mm → m conversion)
2. Render 40k synthetic images per object (32 workers, Blender 5.1 EEVEE)
3. Merge worker annotations, 95/5 train/val split
4. Train RTMPose-t, 300 epochs, 8 GPUs
5. Eval: RTMPose inference → PnP → BOP metrics on real HOPE val

Repository structure

obj_000001/
  best_coco_AP_epoch_NNN.pth   ← best checkpoint
  obj_000001.blend             ← Blender scene with keypoint empties
  obj_000001.png               ← UV texture
  obj_000001_keypoints_3d.json ← 3D keypoint positions (meters)
  real_val_grid.jpg            ← 5×5 PnP visualisation on real val images
  synth_val_grid.jpg           ← 5×5 sample of synthetic val images
  bop_summary.json             ← per-object BOP metrics
...

BOP Metrics — HOPE val split

ID	Object	ADD-AUC	ADDS-AUC	ADD<10%	ADDS<10%	Best epoch
1	AlphabetSoup	0.4607	0.5672	0.111	0.222	200
2	BBQSauce	0.4783	0.5977	0.178	0.356	300
3	Butter	0.3786	0.4348	0.244	0.311	90
4	Cherries	0.5712	0.7205	0.150	0.450	260
5	ChocolatePudding	0.6897	0.7764	0.440	0.560	220
6	Cookies	0.8272	0.9123	0.200	0.600	300
7	Corn	0.3038	0.4866	0.130	0.217	140
8	CreamCheese	0.3328	0.4646	0.040	0.160	30
9	GranolaBars	0.4354	0.5482	0.200	0.320	120
10	GreenBeans	0.2668	0.3570	0.125	0.281	80
11	Ketchup	0.5817	0.7010	0.350	0.575	80
12	Macaroni	0.4228	0.6292	0.180	0.420	180
13	Mayo	0.4735	0.5996	0.167	0.233	230
14	Milk	0.7333	0.8359	0.160	0.440	100
15	Mushrooms	0.1255	0.2801	0.000	0.030	90
16	Mustard	0.6703	0.7290	0.340	0.540	200
17	OrangeJuice	0.5128	0.5882	0.235	0.412	230
18	Parmesan	0.6658	0.7573	0.333	0.567	210
19	Peaches	0.5019	0.6263	0.150	0.250	70
20	PeasAndCarrots	0.1679	0.2462	0.100	0.100	100
21	Pineapple	0.4367	0.5948	0.167	0.396	270
22	Popcorn	0.3282	0.4506	0.050	0.150	290
23	Raisins	0.3327	0.5173	0.075	0.225	180
24	SaladDressing	0.4628	0.4756	0.400	0.400	260
25	Spaghetti	0.7319	0.7985	0.486	0.571	220
26	TomatoSauce	0.4514	0.6269	0.241	0.345	290
27	Tuna	0.2392	0.3377	0.000	0.057	60
28	Yogurt	0.6595	0.8333	0.167	0.278	190

Mean (28/28): ADD-AUC = 0.4722 | ADDS-AUC = 0.5924

Notes

• obj_000003 (Butter): previously failed (ADD-AUC = 0.0). Fixed by regenerating keypoints as 8 symmetric pairs + 1 YZ-plane anchor to handle the object's YZ-plane reflection symmetry. Retrained from scratch; ADD-AUC improved from 0.000 → 0.379.
• obj_000022 (Popcorn): previously ADD-AUC = 0.085. Fixed by regenerating keypoints as 8 symmetric pairs + 1 Z-axis anchor to handle the object's 180-deg rotational symmetry around Z. Retrained from scratch; ADD-AUC improved from 0.085 → 0.328, ADDS-AUC from 0.163 → 0.451.
• All models use RTMPose-t architecture.
• Synthetic data: Blender 5.1 EEVEE, HDRI lighting, random CCO textures, 5–8 USD distractor objects.
• BOP PLY meshes are in mm; converted to meters for Blender. Evaluation uses original mm scale.