792 MB
130 files
Updated 1 day ago
Name
Size
moge_depth
sheets
thumbs
videos
README.md10.4 kB
xet
index.csv12.6 kB
xet
manifest.json29.9 kB
xet
README.md

DROID coverage sample (56 episodes)

Chosen to cover DROID's axes of variation, not sampled uniformly. A uniform draw returns mostly TRI-lab pick-and-place of rigid objects; this spreads over the properties that change what a geometry pipeline has to do.

Only camera ext1 is included. Every episode here is also covered by PointWorld's camera set, so annotations line up if you go further.

Two properties of DROID worth knowing before you look

  1. current_task is usually a category template, not a description. Strings like "Move object into or out of container (ex: drawer, clothes hamper, plate, trashcan, washer)" or "Do any task, and then reset the scene." are the norm. Object identity has to come from the separate human-annotation file (aggregated-annotations-030724.json, 3 instructions per episode).
  2. The human annotations cover successful episodes only. Measured over 1,153 episodes' metadata: all 832 with annotations are successes, all 125 failures have none. So any object-based filter silently selects for success. The unknown_no_annotation rows below are failure episodes drawn deliberately, and their object identity is genuinely unknown.

Object physics class

The axis that decides whether one rigid SE(3) pose per frame is even the right model.

class n
articulated 7
deformable 7
flat_thin 7
organic 6
rigid_compact 7
rigid_symmetric 6
rigid_thin 6
transparent 2
unknown_no_annotation 8

Task template

class n
clean_with_cloth 5
concrete 8
fold 5
free_form 6
hang 4
into_container 7
lid_on_off 7
open_close 5
press_button 4
reposition 5

Lab

lab n
AUTOLab 6
CLVR 7
GuptaLab 6
ILIAD 6
IPRL 7
IRIS 8
PennPAL 4
RAD 4
RAIL 5
REAL 3

Outcome

success n
False 8
True 48

Episode length (frames)

min 117 / median 307 / max 2449

Files

  • index.csv -- one row per episode
  • manifest.json -- the picker's raw output, including all 3 human instructions
  • thumbs/<uuid>.jpg -- 3-frame strip per episode (5% / 45% / 90%)
  • sheets/<object_class>.jpg -- all episodes of one class stacked
  • videos/<uuid>.mp4 -- the ext1 clip

Episodes

uuid object class task lab frames ok instruction
CLVR+13759f6e+2023-06-03-23h-29m-55s articulated press_button CLVR 207 y Close the lid on the silver flask
CLVR+13759f6e+2023-06-03-23h-27m-33s articulated press_button CLVR 140 y Close the lid on the top of the thermos flask
CLVR+13759f6e+2023-06-03-00h-10m-07s articulated press_button CLVR 132 y Open the lid at the top of the thermos flask
GuptaLab+553d1bd5+2023-05-28-15h-35m-41s articulated open_close GuptaLab 227 y Open the bottom drawer of the cabinet to the right.
PennPAL+06b0ffa5+2023-04-29-20h-25m-37s articulated open_close PennPAL 117 y Close the drawer
RAD+62828209+2023-09-09-07h-22m-33s articulated press_button RAD 175 y Press the button on the microwave
RAIL+80edfcb1+2023-06-29-12h-44m-09s articulated lid_on_off RAIL 428 y Pick the lid and put it on the pot
CLVR+236539bc+2023-05-29-21h-48m-38s deformable hang CLVR 225 y Put the cloth on the stand
ILIAD+50aee79f+2023-08-17-21h-00m-34s deformable fold ILIAD 961 y Turn the towel over unto the left
ILIAD+50aee79f+2023-08-17-21h-23m-04s deformable fold ILIAD 637 y Unfold the grey towel.
IRIS+226b9f35+2023-07-11-14h-00m-58s deformable fold IRIS 600 y Fold the towel
PennPAL+acda9df3+2023-07-07-20h-24m-56s deformable hang PennPAL 254 y Unhang the light yellow cloth from the backrest of the chair
PennPAL+acda9df3+2023-07-07-20h-09m-22s deformable fold PennPAL 393 y Fold the thing on the table
RAIL+80edfcb1+2023-04-26-15h-45m-39s deformable clean_with_cloth RAIL 191 y Move the towel
AUTOLab+5d05c5aa+2023-11-12-17h-00m-03s flat_thin concrete AUTOLab 1468 y Pick up all the objects surrounding the white plate and arrange them o
CLVR+13759f6e+2023-06-19-17h-10m-03s flat_thin into_container CLVR 333 y Remove the objects from the tray
ILIAD+5e938e3b+2023-07-20-10h-24m-39s flat_thin open_close ILIAD 296 y Pull and push the pullout table
IRIS+226b9f35+2023-07-11-22h-20m-42s flat_thin lid_on_off IRIS 317 y Pick up the slice of bread from the left side of the counter and place
RAD+62828209+2023-11-26-13h-45m-02s flat_thin reposition RAD 1347 y Take out two books from the right side of the second shelf on the righ
RAIL+80edfcb1+2023-04-19-15h-12m-55s flat_thin free_form RAIL 173 y Put the white and orange object onto the orange plate
REAL+4dbb5646+2023-05-17-11h-19m-38s flat_thin concrete REAL 370 y Open the notebook on the tray
AUTOLab+84bd5053+2023-07-08-09h-31m-28s organic into_container AUTOLab 535 y Put the orange, yellow and green blocks in the gray pot one at a time
IPRL+2fe9715d+2023-08-24-22h-22m-32s organic fold IPRL 394 y Place a slice of bread in the toaster and switch it on
IPRL+w026bb9b+2023-09-22-23h-48m-36s organic lid_on_off IPRL 187 y Put two slices of bread in the toaster
IRIS+226b9f35+2023-07-11-23h-59m-48s organic hang IRIS 366 y Put the orange cap on the table
IRIS+226b9f35+2023-07-11-23h-56m-26s organic hang IRIS 301 y Put the red cap on top of the orange cap
PennPAL+acda9df3+2023-06-15-17h-55m-20s organic free_form PennPAL 263 y Pick up the banana plush toy and put it on top of the toaster
ILIAD+50aee79f+2023-07-12-21h-45m-13s rigid_compact free_form ILIAD 676 y Put the book on the left in the black box, put the grape toy in the bo
IPRL+2fe9715d+2023-08-21-17h-37m-45s rigid_compact clean_with_cloth IPRL 451 y Take a tissue out of the box and use it to wipe the table
IPRL+2fe9715d+2023-08-21-17h-34m-50s rigid_compact clean_with_cloth IPRL 442 y Remove a tissue from the box and put it on the table
RAD+ac111655+2023-09-18-17h-15m-37s rigid_compact reposition RAD 2449 y Unstack all the toy blocks then put the orange block in the marked zon
RAIL+80edfcb1+2023-06-30-15h-03m-19s rigid_compact free_form RAIL 195 y Remove the doll from the plate, then put the yellow block on the plate
REAL+4f8ca688+2023-05-31-10h-33m-23s rigid_compact into_container REAL 407 y Open the lid of the box then put the red cup in the box
REAL+4dbb5646+2023-06-07-12h-03m-19s rigid_compact lid_on_off REAL 242 y Put the book on the right in the box
GuptaLab+553d1bd5+2023-04-30-12h-57m-10s rigid_symmetric into_container GuptaLab 210 y Put the marker inside the bowl
IPRL+2fe9715d+2023-08-21-21h-03m-17s rigid_symmetric clean_with_cloth IPRL 597 y Use the white towel to wipe the table, then take the white lid and put
IRIS+226b9f35+2023-07-11-12h-03m-17s rigid_symmetric free_form IRIS 479 y Put the black flask on the right side of the countertop and the blue b
IRIS+226b9f35+2023-07-11-12h-01m-54s rigid_symmetric lid_on_off IRIS 358 y Press the soap bottle nozzle
IRIS+226b9f35+2023-07-11-11h-55m-54s rigid_symmetric open_close IRIS 259 y Remove the cup from the sink and put it on the countertop near the toa
RAIL+80edfcb1+2023-04-26-16h-06m-56s rigid_symmetric clean_with_cloth RAIL 135 y Pour the contents of the jug into the bowl
AUTOLab+44bb9c36+2023-11-28-08h-46m-19s rigid_thin concrete AUTOLab 992 y Place the green marker on the yellow towel, fold the towel from bottom
AUTOLab+44bb9c36+2023-11-28-09h-00m-36s rigid_thin concrete AUTOLab 655 y Put the green marker on the towel, fold the towel twice and place it i
GuptaLab+553d1bd5+2023-04-20-12h-38m-54s rigid_thin reposition GuptaLab 225 y Pick the marker on the counter and put it in the bowl
GuptaLab+553d1bd5+2023-04-20-12h-41m-59s rigid_thin reposition GuptaLab 210 y Put the marker inside the bowl
IRIS+226b9f35+2023-07-11-21h-54m-44s rigid_thin into_container IRIS 543 y Place the wooden spoon in the utensil holder
RAD+62828209+2023-09-09-09h-01m-13s rigid_thin reposition RAD 602 y Pick up the green marker and then put it back down
IPRL+7790ec0a+2023-05-05-17h-22m-57s transparent concrete IPRL 194 y Pull out the storage bucket slightly under the drawers
IPRL+7790ec0a+2023-05-05-17h-12m-15s transparent concrete IPRL 164 y Pull the transparent bin out
AUTOLab+44bb9c36+2023-11-28-10h-43m-38s unknown_no_annotation concrete AUTOLab 269 FAIL Place pen in cloth, wrap pen in cloth, and put in box.
AUTOLab+5d05c5aa+2023-07-26-16h-48m-43s unknown_no_annotation concrete AUTOLab 236 FAIL insert a cable into a straight custom build channel
CLVR+13759f6e+2023-05-16-15h-05m-43s unknown_no_annotation into_container CLVR 253 FAIL Move object into or out of container (ex: drawer, clothes hamper, plat
CLVR+13759f6e+2023-05-17-21h-21m-24s unknown_no_annotation free_form CLVR 310 FAIL Do anything you like that takes multiple steps to complete.
GuptaLab+553d1bd5+2023-04-20-13h-15m-30s unknown_no_annotation into_container GuptaLab 176 FAIL Move object into or out of container (ex: drawer, clothes hamper, plat
GuptaLab+553d1bd5+2023-05-28-15h-38m-14s unknown_no_annotation open_close GuptaLab 307 FAIL Open or close slidable objects (ex: toaster, drawers, sliding doors, d
ILIAD+50aee79f+2023-08-18-10h-16m-14s unknown_no_annotation lid_on_off ILIAD 530 FAIL Move lid on or off of container (ex: pot, cup, pill bottle)
ILIAD+50aee79f+2023-08-30-15h-41m-59s unknown_no_annotation lid_on_off ILIAD 152 FAIL Move lid on or off of container (ex: pot, cup, pill bottle)
Total size
792 MB
Files
130
Last updated
Aug 5
Pre-warmed CDN
US EU US EU

Contributors