RT-1 Robot Action Dataset
Google (Robotics at Google / Everyday Robots) · 2022 · datasets.bot · datasets.bot page
One-liner. Google RT-1 teleop demos: 87k+ episodes of Everyday Robots tabletop manipulation with language, RLDS format (OXE fractal).
Setup
- Datasets / benchmarks: The RT-1 robot action dataset (fractal20220817_data in Open X-Embodiment) contains 87,212 human-teleoperated demonstration episodes of tabletop manipulation with 17 objects, collected on Google's Everyday Robots mobile manipulators. Each step pairs an RGB image and natural-language instruction with a discretized arm+base action, plus success/feasible/undesirable labels, stored in RLDS/TFDS format. License: CC-BY-4.0. Download: https://www.tensorflow.org/datasets/catalog/fractal20220817_data.
- Hardware / simulator: Embodiment: not listed. Environment: kitchen, tabletop. Realness: physical.
Schema
episodes -> steps -> {observation: {image(256x320x3), natural_language_instruction, natural_language_embedding(512), base_pose_tool_reached(7), gripper_closed, ...}, action: {world_vector(3), rotation_delta(3), gripper_closedness_action(1), base_displacement_vector(2), base_displacement_vertical_rotation(1), terminate_episode(3)}, aspects: {success, feasible, undesirable, already_success}}
Links