RoboSet (RoboAgent)
Carnegie Mellon University and Meta AI · 2023 · datasets.bot · datasets.bot page
One-liner. Real-world multi-task Franka manipulation dataset (kitchen scenes): 28.5K trajectories, 12 skills, 38 tasks, 4 RGB-D camera views.
Setup
- Datasets / benchmarks: RoboSet is a large-scale real-world multi-task robot manipulation dataset collected on Franka-Emika Panda arms with Robotiq grippers across everyday kitchen scenes, combining teleoperated (9,500) and kinesthetic-playback (19,000) demonstrations. It spans 12 manipulation skills expressed across 38 tasks, recording RGB and depth frames from 4 camera views (top, left, right fixed plus a wrist camera) along with Franka joint positions/velocities, end-effector/gripper pose/velocities, and applied controls. It was released alongside the RoboAgent / MT-ACT system (CMU and Meta AI/FAIR). Data is stored as HDF5 trials containing image, proprioception, and action fields. License: MIT. Download: https://huggingface.co/datasets/jdvakil/RoboSet-Teleoperation/tree/main.
- Hardware / simulator: Embodiment: franka_panda. Environment: kitchen. Realness: physical.
Schema
trials (Trial0, Trial1, ...) -> {data: {rgb+depth camera views (top/left/right/wrist), proprioception (joint pos/vel, ee/gripper pos/vel), actions}, derived, config}
Links