{"RequestId":"2456cba2-0934-4210-9493-b708b16f3294","Code":200,"Message":"success","Data":{"isTop":null,"relatedPaperId":null,"nexa":null,"Id":138163,"Namespace":"agibot_world","Name":"AgiBotDigitalWorld","CreatedBy":"Cherrytest","ChineseName":"","License":"Apache License 2.0","Description":"","Visibility":5,"Type":4,"Owner":"agibot_world","UserDefineTags":"Robotics manipulation,real-world,dual-arm","Likes":1,"Downloads":5612,"Size":null,"ReadmeContent":"\n\n\n\n
\n \n \"Project\n \n
\n\n\n\n# Key Features \uD83D\uDD11\n- **1 million+** steps of enhanced digital twins of long-horizon real-world tasks from Agibot World.\n- **500,000+** steps of atomic tasks automatically generated by agents.\n- **180+ classes of objects**.\n- **5 classes of scenes**.\n\n\n# News \uD83C\uDF0D\n- **`[2025/2/24]`** AgiBot Digital World released on Huggingface. [Download Link](https://huggingface.co/datasets/agibot-world/AgiBotDigitalWorld/tree/main)\n\n\n# TODO List \uD83D\uDCC5\n- [ ] **AgiBot Digital World**: More high-quality simulation data, including atomic skill task data and digital twin-enhanced data aligned with tasks in Agibot World. (ongoing open source)\n\n\n\n# Table of Contents\n- [Key Features \uD83D\uDD11](#key-features-)\n- [News \uD83C\uDF0D](#news-)\n- [TODO List \uD83D\uDCC5](#todo-list-)\n- [Get started \uD83D\uDD25](#get-started-)\n - [Download the Dataset](#download-the-dataset)\n - [Dataset Structure](#dataset-structure)\n - [Explanation of Proprioceptive State](#explanation-of-proprioceptive-state)\n - [Dataset Preprocessing](#dataset-preprocessing)\n- [License and Citation](#license-and-citation)\n\n# Get started \uD83D\uDD25\n## Download the Dataset\nTo download the full dataset, you can use the following code. If you encounter any issues, please refer to the official Hugging Face documentation.\n```\n# Make sure you have git-lfs installed (https://git-lfs.com)\ngit lfs install\n\n# When prompted for a password, use an access token with write permissions.\n# Generate one from your settings: https://huggingface.co/settings/tokens\ngit clone https://huggingface.co/datasets/agibot-world/AgiBotDigitalWorld\n\n# If you want to clone without large files - just their pointers\nGIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/agibot-world/AgiBotDigitalWorld\n```\n\nIf you only want to download a specific task from the AgiBotDigitalWorld dataset, such as `digitaltwin_5`,follow these steps:.\n```\n# Ensure Git LFS is installed (https://git-lfs.com)\ngit lfs install\n\n# Initialize an empty Git repository\ngit init AgiBotDigitalWorld\ncd AgiBotDigitalWorld\n\n# Set the remote repository\ngit remote add origin https://huggingface.co/datasets/agibot-world/AgiBotDigitalWorld\n\n# Enable sparse-checkout\ngit sparse-checkout init\n\n# Specify the folders and files you want to download\ngit sparse-checkout set observations/digitaltwin_5 task_info/digitaltwin_5.json scripts proprio_stats parameters\n\n# Pull the data from the main branch\ngit pull origin main\n\n```\n\nTo facilitate the inspection of the dataset's internal structure and examples, we also provide a sample dataset. Please refer to `sample_dataset.zip`.\n\n## Dataset Preprocessing\nOur project relies solely on the [lerobot library](https://github.com/huggingface/lerobot) (dataset `v2.0`), please follow their [installation instructions](https://github.com/huggingface/lerobot?tab=readme-ov-file#installation).\nHere, we provide scripts for converting it to the lerobot format. \n\n### Requirements\nRequeir ffmpeg>=7.1(You can install with **conda install -y -c conda -forge ffmpeg**)\n```\nexport SVT_LOG=0\npython scripts/convert_to_lerobot.py --data_dir DATASET_FOLDER --save_dir SAVE_FOLDER --repo_id=agibot/agibotdigital --preprocess_video\n\n## Example\n# python scripts/convert_to_lerobot.py --data_dir final_format_data --save_dir ./output --repo_id=agibot/agibotdigital --preprocess_video\n\n```\n### Visualization\n```\npython scripts/visualize_dataset.py --repo-id='agibot/agibotdigital' --episode-index=0 --dataset-path=SAVE_FOLDER\n\n## Example\n# python scripts/visualize_dataset.py --repo-id='agibot/agibotdigital'--episode-index=0 --dataset-path=output\n```\nWe sincerely thank the developers of lerobot for their exceptional contributions to the open-source community..\n\n## Dataset Structure\n### Folder hierarchy\n```\ndata\n├── observations\n│ ├── digitaltwin_0 # This represents the task id.\n│ │ ├── 9b21cf2e-829f-4aad-9b61-9edc5b947163 # This represents the episode uuid.\n│ │ │ ├── depth # This is a folder containing depth information saved in PNG format.\n│ │ │ ├── video # This is a folder containing videos from all camera perspectives.\n│ │ ├── 131e407a-b828-4937-a554-e6706cbc5e2f\n│ │ │ └── ...\n│ │ └── ...\n│ ├── digitaltwin_1\n│ │ ├── 95808182-501f-4dca-984b-7404df844d31\n│ │ │ ├── depth\n│ │ │ ├── video\n│ │ ├── edb0774b-13bb-4a8b-8bb0-71e82fe3ff6a\n│ │ │ └── ...\n│ └── ...\n├── meta_info\n│ ├── digitaltwin_0 # This represents the task id.\n│ │ ├── 9b21cf2e-829f-4aad-9b61-9edc5b947163 # This represents the episode uuid.\n│ │ │ ├── task_info.json # This represents the task information.\n│ │ │ ├── proprio_meta_info.h5 # This file contains all the robot's proprioceptive information.\n│ │ │ ├── camera_parameter.json # This contains all the cameras' intrinsic and extrinsic parameters.\n│ │ ├── 131e407a-b828-4937-a554-e6706cbc5e2f\n│ │ │ ├── task_info.json \n│ │ │ ├── proprio_meta_info.h5\n│ │ │ ├── camera_parameter.json\n│ │ └── ...\n│ └── digitaltwin_1\n│ ├── 95808182-501f-4dca-984b-7404df844d31\n│ │ │ ├── task_info.json \n│ │ │ ├── proprio_meta_info.h5\n│ │ │ ├── camera_parameter.json\n│ └── edb0774b-13bb-4a8b-8bb0-71e82fe3ff6a\n│ │ │ ├── task_info.json \n│ │ │ ├── proprio_meta_info.h5\n│ │ │ ├── camera_parameter.json\n| └── ...\n```\n### json file format\nIn the `task_info.json` file, we store the basic information of every episode along with the language instructions. Here, we will further explain several specific keywords.\n- **action_config**: The content corresponding to this key is a list composed of all **action slices** from the episode. Each action slice includes a start and end time, the corresponding atomic skill, and the language instruction.\n- **key_frame**: The content corresponding to this key consists of annotations for keyframes, including the start and end times of the keyframes and detailed descriptions.\n\n```\n{\n \"episode_id\": \"9b21cf2e-829f-4aad-9b61-9edc5b947163\",\n \"task_id\": \"digitaltwin_5\",\n \"task_name\": \"pick_toys_into_box\",\n \"init_scene_text\": \"\",\n \"label_info\": {\n \"objects\": {\n \"extra_objects\": [\n {\n \"object_id\": \"omni6DPose_book_000\",\n \"workspace_id\": \"book_table_extra\"\n }\n ],\n \"task_related_objects\": [\n {\n \"object_id\": \"omni6DPose_toy_motorcycle_023\",\n \"workspace_id\": \"book_table_dual_left\"\n },\n {\n \"object_id\": \"omni6DPose_toy_truck_030\",\n \"workspace_id\": \"book_table_dual_right\"\n },\n {\n \"object_id\": \"genie_storage_box_002\",\n \"workspace_id\": \"book_table_dual_middle\"\n }\n ]\n },\n \"action_config\": [\n {\n \"start_frame\": 0,\n \"end_frame\": 178,\n \"action_text\": \"\",\n \"skill\": \"Pick\",\n \"active_object\": \"gripper\",\n \"passive_object\": \"omni6DPose_toy_motorcycle_023\"\n },\n {\n \"start_frame\": 179,\n \"end_frame\": 284,\n \"action_text\": \"\",\n \"skill\": \"Place\",\n \"active_object\": \"omni6DPose_toy_motorcycle_023\",\n \"passive_object\": \"genie_storage_box_002\"\n },\n {\n \"start_frame\": 285,\n \"end_frame\": 430,\n \"action_text\": \"\",\n \"skill\": \"Pick\",\n \"active_object\": \"gripper\",\n \"passive_object\": \"omni6DPose_toy_truck_030\"\n },\n {\n \"start_frame\": 431,\n \"end_frame\": 536,\n \"action_text\": \"\",\n \"skill\": \"Place\",\n \"active_object\": \"omni6DPose_toy_truck_030\",\n \"passive_object\": \"genie_storage_box_002\"\n }\n ],\n \"key_frame\": []\n }\n}\n```\n### h5 file format\nIn the `proprio_stats.h5` file, we store all the robot's proprioceptive data. For more detailed information, please refer to the [explanation of proprioceptive state](#explanation-of-proprioceptive-state).\n\n```\n|-- timestamp\n|-- state\n |-- effector\n |-- force\n |-- index\n |-- position\n |-- end\n |-- angular\n |-- orientation\n |-- position\n |-- velocity\n |-- wrench\n |-- joint\n |-- current_value\n |-- effort\n |-- position\n |-- velocity\n |-- robot\n |-- orientation\n |-- orientation_drift\n |-- position\n |-- position_drift\n|-- action\n |-- effector\n |-- force\n |-- index\n |-- position\n |-- end\n |-- angular\n |-- orientation\n |-- position\n |-- velocity\n |-- wrench\n |-- joint\n |-- effort\n |-- index\n |-- position\n |-- velocity\n |-- robot\n |-- index\n |-- orientation\n |-- position\n |-- velocity\n```\n## Explanation of Proprioceptive State\n### Terminology\n*The definitions and data ranges in this section may change with software and hardware version. Stay tuned.*\n\n**State and action**\n1. State\nState refers to the monitoring information of different sensors and actuators.\n2. Action\nAction refers to the instructions sent to the hardware abstraction layer, where controller would respond to these instructions. Therefore, there is a difference between the issued instructions and the actual executed state.\n\n**Actuators**\n1. ***Effector:*** refers to the end effector, for example dexterous hands or grippers.\n2. ***End:*** refers to the 6DoF end pose on the robot flange.\n4. ***Joint:*** refers to the joints of the robot, with 34 degrees of freedom (2 DoF head, 2 Dof waist, 7 DoF each arm, 8 Dof each gripper).\n5. ***Robot:*** refers to the robot's pose in its surrouding environment. The orientation and position refer to the robot's relative pose in the odometry coordinate syste\n\n### Common fields\n1. Position: Spatial position, encoder position, angle, etc.\n2. Velocity: Speed\n3. Angular: Angular velocity\n4. Effort: Torque of the motor. Not available for now.\n5. Wrench: Six-dimensional force, force in the xyz directions, and torque. Not available for now.\n\n### Value shapes and ranges\n| Group | Shape | Meaning | \n| --- | :---- | :---- |\n| /timestamp | [N] | timestamp in seconds:nanoseconds in simulation time |\n| /state/effector/position (gripper) | [N, 2] | left `[:, 0]`, right `[:, 1]`, gripper open range in mm |\n| /state/end/orientation | [N, 2, 4] | left `[:, 0, :]`, right `[:, 1, :]`, flange quaternion with wxyz |\n| /state/end/position | [N, 2, 3] | left `[:, 0, :]`, right `[:, 1, :]`, flange xyz in meters |\n| /state/joint/position | [N, 34] | joint position based on joint names | \n| /state/joint/velocity | [N, 34] | joint velocity based on joint names | \n| /state/joint/effort | [N, 34] | joint effort based on joint names | \n| /state/robot/orientation | [N, 4] | quaternion in wxyz |\n| /state/robot/position | [N, 3] | xyz position, where z is always 0 in meters |\n| /action/*/index | [M] | actions indexes refer to when the control source is actually sending signals |\n| /action/effector/position (gripper) | [N, 2] | left `[:, 0]`, right `[:, 1]`, gripper open range in mm |\n| /action/end/orientation | [N, 2, 4] | same as /state/end/orientation |\n| /action/end/position | [N, 2, 3] | same as /state/end/position |\n| /action/end/index | [M_2] | same as other indexes |\n| /action/joint/position | [N, 14] | same as /state/joint/position |\n| /action/joint/index | [M_4] | same as other indexes |\n| /action/robot/velocity | [N, 2] | vel along x axis `[:, 0]`, yaw rate `[:, 1]` |\n| /action/robot/index | [M_5] | same as other indexes | \n\n\n# License and Citation\nAll the data and code within this repo are under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Please consider citing our project if it helps your research.\n\n```BibTeX\n@misc{contributors2025agibotdigitalworld,\n title={AgiBot DigitalWorld},\n author={Jiyao Zhang, Mingjie Pan, Baifeng Xie, Yinghao Zhao, Wenlong Gao, Guangte Xiang, Jiawei Zhang, Dong Li, Zhijun Li, Sheng Zhang, Hongwei Fan, Chengyue Zhao, Shukai Yang, Maoqing Yao, Chuanzhe Suo, Hao Dong},\n howpublished={\\url{https://agibot-digitalworld.com/}},\n year={2025}\n}\n\n```","AlreadyStar":false,"GmtCreate":1756282941,"GmtModified":1792167469,"Tags":null,"Status":1,"FullName":"AgiBot World","Organization":{"Id":2295,"GmtCreated":1752003016,"GmtModified":1752035019,"Name":"agibot_world","FullName":"AgiBot World","Type":2,"Avatar":"https://resouces.modelscope.cn/avatar/fe1723d7-f23e-4aa8-8b50-6a82dee86ebe.png","CreatedBy":"AgiBotWorld","GithubAddress":"https://agibot-world.com/","Description":"[\"root\",{},[\"p\",{},[\"span\",{\"data-type\":\"text\"},[\"span\",{\"data-type\":\"leaf\"},\"大多数现有的机器人学习数据集主要以受限环境中的短程任务为主，低质量数据以及有限的感知能力限制了其在现实场景中的应用。\"]]],[\"p\",{},[\"span\",{\"data-type\":\"text\"},[\"span\",{\"data-type\":\"leaf\"},\"为此，我们推出了AgiBot World，标志着世界上首个大规模高质量机器人学习数据集的诞生。AgiBot World 配套提供了基准模型、基准测试并构建出一套完整的生态系统，让高质量机器人数据在学术界与工业界都触手可及，具身智能领域的“ImageNet 时刻”已经到来！\"]]],[\"p\",{},[\"span\",{\"data-type\":\"text\"},[\"span\",{\"data-type\":\"leaf\"},\"AgiBot World 包含了超过百万条原子动作轨迹，采集自 100 个同构型机器人，涵盖五大领域中的 100 余种真实场景，涉及精细操作、工具使用、多机协作等复杂任务。AgiBot World 基于全身可控的移动式双臂机器人进行数据采集，配备了视觉触觉传感器、六维力传感器、六自由度灵巧手等先进设备，可用于模仿学习、多智能体协作等前沿研究。\"]]],[\"p\",{},[\"span\",{\"data-type\":\"text\"},[\"span\",{\"data-type\":\"leaf\"},\"AgiBot World 致力于变革大规模机器人学习，推动可扩展的机器人系统在现实场景下的落地应用。作为一个开源平台，我们邀请广大研究者和相关从业人员共同创造具身智能的未来。\"]]]]","DatasetCount":null},"UsedFor":null,"CertificationMark":0,"LastUpdatedTime":1782580228,"FromSite":"maas","SourcePlatform":null,"topIndex":null,"IsFlex":0,"StorageSize":576636661398,"RelateArxivId":null,"Avatar":null,"ProtectedMode":2,"ApprovalMode":1,"ApplyMeta":null,"NEXA":null},"PageNumber":null,"PageSize":null,"TotalCount":null}