~ Lowering the barrier to entry for robotics ~ Crafted with care by @HuggingFace 🤗 Join our discord: https://xitter.twitterapi.xyz/t.co/Sx2jdT0jeF
Worldwide
Joined July 2024
- Tweets693
- Following30
- Followers21.9K
- Likes970
10,000 hours of open egocentric data with SOTA 3D head and hand tracking from @GroundedSI are now available 🤗
RoboCap-10k spans 21 industries in 5 continents with 619 real, paid work tasks all using 6-camera/dual IMU RoboCap hardware.
Explore the dataset and research ↓
Introducing RoboCap-10k. 10,000 hours of open egocentric data with metric hand / head pose on every frame.
Captured on RoboCap across 11 countries and 21 industries.
GroundedAPI produces 3D enrichment giving robot policies high-fidelity action grounding.
with @BitRobotNetwork @huggingface @rerundotio ↓
Praxis-1: an open-weight world action model for real robots 🤖
New from @runwayml and now in LeRobot.
Praxis-1 learns how objects behave and how tasks unfold from large-scale video, which gives it a head start on a new robot. One checkpoint adapts with light fine-tuning or LoRA: fine-tuned on Google's RT-1 robot, which it never saw in training, it reaches 80.0% on SimplerEnv. Weights, inference and fine-tuning code are open under Apache 2.0, commercial use included, and it runs on NVIDIA from H100 to Jetson Thor.
Model, blog and docs in the comments 👇
Robots that imagine before they act 🔮🤖
World action models predict what happens next, then act. LeRobot 0.7 adds four of them:
- Cosmos3 from @NVIDIARobotics: 16.5B Nano and 4.6B Edge, 97.9% LIBERO average. Edge runs on one GPU
- FLUX 3 Action from @bfl_ai: one open 7B diffusion transformer predicts future video and actions together, with an SO-101 checkpoint
- LaWAM from RLinf: imagines the future in latent space (DINOv3 features), 98.5% on LIBERO measured in LeRobot
- Long-WAM from NVIDIA: looks back over 19.2 s of video history, 99.0% on LIBERO measured in LeRobot
Docs in the comments 👇
Cosmos3: huggingface.co/docs/lerobot/…
FLUX 3 Action: huggingface.co/docs/lerobot/…
LaWAM: huggingface.co/docs/lerobot/…
Long-WAM: huggingface.co/docs/lerobot/…
3
503
LeRobot retweeted
Robot policies work in the lab. The real world is where they break.
To close that gap, we are open-sourcing the most diverse egocentric dataset with the help of @LeRobotHF.
Coming to 🤗 @huggingface on Thursday November 12th.
Train on terabytes. Don't download them. 📦
LeRobot 0.7 trains straight from the Hub:
- Streaming: each worker fetches only the video bytes it needs. 0.52% of step time spent waiting for data (π0.5, 4× H100, 10.7 TB dataset)
- LanceDB, built with @lancedb: frames are read in place. 1.7% waiting for data, vs 37.4% from local disk after a 384 GB download
- GPU augmentations: 2.8× faster ACT training on an H100, with 8 new robotics augmentations
Plus a training stack for big models: FSDP2 + HSDP, checkpoints saved 3.81× faster and resumable on any GPU count, FP16, EMA and Trackio.
--dataset.streaming=true
Docs in the comments 👇
📚 Dataset streaming: huggingface.co/docs/lerobot/…
📚 LanceDB datasets: huggingface.co/docs/lerobot/…
📚 Multi-GPU training: huggingface.co/docs/lerobot/…
5
681
LeRobot retweeted
“Training starts as soon as the download finishes” is a surprisingly expensive sentence.
@huggingface’s @LeRobotHF v0.7.0 reads only the bytes each batch needs from object storage with @lancedb.
Skip the full-dataset download. Same training API.
na2.hubs.ly/H08Sz_F0
Talk to your robot while it works 🗣️🤖
New in LeRobot 0.7: interactive rollouts. Start a policy, then steer it mid-run, without a reset:
- /subtask: switch what it's doing
- /vqa: ask what it sees
- /autosteer: give it a goal and let a planner pick the steps
- record episodes while you steer
lerobot-rollout --interactive=true
No text-generating policy? Bring any VLM as the planner: transformers serve, vLLM or Inference Providers.
And FineART-VLA, trained on FineART from @scale_AI and Hugging Face (1,718 h, 533,913 subtask labels), says its next subtask out loud, then does it.
Docs in the comments 👇
📚 Interactive rollouts: huggingface.co/docs/lerobot/…
📚 Train policies that read and write language: huggingface.co/docs/lerobot/…
🤖 FineART-VLA: huggingface.co/docs/lerobot/…
📊 FineART dataset: huggingface.co/datasets/lero…
5
738
LeRobot retweeted
LeRobot v0.7.0 is out, and this is a very big one 🦾
You can now talk/steer a robot while it’s running; change its task:
`/subtask pick up the red cube`
Ask what it sees:
`/vqa how many cubes are left?`
Or give it a higher-level goal:
`/autosteer clear the table`
and let the policy or an external VLM planner figure out the subtasks!
And this just scratches the surface of v0.7.0! Other banger features include 👇
Training directly on huge datasets from the Hub, Storage Buckets or @lancedb without downloading them first; particularly relevant when the Hub is already approaching !!300 TB of open Physical AI data!!
The training stack has also been rebuilt for 5-20B parameter world action models: FSDP2, HSDP, distributed checkpoints, FP16, EMA and more.
The model zoo keeps growing:
- 4 new world action models, including Cosmos 3 and FLUX 3 Action
- 4 new VLAs
- remote inference, so the robot can stay lightweight while the policy runs on a GPU server
- a common simulation stack that works much more like real hardware
LeRobot is becoming less about individual robot policies and more about having one open stack from TBs of demonstrations, all the way to a robot you can interact with in real time!
📄 Blog: huggingface.co/blog/lerobot/…
📚 Docs: huggingface.co/docs/lerobot/…
📦 Release notes: github.com/huggingface/lerob…
9
1,072
The SO-101 you can build at home is now part of a 100+ robot real-world eval fleet, alongside G1s, YAMs and Frankas 🦾
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
LeRobot retweeted
Introducing the Lattice 4D Sample Dataset!
Here's an early look at real-world multiview captures transformed into 3D/4D reconstructions, human and object motion, derived egocentric views, and LeRobot-compatible data.
Read our Hugging Face article:
huggingface.co/blog/LatticeA…
G0.5 from Galaxea is now available in LeRobot! 🤖
G0.5 is a 2B vision-language-action model built on Qwen3.5.
Before each action chunkmit predicts:
- the current subtask ("pick up the red cube") from the overall task
- bounding boxes around the objects involved
- then the actions, which are conditioned on that reasoning
LeRobot's annotation pipeline can also generate plans, memory and visual Q&A into your datasets, opening the door to longer, more complex multi-step setups for future policies!
Thanks to the Galaxea team for the open release!
Links to the blog, docs and our walkthrough in the comments 👇