Data and Simulation Infrastructure for Physical AI
New York
Joined December 2025
- Tweets82
- Following11
- Followers2.8K
- Likes264
Pinned Tweet
Introducing Midcentury.
We’re building the data and simulation infra for physical AI.
Today, we’re coming out of stealth with a $15M Series Seed to scale robotics beyond polished demos.
We’re already supporting frontier labs with:
→ The world’s largest egocentric dataset: 2M+ hours, 50+ environments, 20,000+ tasks
→ Matrix: a frontier simulation platform scaled with our real-world data to evaluate and post-train policies at scale
Midcentury retweeted
sims working is the best case for robotics data labs, not the worst
sim will unlock the superintelligence moment for physical ai but it's not sim vs real. real world data like ego and tactile grounds sim in the right physics and distribution and sim scales it from there
AVs already did this. waymo built simulationcity on top of 20m+ real miles, recreating the rare encounters its fleet actually saw and running billions of variations in sim
LLMs are the latest proof. RL environments are carrying frontier post-training and they've made labs way hungrier for real workplace data. reliable sim starts with real world data
Midcentury retweeted
Electric panel with researchers from @SkildAI , @physical_int, @amazon, and @nvidia. Thanks for partnering @MidcenturyAI
Midcentury retweeted
“Almost nobody has stereo off the shelf.”
At CoRL this week - we’ve built one of the largest OTS stereo datasets and benchmark it obsessively.
Would love to talk stereo, data quality, and post-training results. DMs are open.
Midcentury retweeted
CoRL is unreal this year. our rooftop happy hour got ~10x more signups than the space can hold
would love to meet more researchers and builders at the frontier. dm me to chat tactile data, simulation, deployment, or research partnerships
we’re renting out a rooftop at CoRL next Monday for a happy hour
a stacked room of researchers from NVIDIA, Physical Intelligence, Skild, Dyna + more working across robotics data, simulation, and deployment
few spots left, join us!
luma.com/u6rr5lql
Midcentury retweeted
real2sim (and most of sim for that matter) is one of those low floor/high ceiling skills— anyone with an Astra subscription can participate, but it takes real technical depth to be at the top.
when you are training perception-heavy models like RFMs, every detail matters.
Midcentury retweeted
we’re renting out a rooftop at CoRL next Monday for a happy hour
a stacked room of researchers from NVIDIA, Physical Intelligence, Skild, Dyna + more working across robotics data, simulation, and deployment
few spots left, join us!
luma.com/u6rr5lql
Midcentury retweeted
learned models supervised on physics-rich data are the best bet for closing the sim2real gap
data is still the bottleneck. get enriched ego data with tactile and depth, at scale and diversity, and you can solve sim for dexterous, contact-rich manipulation
good to see this direction picking up. it's been core to our thesis, more soon!
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Midcentury retweeted
the best robotics models will come from teams that understand failure modes best and know exactly what data and supervision it takes to fix them
three failure modes stand out here: grasp selection, 3D positioning, and failure recovery
our post-training experiments are already showing meaningful gains from more targeted data and supervision. if you're curious what we're finding so far, reach out!
This halloween @archiexzzz , @notnoahphan and I were testing 4 models on a pumpkin 🎃 @i2rt_robotics YAM gripper can lift only by the stem tip.
TL;DR:
> @dreamlabs_ai - Dream-1: tried the body once, switched to the stem, and put it in the box.
> @OpenAI - Astra: aimed for the stem but closed too high. We suspect a depth or positioning error.
> @PokeBot_Global / RoboAPI: kept trying around the body and top without securing a lift.
> @Google @GeminiApp : reached for the stem but never got a grip.
> Failure analysis: grasp choice, positioning, and recovery need to be measured separately.
> Data needed: grasp labels for this gripper, measured 3D targets, and failed attempts paired with successful corrections.
1/5
Midcentury retweeted
This halloween @archiexzzz , @notnoahphan and I were testing 4 models on a pumpkin 🎃 @i2rt_robotics YAM gripper can lift only by the stem tip.
TL;DR:
> @dreamlabs_ai - Dream-1: tried the body once, switched to the stem, and put it in the box.
> @OpenAI - Astra: aimed for the stem but closed too high. We suspect a depth or positioning error.
> @PokeBot_Global / RoboAPI: kept trying around the body and top without securing a lift.
> @Google @GeminiApp : reached for the stem but never got a grip.
> Failure analysis: grasp choice, positioning, and recovery need to be measured separately.
> Data needed: grasp labels for this gripper, measured 3D targets, and failed attempts paired with successful corrections.
1/5
Midcentury retweeted
For anyone curious about why we’re confident this work is worth pursuing:
We tested whether human egocentric data could help adapt π0.5 to LIBERO-10 with limited robot demos. The comparison with EgoDex motivates further testing for a number of reasons. 🧵
We post-trained @physical_int's π0.5 on our ego data and found it outperformed Apple’s EgoDex on LIBERO-10.
Equal amounts of data were sampled from both datasets with matched tasks, environments, and objects.
Our results suggest that viewpoint diversity and richer action supervision can overcome a fundamental limitation of robot learning: dependence on fixed camera positions.
Midcentury retweeted
The research team has been loving this quick research/experimentation format. It really gives us a way to probe our progress, identify improvements, and share simple learnings with the community quickly— expect a lot more of these in the near future.
Check out our latest one where we put our proprietary (not for long 😉) data up against EgoDex!
We post-trained @physical_int's π0.5 on our ego data and found it outperformed Apple’s EgoDex on LIBERO-10.
Equal amounts of data were sampled from both datasets with matched tasks, environments, and objects.
Our results suggest that viewpoint diversity and richer action supervision can overcome a fundamental limitation of robot learning: dependence on fixed camera positions.
We post-trained @physical_int's π0.5 on our ego data and found it outperformed Apple’s EgoDex on LIBERO-10.
Equal amounts of data were sampled from both datasets with matched tasks, environments, and objects.
Our results suggest that viewpoint diversity and richer action supervision can overcome a fundamental limitation of robot learning: dependence on fixed camera positions.
To post-train the full π0.5 model, we converted estimated human hand motion into action targets and mixed the data with robot demonstrations.
Each training arm used the same budget of 20 robot demonstrations, two for each LIBERO-10 task.
1
10
639
This is the first in a series of experiments to better understand what makes human data useful for post-training.
We're continuing to study how data selection, viewpoint diversity, and action supervision affect policy generalization.
Full report by @robot_xyz_rpy below.
midcentury.xyz/blog/shakey
1
11
578
RT @ckxyz_: 10m+ hour robotics datasets are about to be normal
the kicker is frontier labs will only care about usable hours. sub-scale p…
This quoted post is unavailable.
Midcentury retweeted
“As more data becomes available, the bottleneck is shifting to data quality.”
We’ve 2M+ hours of ego data at @MidcenturyAI . With Sortex, we’re making that corpus searchable and curating it around specific robot training needs.
Enjoy reading Sortex below
Sortex: our data quality platform built to search and filter the world's largest egocentric dataset.
Across 50+ environments and 20k+ tasks, it uses an in-house model and evaluation harness to score each clip's usability, selecting for specific training objectives by:
→ Narrating tasks and steps
→ Checking hand and action visibility
→ Comparing against the rest of the corpus
We also tested Sortex on popular open datasets and found they differ widely in quality and coverage.
Midcentury retweeted
We built Sortex because “2M+ hours” tells you almost nothing by itself. What matters is what’s in there: whether each clip actually shows the work clearly, which tasks and environments the corpus covers, and what every new recording adds. Our goal is to make quality measurable and inspectable at both levels, from a single frame up to the whole dataset. Read more below!
Sortex: our data quality platform built to search and filter the world's largest egocentric dataset.
Across 50+ environments and 20k+ tasks, it uses an in-house model and evaluation harness to score each clip's usability, selecting for specific training objectives by:
→ Narrating tasks and steps
→ Checking hand and action visibility
→ Comparing against the rest of the corpus
We also tested Sortex on popular open datasets and found they differ widely in quality and coverage.
Sortex: our data quality platform built to search and filter the world's largest egocentric dataset.
Across 50+ environments and 20k+ tasks, it uses an in-house model and evaluation harness to score each clip's usability, selecting for specific training objectives by:
→ Narrating tasks and steps
→ Checking hand and action visibility
→ Comparing against the rest of the corpus
We also tested Sortex on popular open datasets and found they differ widely in quality and coverage.
We also ran Sortex on random clips from three open egocentric datasets.
About one in six clips failed screening in EPIC-KITCHENS-100 and RekaDaily-10k, mostly for darkness, shake, blur, and faces.
In Gen-HumanEgo, 42% of the clips that passed were near-duplicates of another clip in the sample.
1
8
462
Sortex was built by Addea Gupta (@addeagupta) and David Guo (@dgmonsoon). Read more below.
midcentury.xyz/blog/sortex
10
418