This one isn't a research claim. Calibrating a simulator against real hardware deviation data is an established pattern - digital twins, domain randomization tuned from real rollouts, the whole “sim gets better the more it disagrees with reality” idea has been around the field for years. What we can talk about honestly is how we're actually building it, what's working, and what isn't yet.
What we actually built first
The first real decision was what to build the simulator on top of, since building a physics engine from scratch was never on the table - the value we can add is in orchestration, calibration, and the feedback loop, not in reinventing contact physics. We ran the choice as an actual comparison rather than a default pick: MuJoCo + MJX against Genesis, the newer, faster, visually flashier alternative.
Genesis lost for reasons that had nothing to do with raw speed. Our MVP hardware - the SO-101 arm - already has official MuJoCo MJCF models with native support across LeRobot, Gymnasium-Robotics, and MuJoCo Menagerie; Genesis's equivalent integrations are third-party and flagged as immature by Genesis's own maintainers. MuJoCo's WASM bindings also let physics run client-side in the browser, so our 3D web viewer costs us close to nothing per customer - Genesis's superior native renderer would need expensive server-side pixel streaming to reach a browser at all. And there's a governance detail worth naming plainly: Genesis's open-source project is now steered by a startup building its own full-stack robot hardware, a potential direct competitor whose roadmap incentives don't obviously stay aligned with ours. None of this needed a week-long exploratory spike - the evidence was specific enough to our exact use case that we ran a two-day validation spike instead, just to sanity-check the choice, not re-litigate it.
What “Sim Fabric” actually does today
Sim Fabric takes a sentence - “a lab bench with a robot arm and a mug” - and turns it into a real simulator package: worlds/*.sdf for Gazebo, mjcf/scene.xml for MuJoCo, usd/*.usda for Isaac or any DCC tool, plus config, launch files, and the source assets with an auto-generated attribution notice. It does this through retrieval and composition against an indexed library of 2,621 assets, not generation - a generated mesh has no collision geometry, no inertia, no joints, and physical fidelity is the whole point here, so we made that call early and it's recorded as an explicit architecture decision, not an oversight.
The honest state of it, in our own words
This is the part we think matters most to actually publish, because it's the part most companies leave out. Right now:
- Scenes are “still lifes that now move” - position servos hold a retrieved rest pose and reach toward a graspable object within range. Nothing is actually grasped yet; the arm reaches and returns.
- Torque-controlled robots - quadrupeds - hold their pose but aren't driven. That needs a tuned controller, and an untuned one reads as a fault, not as walking, so we haven't shipped it as a feature yet.
- 94% of placeable corpus assets render correctly. The rest are mostly upstream mesh problems in the source projects, not ours to silently paper over.
- A stored run restores inside the studio, but isn't shareable yet, and its generated files aren't kept - re-viewing a past run currently means re-running it.
We're publishing these gaps on purpose. A sim-to-real pipeline is only trustworthy if the deviation reports it produces are honest, and that starts with being honest about the simulator's own limitations first.
Where the feedback loop actually closes
The part we think is the real moat isn't the simulator - it's what happens after a policy runs on real hardware. Every real run against our SO-101 fleet produces a deviation report: where the real robot's telemetry (joint torques, positions, timing) diverged from what the sim predicted. That deviation data feeds back into domain-randomization and physics-calibration parameters, so the simulator gets more accurate specifically where it was wrong, calibrated against hardware we actually operate rather than generic priors. This is the part that compounds - a generic open-source simulator has no equivalent feedback source, because it has no fleet to disagree with.
We standardized on the LeRobot dataset format for every recording, sim and real, from day one - not because it's the most sophisticated format available, but because it's what most of our target customers already know, and it lowers the switching cost to move data in and out of the platform freely.
Where this goes next
Phase 1 of our roadmap is procuring five SO-101 rigs and building the actual “submit policy → run overnight → get report” pipeline end to end - that's the step that turns this from an architecture decision into a working feedback loop with real deviation data behind it. The honest test of whether this thesis holds isn't whether the idea is novel - it isn't - it's whether the deviation reports six months from now show the simulator visibly converging toward our specific hardware in a way a generic sim never would.