The Allen Institute for AI (Ai2) has released ToyBlocks, an open simulation framework built around a robot arm and blocks on a tabletop. It is the first of what the team calls FoundationWorlds, controlled simulated environments designed so researchers can change training data or model design, repeat a test, and trace why a robot fails. Ai2 argues that demonstrations from narrow robotics tasks often fail to show what a robot has actually learned, and that small controlled settings are a standard way to study complex systems.
ToyBlocks generates skills and task demonstrations automatically, avoiding the time-consuming teleoperation usually needed to collect training data. Reinforcement learning, run in Nvidia's Isaac Lab, produces short skills such as pushing a block or flipping it without picking it up. Skill training can use privileged simulator data, such as exact block position and orientation, that a real robot cannot access directly. A task-and-motion planner then sequences those skills into full demonstrations, with hundreds of plans executed in parallel on a single GPU at up to 24 times real-time speed. Tasks can involve letters and pictures on block faces, so a robot may need to turn a block before deciding where it belongs, and domain randomization varies block size and table height to test generalisation. Ai2 also demonstrates models that act from raw sensor input, pairing its Molmo 2 model, which picks skills and points to the target block, with the ManiFlow low-level model, which controls the simulated arm.
The release builds on Ai2's earlier open robotics work, including MolmoAct 2, MolmoBot and MolmoSpaces, and code, data, models and documentation are published openly. The announcement focuses on the framework's design and intended uses rather than on independent evaluation, and Ai2 says it plans to extend the approach to more complex environments. For Australian businesses and professionals, ToyBlocks changes nothing immediately. It is a research tool for simulated robot arms, with no direct cost, regulatory or Privacy Act implications. Robotics teams working on manipulation or robot learning may find the open code worth a look, while everyone else can safely ignore it for now.
Key Points
Ai2 released ToyBlocks, an open simulation framework for robot learning built around a robot arm and blocks.
ToyBlocks is the first 'FoundationWorld', designed so researchers can vary training data and model design and trace failures.
Skills are trained with reinforcement learning in Isaac Lab and sequenced into demonstrations by a task-and-motion planner.
Automatic demonstration generation avoids teleoperation and can run hundreds of plans in parallel on one GPU at up to 24x real time.
The framework builds on Ai2's MolmoAct 2, MolmoBot and MolmoSpaces work, and its code, data and models are openly published.