Backends
Choosing between MuJoCo and MuJoCo Warp, and the vectorization, rendering, and physics differences between them.
SO101-Nexus uses MuJoCo as its default simulation backend. A MuJoCo Warp backend adds native GPU-parallel batching for reinforcement learning. Both cover the same six tasks (Touch, LookAt, Move, PickLift, PickAndPlace, StackCube) on the same SO-101 arm, with the same env IDs modulo the MuJoCo/Warp prefix. See Environments for the ID list.
The MuJoCo backend is stable. The MuJoCo Warp backend is experimental: its API and physics may change between minor releases. See Stability and versioning.
Vectorization
MuJoCo environments are vectorized through Gymnasium's standard gym.vector.SyncVectorEnv wrapper, which runs multiple environment instances on CPU.
The MuJoCo Warp backend instead batches simulation natively on the GPU. Construct any task with gym.make_vec("WarpPickLift-v1", num_envs=N, device="cuda", vectorization_mode="vector_entry_point"); reset and step return torch CUDA tensors of shape (num_envs, ...).
Warp envs route the same composable state observation components as MuJoCo (see the component table for the full list, which the two backends share). They also support WristCamera and OverheadCamera observation components, rendered in one batched ray-tracing pass on the simulation device. Camera observations return uint8 image tensors of shape (num_envs, height, width, 3), with per-world wrist-camera domain randomization.
Render Modes
MuJoCo envs support Gymnasium render_mode="human" and render_mode="rgb_array" for visualization. The rendered view is selected by RenderConfig(camera=...): the default top-down overhead view, or camera="side" for an angled tabletop bystander view (visualization only, never an observation). Warp envs do not implement render() because images are produced as observation tensors; passing a render_mode to gym.make_vec() is accepted for compatibility, warns, and is ignored, and RenderConfig.camera consequently has no effect there. Configure WristCamera or OverheadCamera in observations when a Warp policy needs pixels.
Object Pools and the Batched Model
WarpPickLift and WarpTouch accept the same heterogeneous object pools as MuJoCo (CubeObject, YCBObject, MeshObject, plus n_distractors), WarpPickAndPlace accepts a carried-object pool plus one slot per distractors pool entry when n_distractors > 0, and WarpStackCube compiles one cube slot per configured color in cube_a_colors/cube_b_colors plus the same distractor slots. Each pool object is compiled as a freejoint slot, the per-world target (or, for pick-and-place and stack-cube, the role slots plus the sampled distractors) is selected at reset, and inactive slots are parked off-world. A pick-and-place target is only ever drawn from the carried pool, so reset(options={"target_index": k}) indexes that pool alone.
Because all worlds share one compiled model, per-episode geom_rgba color randomization of a single object is unsupported. Distinct colored cube slots still give per-world color variation through selection (this is how WarpStackCube matches the MuJoCo backend's per-episode resampling of both cube colors), and the WarpPickAndPlace goal disc color is fixed to the first configured color.
Vectorized worlds may carry different tasks, so env.task_descriptions holds one string per world and info["task_description"] is returned from reset() and step(). env.task_description reduces to the shared string, or a generic family string when worlds differ.
Physics Divergence
Warp uses the implicit integrator without the noslip constraint, so policies may need re-tuning when moving between backends.
Installation and Hardware
Install the backend with the warp extra, which needs an NVIDIA GPU and CUDA >= 12.4 for GPU execution. Warp has no ROCm/AMD support. To train on an AMD GPU you install the rocm extra instead, which affects only the MuJoCo backend's training loop; see Installation.
Objects
Both backends support CubeObject, YCBObject, and MeshObject, configured through the same object-pool fields (objects, n_distractors, and distractors on the pick-and-place and stack-cube families). See Customization for the recipes and Objects for the constructor reference.