SO101-Nexus

SO101-Nexus is an end-to-end Python library for taking an SO-101 robot from demonstrations to a trained policy. It connects physical leader-arm teleoperation, LeRobot-compatible dataset recording, Gymnasium/MuJoCo manipulation environments, and training/evaluation hooks in one installable package.

The goal is a boring record -> clone -> reinforce workflow: collect demonstrations, replay and evaluate them in matching SO-101 environments, bootstrap with imitation learning, then fine-tune with RL.

MuJoCo is the default backend for SO-101 tasks. An optional MuJoCo Warp backend adds GPU-parallel, batched environments for large-scale RL.

Demo Rollouts

Recorded MuJoCo teleoperation rollouts are available as LeRobot datasets:

Key Features

• End-to-end SO-101 workflow -- record demonstrations, build LeRobot datasets, run environments, and train or evaluate policies
• Built-in teleoperation -- drive a simulated follower with a physical SO-100 or SO-101 leader arm
• LeRobot dataset conventions -- save SO follower state/action units plus wrist and overhead camera fields
• GPU PPO training -- train Warp baselines with tuned per-task commands and seed-validated PickLift hyperparameters
• Standard Gymnasium API -- use the environments with RL libraries and existing Gymnasium tooling
• MuJoCo and MuJoCo Warp backends -- five SO-101 manipulation tasks on MuJoCo, with an optional GPU-parallel MuJoCo Warp backend (so101-nexus[warp]) for batched RL
• Configurable scenes -- swap objects, add distractors, randomize colors, tune rewards, and choose observation components
• YCB object support -- use real-world object meshes from the YCB dataset

Quick Examples

Pick up and lift an object in five lines:

import gymnasium as gym
import so101_nexus.mujoco

env = gym.make("MuJoCoPickLift-v1", render_mode="human")
obs, info = env.reset()

for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

env.close()

Use a banana from the YCB dataset instead of the default cube:

import gymnasium as gym
import so101_nexus.mujoco
from so101_nexus import PickConfig, YCBObject

config = PickConfig(objects=YCBObject(model_id="011_banana"))
env = gym.make("MuJoCoPickLift-v1", config=config, render_mode="human")

Add distractors and randomize colors for domain randomization:

import gymnasium as gym
import so101_nexus.mujoco
from so101_nexus import PickConfig, CubeObject, YCBObject

config = PickConfig(
    objects=[
        CubeObject(color="green"),
        CubeObject(color="blue"),
        YCBObject(model_id="011_banana"),
        YCBObject(model_id="058_golf_ball"),
    ],
    n_distractors=2,
    ground_colors=["gray", "white", "black"],
    robot_colors=["yellow", "orange"],
)
env = gym.make("MuJoCoPickLift-v1", config=config, render_mode="human")

Tasks

SO101-Nexus ships five manipulation primitives that cover the fundamentals of robotic grasping and movement:

• PickLift -- grasp an object and lift it above a height threshold
• PickAndPlace -- pick up a cube and place it at a marked target location
• Touch -- bring the gripper to an object resting on the table
• LookAt -- orient the end-effector to gaze at a target object
• Move -- move the end-effector a set distance in a cardinal direction

All five tasks run on the MuJoCo backend on the SO-101 arm. The MuJoCo Warp backend covers all five (Touch, LookAt, Move, PickLift, PickAndPlace) with GPU-parallel simulation, including heterogeneous object pools (cubes, YCB, and mesh objects, with distractors), subject to the documented batched-model limitation that a single object's colour cannot be randomized per episode (the shared model's geom_rgba is global).

Get Started

• End-to-end workflow -- record demonstrations, clone them with behavior cloning, then reinforce with RL (or run it with no hardware)
• Install SO101-Nexus -- install the library with pip, uv, or from source
• Record demonstrations -- teleoperate a simulated follower and save LeRobot datasets
• Run MuJoCo environments -- create your first Gymnasium SO-101 task
• Customize tasks and objects -- tune rewards, cameras, spawn regions, colors, and observation components
• Train with PPO -- run the included baseline and adapt it for your experiments