SO101-Nexus
Get Started

Installation

Requirements, pip and from-source installs, the optional extras, and ROCm setup for AMD GPUs.

Requirements

  • Python >= 3.12. Versions 3.12 and 3.13 are verified in continuous integration.
  • Linux or macOS for the MuJoCo backend. Both are exercised in continuous integration.

Install with pip

pip install so101-nexus

Install from source

Clone the repository, then sync with uv:

git clone https://github.com/johnsutor/so101-nexus.git
cd so101-nexus
uv sync

Run the Quickstart to verify the install with a simulated robot.

Extras

The base install ships the MuJoCo backend and the environment API. Everything else is opt-in.

ExtraAddsInstall
teleopGradio demonstration recorder and leader return IK: lerobot[dataset,feetech,placo-dep], gradio, plotly, opencv-pythonpip install "so101-nexus[teleop]"
molmoact2LeRobot 0.7 MolmoAct2 model dependencies; combine with teleop for correction collectionpip install "so101-nexus[teleop,molmoact2]"
dm05Native DM0.5 model and LoRA dependencies from LeRobot 0.7See Native DM0.5
decompMeasured convex decomposition of YCB collision meshes (coacd, rtree, threadpoolctl). Without it YCB collision geometry falls back to a single convex hull. The packages ship wheels for the supported platformspip install "so101-nexus[decomp]"
vizPillow image labels and high-quality resizing in visualization helperspip install "so101-nexus[viz]"
trainTorch training dependencies for the PPO and BC examplespip install "so101-nexus[train]"
sysidMuJoCo system identification and diagnostic reportspip install "so101-nexus[teleop,sysid]"
warpGPU-parallel MuJoCo Warp backend. Needs an NVIDIA GPU and CUDA >= 12.8pip install "so101-nexus[warp]"
rocmPyTorch from AMD's ROCm 7.2 wheel index. See belowuv sync --extra train --extra rocm --no-default-groups

Extras combine: pip install "so101-nexus[teleop,train]".

From source, use uv sync --extra teleop --extra molmoact2 for MolmoAct2 collection. Other LeRobot policy checkpoints may need their own model dependencies. For example, uv run --extra teleop --with "lerobot[smolvla]==0.7.0" so101-nexus teleop ... adds SmolVLA dependencies for that invocation. Use the matching LeRobot extra for other supported policy families. The base install stays independent of LeRobot.

Native DM0.5

LeRobot 0.7 includes the native DM0.5 integration. Install it with the dm05 extra:

pip install "so101-nexus[teleop,dm05]"

From a source checkout, run uv sync --extra teleop --extra dm05. Both policy extras use Transformers 5.22. The model runs in the teleop process. Weights download through the Hugging Face cache on first use. Your account must have access to the model and its license. See Teleoperation for the launch command.

Native CPU batching

The default installation requires MuJoCo 3.17 or newer, which includes mujoco.batch.Batch. No extra is required. See Backends for a runnable example and batching limits.

Try without installing

uvx runs the teleop recorder in an ephemeral environment with no permanent install:

uvx --from "so101-nexus[teleop]" so101-nexus teleop

Training on an AMD GPU (ROCm)

The rocm extra installs PyTorch from the ROCm 7.2 wheel index instead of the default CUDA build, on Linux x86_64:

uv sync --extra train --extra rocm --no-default-groups

--no-default-groups skips the dev dependency group. The test suite pins compatible Torch/TorchCodec pairs. Desktop platforms retain Torch <2.10 and TorchCodec <0.10. Linux ARM64 uses Torch 2.11 and TorchCodec 0.11 because those dataset decoder wheels require that pair. The ROCm extra selects a newer Torch build that conflicts with these test pins. uv therefore rejects combining rocm with teleop, molmoact2, dm05, dev, or test.

rocm affects MuJoCo-backend training only, meaning behavior cloning and PPO on CPU-simulated environments. It does not enable the Warp backend: Warp is built on NVIDIA Warp, which has no ROCm/AMD support and always requires a CUDA GPU.

Warp box collision patch

The optional patch in patches/mujoco-warp-3.13.0/ replaces box-pair collision detection with the CPU SAT, clipping, and edge algorithm. It preserves other collision dispatch and uses double-precision intermediates with float32 inputs and outputs. The normal warp extra does not include this patch. The grasp retention profile requires a separate physics configuration.

The patch tests require MuJoCo 3.13, which predates native CPU batching and cannot satisfy this library's current dependency requirement. From a source checkout, build and test the patch in a separate environment without the library:

nexus_root="$PWD"
engine_build=$(mktemp -d)
cd "$engine_build"
curl -fL -o mujoco_warp-3.13.0.tar.gz \
  https://files.pythonhosted.org/packages/3f/1a/acc6851ba3b6d2e51d4af0ddb5d1924618c8293512a39abe8521a6f56a5d/mujoco_warp-3.13.0.tar.gz
echo '5d0a25560fed1b6735138159bac6d138cf123d2007e81e1f86304cf15182506c  mujoco_warp-3.13.0.tar.gz' | sha256sum --check
tar -xzf mujoco_warp-3.13.0.tar.gz
cd mujoco_warp-3.13.0
git apply --check "$nexus_root/patches/mujoco-warp-3.13.0/box_box_cpu_sat_port_double.patch"
git apply "$nexus_root/patches/mujoco-warp-3.13.0/box_box_cpu_sat_port_double.patch"
uv build --wheel --out-dir "$engine_build/wheels"
uv venv --python 3.12 "$engine_build/venv"
uv pip install --python "$engine_build/venv/bin/python" \
  pytest \
  "mujoco==3.13.0" "warp-lang==1.15.0" "torch==2.9.1" \
  "$engine_build/wheels/mujoco_warp-3.13.0-py3-none-any.whl"
"$engine_build/venv/bin/python" -m pytest -q \
  "$nexus_root/patches/mujoco-warp-3.13.0/tests/test_patched_box_collision.py"
BOX_TEST_DEVICE=cuda:0 "$engine_build/venv/bin/python" -m pytest -q \
  "$nexus_root/patches/mujoco-warp-3.13.0/tests/test_patched_box_collision.py"

The standalone tests compare contact geometry and normal force against CPU MuJoCo for thin fingers, aligned faces, and edge contact. CUDA tests also exercise captured graphs. The stock dependency fails the thin-finger and edge cases.

This environment validates the standalone patch. It does not support the current SO101-Nexus library. The patch targets only mujoco-warp==3.13.0. Identical collision algorithms do not imply identical trajectories or measured hardware behavior.

Next steps

On this page