Scene Objects and Assets

Scene objects define what the robot interacts with. All object types live in so101_nexus and share the abstract SceneObject base class.

from so101_nexus import CubeObject, MeshObject, YCBObject

All three concrete object types work on the MuJoCo and MuJoCo Warp backends, which build their scenes through the same so101_nexus.object_slots.build_object_scene_xml.

SceneObject (abstract)

Base class for all scene objects. Subclasses must implement __repr__(), which returns a natural-language description of the object (used in logging and in the task string). SceneObject cannot be instantiated directly; use one of the concrete subclasses below.

CubeObject

A solid-color cube with configurable size and mass.

from so101_nexus import CubeObject

cube = CubeObject(half_size=0.02, mass=0.015, color="blue")
repr(cube)  # "blue cube"

__repr__() returns "{color} cube", for example "red cube".

YCBObject

A YCB benchmark object loaded from mesh files. Assets are downloaded on first use and cached (see YCB assets).

from so101_nexus import YCBObject

banana = YCBObject("011_banana")
repr(banana)  # "banana"

__repr__() returns the human-readable name from the YCB_OBJECTS dictionary.

Model IDs

model_id must be one of the ten supported IDs, which are also the members of the YcbModelId type alias:

Passing any other model_id raises an error. The same mapping is available at runtime:

from so101_nexus import YCB_OBJECTS

for model_id, name in YCB_OBJECTS.items():
    print(f"{model_id}: {name}")

MeshObject

A custom object defined by your own collision and visual mesh files.

from so101_nexus import MeshObject

obj = MeshObject(
    collision_mesh_path="/path/to/collision.obj",
    visual_mesh_path="/path/to/visual.obj",
    mass=0.05,
    name="custom widget",
    scale=0.8,
)
repr(obj)  # "custom widget"

__repr__() returns the name provided at construction.

YCB assets

YCB mesh assets are downloaded from the Hugging Face Hub on first use and cached at:

~/.cache/so101_nexus/ycb/{model_id}/

Each model directory holds visual.obj, the convex collision parts under collision_v2/ (collision_000.obj ... plus a parts.json manifest), and, when extraction succeeds, texture.png.

Creating a YCBObject and stepping an environment downloads whatever is missing. You can also trigger the download explicitly:

from so101_nexus import ensure_ycb_assets

path = ensure_ycb_assets("011_banana")

ensure_ycb_assets()

def ensure_ycb_assets(model_id: str) -> Path

Downloads the mesh assets for model_id if they are not already cached, then returns the cache directory.

visual.obj and the collision_v2/ parts are the required geometry cache. The collision geometry is a convex decomposition of the visual scan, computed once with CoACD and cached, so physics sees the concavities a single hull would fill in. It needs the decomp extra; see Installation for the extras matrix. Nearly convex models (the golf ball, the gelatin box) keep a single hull either way. The manifest records which decomposer produced the parts, so installing the extra later, or upgrading CoACD, rebuilds the cache instead of reusing coarser geometry. The function additionally attempts to extract texture.png from meshes/{model_id}/google_16k/textured.glb. Texture extraction is best effort: if a model or the locally installed trimesh does not expose a texture image, the environment still runs with the untextured visual mesh.

Path helpers

These helpers resolve cache paths and never trigger a download. Call ensure_ycb_assets first if the assets may not be present yet; the three collision helpers read the decomposition manifest and raise FileNotFoundError when it is missing.

mass_fraction is the part's volume-weighted share of the object's mass; the fractions sum to 1, so total body mass does not depend on how many parts the decomposition produced.

from so101_nexus import (
    get_ycb_collision_meshes,
    get_ycb_mesh_dir,
    get_ycb_texture_file,
    get_ycb_visual_mesh,
)

mesh_dir = get_ycb_mesh_dir("011_banana")
collision_parts = get_ycb_collision_meshes("011_banana")
visual = get_ycb_visual_mesh("011_banana")
texture = get_ycb_texture_file("011_banana")

Each helper validates model_id against the supported set and raises on an unknown ID.

Custom source repository

Assets are fetched from the ai-habitat/ycb Hugging Face dataset by default. Override the source with SO101_YCB_HF_REPO, which is useful for private mirrors or custom object sets that follow the same directory layout:

export SO101_YCB_HF_REPO="your-org/your-ycb-repo"

Download troubleshooting

Interrupted download. A partial cache directory is not repaired automatically. Delete it and retry:

rm -rf ~/.cache/so101_nexus/ycb/011_banana

Firewall or proxy. Downloads go through the Hugging Face Hub client. Behind a proxy, set the standard HTTPS_PROXY, HF_ENDPOINT, or HF_HUB_DISABLE_TELEMETRY variables.

Simulation assets

These helpers resolve paths to the assets bundled with the package.

get_so101_simulation_dir()

def get_so101_simulation_dir() -> Path

Returns the SO-101 asset directory (SO101/). It holds the URDF/XML that the teleop tooling reads for calibration metadata. The MuJoCo backend does not load this model.

get_so101_mujoco_model_dir() and get_so101_mujoco_model_path()

def get_so101_mujoco_model_dir() -> Path
def get_so101_mujoco_model_path() -> Path

Return the directory and the so101.xml path of the vendored MuJoCo Menagerie model under SO101_menagerie/. This is the model the MuJoCo backend loads.

get_mujoco_ycb_rest_pose()

def get_mujoco_ycb_rest_pose(
    verts: np.ndarray,
    margin: float = 0.002,
) -> tuple[np.ndarray, float]

Computes a stable rest orientation and spawn height for a YCB mesh by rotating its thinnest axis to point up.

Returns (quaternion, spawn_z), where quaternion is a NumPy array in wxyz order and spawn_z is the spawn height in meters.

Next steps