Scene Objects and Assets

Scene objects define what the robot interacts with. All object types live in so101_nexus and share the abstract SceneObject base class.

from so101_nexus import (
    CubeObject,
    CylinderObject,
    GSOObject,
    MeshObject,
    PyramidObject,
    SphereObject,
    YCBObject,
)

All concrete object types work on the MuJoCo and MuJoCo Warp backends. Both backends build dynamic-object scenes through so101_nexus.object_slots.build_object_scene_xml.

SceneObject (abstract)Copy Anchor Link

Base class for all scene objects. Subclasses must implement __repr__(), which returns a natural-language description of the object (used in logging and in the task string). SceneObject cannot be instantiated directly; use one of the concrete subclasses below.

CubeObjectCopy Anchor Link

A solid-color cube with configurable size and mass. Set side_length_mm as a float in millimeters. The existing half_size argument remains available for meter-based code.

from so101_nexus import CubeObject

cube = CubeObject(side_length_mm=25.4, mass=0.015, color="blue")
repr(cube)  # "blue cube"

__repr__() returns "{color} cube", for example "red cube".

CylinderObject, SphereObject, and PyramidObjectCopy Anchor Link

These solid-color primitives accept mass and color like CubeObject. CylinderObject and SphereObject use diameter_mm. PyramidObject uses side_length_mm. Each primitive keeps its existing half_size argument for meter-based code.

from so101_nexus import CylinderObject, PyramidObject, SphereObject

cylinder = CylinderObject(diameter_mm=25.4, color="blue")
sphere = SphereObject(diameter_mm=12.7, color="green")
pyramid = PyramidObject(side_length_mm=25.4, color="yellow")

The cylinder height equals its diameter. The square pyramid height equals its base side length. __repr__() returns "{color} cylinder", "{color} sphere", or "{color} pyramid". The valid colors are the same ColorName values that CubeObject accepts.

YCBObjectCopy Anchor Link

A YCB benchmark object loaded from mesh files. Assets are downloaded on first use and cached (see YCB assets).

from so101_nexus import YCBObject

banana = YCBObject("011_banana")
repr(banana)  # "banana"

__repr__() returns the human-readable name from the YCB_OBJECTS dictionary.

Model IDsCopy Anchor Link

model_id must be one of the ten supported IDs, which are also the members of the YcbModelId type alias:

Every rest pose above is settle-tested against real MuJoCo physics (see get_mujoco_ycb_rest_pose()); a few needed a corrected orientation because the default "thinnest axis up" guess tips over under real dynamics. A handful of objects (both YCB and GSO) also have a grasp advisory: a geometric screen found no tested cross-section that fits inside the SO-101 gripper's aperture, so the object is likely too wide to close on with this arm. This is advisory only, not a construction-time error - spot-check via teleoperation or a policy rollout before relying on a flagged object for a real grasp task. The current list is in so101_nexus.ycb_geometry.GRASP_ADVISORY.

Passing any other model_id raises an error. The same mapping is available at runtime:

from so101_nexus import YCB_OBJECTS

for model_id, name in YCB_OBJECTS.items():
    print(f"{model_id}: {name}")

GSOObjectCopy Anchor Link

A Google Scanned Objects (GSO) object loaded from mesh files, sharing YCBObject's convex-hull collision pipeline. Assets are downloaded on first use and cached (see GSO assets).

from so101_nexus import GSOObject

clamp = GSOObject("Pony_C_Clamp_1440")
repr(clamp)  # "C-clamp"

__repr__() returns the human-readable name from the GSO_OBJECTS dictionary.

Unlike YCB, GSO ships no benchmark-measured masses. Each model_id's default mass in GSO_MASSES is a hand estimate: convex-hull volume from the mirrored scan times an assumed effective density for the filled/packaged object, rounded to the nearest 5 g. Pass mass_override to replace it with a measured value.

Model IDsCopy Anchor Link

model_id must be one of the twelve supported IDs, which are also the members of the GsoModelId type alias:

Passing any other model_id raises an error. The same mapping is available at runtime:

from so101_nexus import GSO_OBJECTS

for model_id, name in GSO_OBJECTS.items():
    print(f"{model_id}: {name}")

See the YCB section above for the grasp advisory note; it applies to GSO objects too.

MeshObjectCopy Anchor Link

A custom object defined by your own collision and visual mesh files.

from so101_nexus import MeshObject

obj = MeshObject(
    collision_mesh_path="/path/to/collision.obj",
    visual_mesh_path="/path/to/visual.obj",
    mass=0.05,
    name="custom widget",
    scale=0.8,
)
repr(obj)  # "custom widget"

__repr__() returns the name provided at construction.

YCB assetsCopy Anchor Link

YCB mesh assets are downloaded from the Hugging Face Hub on first use and cached at:

~/.cache/so101_nexus/ycb/{repo_key}/{revision}/{model_id}/

Each model directory holds visual.obj, the convex collision parts under collision_v3/ (collision_000.obj and more), and a manifest.json file. The directory also holds texture.png when the texture extraction succeeds.

Creating a YCBObject and stepping an environment downloads whatever is missing. You can also trigger the download explicitly:

from so101_nexus import ensure_ycb_assets

path = ensure_ycb_assets("011_banana")

ensure_ycb_assets()Copy Anchor Link

def ensure_ycb_assets(model_id: str) -> Path

Downloads the mesh assets for model_id if they are not already cached, then returns the cache directory.

visual.obj and the collision_v3/ parts are the required geometry cache. The generator uses 20,000 deterministic surface samples to measure the convex hull against the visual scan. It keeps one hull when the p95 error is at most 3 mm and the maximum error is at most 10 mm. This gate keeps the gelatin box, large marker, and golf ball as one hull.

The other models use the collision-aware CoACD decomposition. CoACD has no object-level part limit. It limits each part to 128 vertices, which matches the MuJoCo mesh hull limit. The audited models use 3 to 19 parts. Their collision surfaces have 2.07 mm to 3.02 mm of p95 error against the visual scans.

The manifest records the repository commit, source hash, collision-part hashes, CoACD version, configuration, quality gates, and part sizes. The cache rebuilds after the source, a collision part, or a generation input changes. An installation without the decomp extra can use an existing decomposition. Otherwise, it falls back to one convex hull. See Installation for the extras matrix.

The function also attempts to extract texture.png from meshes/{model_id}/google_16k/textured.glb. Texture extraction is best effort. If no texture image is available, the environment uses the untextured visual mesh.

Path helpersCopy Anchor Link

These helpers resolve cache paths and never trigger a download. Call ensure_ycb_assets first if the assets may not be present yet; the three collision helpers read the decomposition manifest and raise FileNotFoundError when it is missing.

mass_fraction is the part's volume-weighted share of the object's mass; the fractions sum to 1, so total body mass does not depend on how many parts the decomposition produced.

from so101_nexus import (
    get_ycb_collision_meshes,
    get_ycb_mesh_dir,
    get_ycb_texture_file,
    get_ycb_visual_mesh,
)

mesh_dir = get_ycb_mesh_dir("011_banana")
collision_parts = get_ycb_collision_meshes("011_banana")
visual = get_ycb_visual_mesh("011_banana")
texture = get_ycb_texture_file("011_banana")

Each helper validates model_id against the supported set and raises on an unknown ID.

Custom source repositoryCopy Anchor Link

The default source is a pinned commit of ai-habitat/ycb. Each repository and commit has a separate cache. Legacy caches remain untouched. For a custom mirror, set both variables before importing the library. Replace the revision placeholder with the mirror's full 40-character commit SHA:

export SO101_YCB_HF_REPO="your-org/your-ycb-repo"
export SO101_YCB_HF_REVISION="<40-character-commit-sha>"

Download troubleshootingCopy Anchor Link

Interrupted download. A partial cache directory is not repaired automatically. Delete it and retry:

rm -rf ~/.cache/so101_nexus/ycb/<repo_key>/<revision>/011_banana

Firewall or proxy. Downloads go through the Hugging Face Hub client. Behind a proxy, set the standard HTTPS_PROXY, HF_ENDPOINT, or HF_HUB_DISABLE_TELEMETRY variables.

GSO assetsCopy Anchor Link

GSO mesh assets are downloaded from a Hugging Face mirror on first use and cached at:

~/.cache/so101_nexus/gso/{repo_key}/{revision}/{model_id}/

The cache layout, manifest format, CoACD decomposition, and texture extraction are identical to YCB assets above - both datasets share the same generator (so101_nexus.mesh_assets), just with a different source repository and cache subdirectory.

Creating a GSOObject and stepping an environment downloads whatever is missing. You can also trigger the download explicitly:

from so101_nexus import ensure_gso_assets

path = ensure_gso_assets("Pony_C_Clamp_1440")

ensure_gso_assets()Copy Anchor Link

def ensure_gso_assets(model_id: str) -> Path

Downloads the mesh assets for model_id if they are not already cached, then returns the cache directory.

Path helpersCopy Anchor Link

These mirror the YCB path helpers: they resolve cache paths and never trigger a download. Call ensure_gso_assets first if the assets may not be present yet.

from so101_nexus import (
    get_gso_collision_meshes,
    get_gso_mesh_dir,
    get_gso_texture_file,
    get_gso_visual_mesh,
)

mesh_dir = get_gso_mesh_dir("Pony_C_Clamp_1440")
collision_parts = get_gso_collision_meshes("Pony_C_Clamp_1440")
visual = get_gso_visual_mesh("Pony_C_Clamp_1440")
texture = get_gso_texture_file("Pony_C_Clamp_1440")

Each helper validates model_id against the supported set and raises on an unknown ID.

Custom source repositoryCopy Anchor Link

The default source is a pinned commit of johnsutor/gso-so101-nexus. Each repository and commit has a separate cache. Legacy caches remain untouched. For a custom mirror, set both variables before importing the library. Replace the revision placeholder with the mirror's full 40-character commit SHA:

export SO101_GSO_HF_REPO="your-org/your-gso-repo"
export SO101_GSO_HF_REVISION="<40-character-commit-sha>"

Download troubleshootingCopy Anchor Link

Interrupted download. A partial cache directory is not repaired automatically. Delete it and retry:

rm -rf ~/.cache/so101_nexus/gso/<repo_key>/<revision>/Pony_C_Clamp_1440

Firewall or proxy. Downloads go through the Hugging Face Hub client. Behind a proxy, set the standard HTTPS_PROXY, HF_ENDPOINT, or HF_HUB_DISABLE_TELEMETRY variables.

Simulation assetsCopy Anchor Link

These helpers resolve paths to the assets bundled with the package.

get_so101_simulation_dir()Copy Anchor Link

def get_so101_simulation_dir() -> Path

Returns the SO-101 asset directory (SO101/). It holds the URDF/XML that the teleop tooling reads for calibration metadata. The MuJoCo backend does not load this model.

get_so101_mujoco_model_dir() and get_so101_mujoco_model_path()Copy Anchor Link

def get_so101_mujoco_model_dir() -> Path
def get_so101_mujoco_model_path() -> Path

Return the directory and the so101.xml path of the vendored MuJoCo Menagerie model under SO101_menagerie/. This is the model the MuJoCo backend loads.

get_mujoco_ycb_rest_pose()Copy Anchor Link

def get_mujoco_ycb_rest_pose(
    verts: np.ndarray,
    margin: float = 0.002,
    model_id: str | None = None,
) -> tuple[np.ndarray, float]

Computes a stable rest orientation and spawn height for a scanned mesh (YCB or GSO). model_id looks up a settle-test-validated correction in POSE_OVERRIDES first; without a match, the mesh's thinnest axis is rotated to point up as a guess.

Returns (quaternion, spawn_z), where quaternion is a NumPy array in wxyz order and spawn_z is the spawn height in meters.

Next stepsCopy Anchor Link