SO101-Nexus
API Reference

Configuration Classes

Reference for all configuration classes.

All configuration classes live in so101_nexus and are plain Python classes with default constructor arguments. They control rendering, robot parameters, reward shaping, observation composition, and environment behavior.

SimSOFollowerConfig.seed controls the seed for the simulator reset during connection. The default is None, which lets Gymnasium select the random state. Teleoperation sets this field from its effective episode seed.

Robot profiles

RobotProfile stores the encoder calibration, coordinate alignment, and effective dynamics of one physical SO-101 arm. RobotConfig(profile=profile) applies its dynamics before MuJoCo execution or Warp conversion. Raw environment actions retain their existing model coordinates. Both backends intersect model joint and actuator limits with the calibrated hardware travel. The LeRobot adapter applies coordinate alignment at the hardware boundary. See Robot identification for automatic collection and fitting.

from so101_nexus import MoveConfig, RobotConfig, RobotProfile

profile = RobotProfile.load("my_robot.json")
config = MoveConfig(robot=RobotConfig(profile=profile))
FieldMeaning
robot_idIdentity of the physical follower, matching the LeRobot robot ID
calibrationOriginal LeRobot motor calibration, with all six SO-101 motor names and IDs
joint_offsets_degFive body-joint offsets: model degrees equal LeRobot degrees plus the offset
gripper_limits_degModel gripper angles at hardware 0 and 100 percent
dynamicsOptional mapping from joint names to JointDynamics values
model_sha256Fingerprint of vendored robot XML and meshes. An empty value permits an unpinned profile.
control_period_sCommand interval for the identified control loop. Sets the physics timing when physics is omitted. An explicit PhysicsConfig.control_period_s must match.
servo_gainsHardware PID coefficients (P, I, D), default (16, 0, 32). Collection requires matching LeRobot coefficients.
schema_versionVersion of the saved JSON schema, currently 1

RobotProfile.from_lerobot(path, robot_id=...) imports an existing calibration and pins the robot asset fingerprint. with_reference_pose(hardware_body_deg=..., model_body_deg=...) returns a profile aligned from five measured body-joint angles. Both reference vectors describe the same physical pose. Encoder calibration alone does not establish a body joint's physical model zero. The guided app requires an operator-confirmed physical reference before it treats the measured overlay as aligned. The default gripper endpoints provide a nominal mapping. Robot-specific gripper alignment requires separate endpoint measurements. save(path) writes JSON atomically, and load(path) validates the saved schema. export_lerobot(path) writes the original encoder calibration as standard LeRobot JSON. Profiles contain immutable snapshots of calibration and dynamics.

to_model_deg(values) and from_model_deg(values) accept NumPy arrays and torch tensors with shape (..., 6). Hardware vectors contain five body angles in degrees and gripper percent. Model vectors contain six angles in degrees. These affine transforms preserve values outside the endpoints. The simulator follower clamps commands to calibrated hardware travel and model limits.

JointDynamics separates actuator derivative gain from passive joint damping:

FieldUnits
armaturekg m²
frictionloss, force_limitN m
kpN m/rad
kd, joint_dampingN m s/rad

These SI gains describe simulation actuators. They are not servo register values. Profile loading does not write hardware registers.

SimSOFollowerConfig(profile_path=...) loads the profile and its encoder calibration without a separate calibration file. It requires use_degrees=True and a matching robot ID. It rejects conflicting calibration files or environment profiles. When physics configuration is omitted, environments and this adapter select the profile command interval. They choose physics substeps that divide this interval without exceeding the default timestep.

Calibration storage

Robot profiles and dynamics calibration are durable application data, separate from disposable caches. The GUI discovers saved profiles independently of its working directory. On Linux, the defaults are ~/.local/share/so101-nexus/profiles/ and ~/.local/share/so101-nexus/calibration/. platformdirs selects the platform's user data directory, including XDG_DATA_HOME on Linux. It uses Application Support on macOS and local application data on Windows.

Environment variableDefault
SO101_NEXUS_DATA_DIRPlatform user data directory for so101-nexus
SO101_NEXUS_PROFILES_DIRprofiles/ under the application data directory
SO101_NEXUS_CALIBRATION_DIRcalibration/ under the application data directory

Overrides must be absolute paths; ~ is expanded. Back up these directories. Their contents include operator-verified physical calibration and recordings. A durable profile includes the encoder calibration needed to reconnect after clearing LeRobot's cache. Reconnect still checks the live motor registers and servo gains before restoring alignment.

The GUI copies legacy runs/calibration files from its launch directory or source checkout on startup. It preserves the originals and stores complete, versioned snapshots under calibration/legacy/. Valid top-level profiles also enter the saved-profile selector. Repeated migration reuses identical snapshots, and filename conflicts preserve both profiles. Dynamics tuning reuses the latest complete recording set matching the connected profile. It checks each recording before restoring it, including its profile identity and training or validation split. For files elsewhere, use Import previous calibration folder under Advanced storage and migration. Import profile copies a selected JSON file into durable storage. Profile downloads export portable files for another machine.

Excitation timing

ExcitationConfig.max_wakeup_lateness_s defaults to 0.25 seconds. It bounds scheduler delays before reading the current motor positions. After a permitted delay, collection reads a fresh pose, checks tracking, and resumes without catch-up. max_overrun_s retains a separate 0.05-second lateness limit after reads and during command writes. max_read_duration_s limits each position read to 0.1 seconds. Actual measurement and command timestamps are saved for fitting.

Excitation collection callbacks

collect_excitation(robot, profile, config, ...) owns the follower connection and disconnection. Its optional callbacks support guided hardware collection:

ArgumentContract
on_readyCallable without arguments. Runs after connection enables holding torque and before trajectory commands. Can wait for operator confirmation.
should_stopCallable without arguments that returns a Boolean. A true result raises InterruptedError before the next hardware read or command.
on_progressCallable with (episode_name, completed_samples, total_samples). Runs after each recorded command.

Callbacks run synchronously. A progress callback must not delay the command loop. Cancellation uses the follower's configured disconnect behavior. The guided app preserves torque on stop and disconnect, then provides a separate motor-release action. The identification workflow describes that hardware procedure.

PhysicsConfig

Pass physics=PhysicsConfig(...) to any task configuration to select physics timing, solver behavior, and optional jaw contact impedance. Both backends use the same native defaults. An omitted physics uses the same numerical configuration as PhysicsConfig().

from so101_nexus import PhysicsConfig, PickConfig

config = PickConfig(physics=PhysicsConfig())

The default configuration uses eight physics substeps per 20 ms command, including reset settling and Warp CUDA graphs. The control frequency remains 50 Hz. The previous native presets used four physics substeps per command. More physics substeps require more computation per command. An explicit control_period_s changes the command frequency and the duration of episodes with a fixed step count.

ParameterDefaultContract
timestep_s0.0025Positive finite physics timestep in seconds. Must divide control_period_s into an integer number of substeps.
control_period_s0.02Positive finite command interval in seconds
integrator"implicit""implicit" or "implicitfast"
solver"Newton"Newton solver
cone"elliptic"Elliptic friction cone
impratio100.0Positive finite friction-to-normal constraint impedance ratio
iterations20Positive integer solver iteration limit
ls_iterations100Positive integer line-search iteration limit
tolerance1e-6Positive finite solver tolerance. Warp requires at least the float32 representation of 1e-6.
noslip_iterations0Nonnegative integer. Warp requires zero.
gripper_solimpNoneOptional tuple (dmin, dmax, width, midpoint, power) for jaw contact impedance

PhysicsConfig is immutable. A different configuration requires a new environment. Warp rejects unsupported NoSlip and solver tolerance values before device conversion. See Physics divergence for backend limits.

gripper_solimp affects only active condim=6 collision geoms directly attached to gripper and moving_jaw_so101_v1. It preserves material friction, solref, geometry, collision masks, mass, actuator limits, and task thresholds. The tuple requires 1e-4 <= dmin <= dmax <= 0.9999, positive width, midpoint strictly between zero and one, and power at least one. Width uses meters. An omitted tuple preserves the native contact impedance. Both backends use solref=(0.005, 1) and solimp=(0.95, 0.999, 0.001, 0.5, 2.0) for the model's gripper collision classes. The shorter contact time constant works with the smaller timestep to reduce penetration under load. The shared model also contains six convex strips that cover the distal pad edges and tips. These strips preserve the original central contact surfaces and open grasp aperture. tests/core/test_jaw_collision_geometry.py checks that the sampled visible pad surfaces extend less than 0.3 mm outside the collider union.

Grasp retention profile

The native configuration is the starting point for cube picks. A separate numerical profile is not necessary. tests/mujoco/test_grasp_dynamics.py and tests/warp/test_grasp_dynamics.py exercise nominal picks on MuJoCo CPU and Warp CUDA. The cases use cube side lengths of 19.05, 24, and 40 mm, plus a 24 mm cube rotated 45 degrees. Each geometry runs with masses of 5, 10, and 100 g. The sequence includes a 10 cm lift, a three-second hold, 4 cm translations and reversals along both horizontal axes, and release. The tests require less than 5 mm of slip and less than 1.5 mm of contact penetration, without loss of the grasp.

MuJoCo uses soft contacts, so zero overlap is not guaranteed. Shallow grasps forced onto the fingertip edges can still slip or lose contact. The tests do not establish grasp retention for every object, mass, approach, or motion. The contact parameters are simulation assumptions, not measured pad material properties. Contact impedance also affects friction regularization. Validate custom configurations on the intended task. Matching configuration does not remove the collision differences between backends.

RenderConfig

Controls the visualization camera geometry and the inherited rendering quality of all cameras. Image resolution and view selection affect visualization only. camera="side" selects an angled tabletop view for rollout videos; it never enters the observation space. Both backends support these views. See Render modes for backend output formats.

from so101_nexus import RenderConfig

render = RenderConfig(width=640, height=480)

# Angled side view for render_mode="rgb_array" videos and the human viewer
render = RenderConfig(
    camera="side",
    side_azimuth_range_deg=(120.0, 200.0),
    side_elevation_range_deg=(-50.0, -20.0),
    side_distance_range=(0.8, 1.2),
)

Constructor Parameters

ParameterTypeDefaultDescription
widthint640Render image width in pixels
heightint480Render image height in pixels
camera"overhead" | "side""overhead"View for rgb_array, depth_array, and the initial human viewpoint. "overhead" looks straight down. "side" selects an angled view.
side_azimuth_degfloat110.0Azimuth of the side view in degrees
side_elevation_degfloat-25.0Elevation of the side view in degrees, in [-90, 0)
side_azimuth_range_degtuple[float, float] | NoneNoneUniform azimuth bounds in degrees, overriding side_azimuth_deg
side_elevation_range_degtuple[float, float] | NoneNoneUniform elevation bounds in degrees within [-90, 0), overriding side_elevation_deg
side_distance_rangetuple[float, float] | NoneNoneUniform positive straight-line distance bounds in meters from the workspace look-at target; None retains automatic workspace framing
qualityRenderingQuality | NoneNoneShared camera quality. None uses RenderingQuality() defaults.

Ranges apply to every task's side view on both backends, including RGB and depth, and the MuJoCo human viewer. Each reset samples the configured ranges using the environment's seeded RNG. Warp samples independently per world and only resamples worlds that reset, including same-step autoresets. The sampled pose stays fixed during the episode; the human viewer still allows manual orbiting. An open viewer adopts the newly sampled pose on reset.

Automatic side framing fits the spawn region and arm working volume at the selected angles and image aspect ratio. The region follows the configured arc, or the square spawn_half_size region for LookAt. The camera fits the projected workspace and arm bounds within the image, choosing the shortest distance that leaves padding around them. It shifts the aim point only as much as needed to fit those bounds. The default angles favor objects in front of the arm. For custom spawn regions that extend behind the robot, the arm can obscure objects even when they fit in the frame. Adjust the angles for that layout or use the overhead view. With randomized angles, each reset computes both the aim point and distance for the sampled view. An explicit side_distance_range overrides the automatic distance.

Endpoints must be finite and ordered. Equal endpoints select a fixed value. Omitted angle ranges retain the corresponding fixed angles, and camera="overhead" ignores all side ranges. With no ranges, the camera uses fixed angles and automatic distance. Explicit distance bounds can crop the workspace or place the camera inside geometry, so choose bounds that suit your scene. These settings do not change camera observations.

RenderingQuality

Scalar and batched MuJoCo environments apply this configuration before they allocate camera framebuffers. To select the opt-in reproducibility profile, pass it through RenderConfig:

from so101_nexus import PickConfig, RenderConfig, RenderingQuality, WristCamera

config = PickConfig(
    render=RenderConfig(quality=RenderingQuality(profile="reproducible")),
    observations=[WristCamera()],
)

WristCamera and OverheadCamera inherit this quality when their quality argument is None. An omitted RenderConfig uses the default profile. The visualization camera also uses the environment's quality.

ProfileMSAA samplesShadowsShadow map size
default8True8192
reproducible4False8192

Constructor Parameters

ParameterTypeDefaultDescription
profile"default" | "reproducible""default"Profile that supplies omitted values
msaa_samplesint | NoneNoneRequested offscreen MSAA sample count. Zero disables MSAA.
shadowsbool | NoneNoneWhether camera scenes render shadows
shadow_map_sizeint | NoneNonePositive shadow map size in pixels

Explicit values override the selected profile, for example RenderingQuality(profile="reproducible", msaa_samples=0). Camera overrides can change shadows independently. MuJoCo stores MSAA and shadow map size on the shared model. Therefore, camera overrides must match the environment's msaa_samples and shadow_map_size. Conflicting values raise ValueError.

RenderingQuality is immutable. Quality changes require a new environment. Changing the model's sample request after framebuffer allocation does not change the allocated sample count. Warp uses a separate ray renderer without equivalent MSAA or shadow map controls. Warp rejects unsupported quality configurations rather than silently applying this MuJoCo profile.

The reproducible profile retains anti-aliasing. It is a tested workaround without a guarantee of determinism across drivers, GPUs, or render histories. The check_msaa_no_shadow.py probe in the companion so101-tasks project tested NVIDIA RTX 5090, driver 580.178.04, EGL, and MuJoCo 3.14.0. Other installations require their own repeated-state image and policy tests.

env.unwrapped.rendering_metadata reports requested quality, allocated sample counts, camera dimensions, and renderer runtime details. Read it after each relevant camera renders so lazy camera allocations appear in the report. Store it with evaluation results and experiment configuration. See Training reproducibility for baseline and render cadence requirements.


Pose

Defines a named robot arm configuration with fixed and free joints. See Customization for a conceptual overview.

from so101_nexus import Pose

pose = Pose(
    name="custom",
    shoulder_pan_deg=(-110.0, 110.0),  # free: sampled uniformly
    shoulder_lift_deg=-90.0,            # fixed
    elbow_flex_deg=90.0,                # fixed
    wrist_flex_deg=37.8,                # fixed
    wrist_roll_deg=(-157.0, 163.0),     # free
    gripper_deg=(-10.0, 100.0),         # free
)

Constructor Parameters

ParameterTypeDescription
namestrHuman-readable identifier
shoulder_pan_degfloat | tuple[float, float]Shoulder pan angle or range (degrees)
shoulder_lift_degfloat | tuple[float, float]Shoulder lift angle or range (degrees)
elbow_flex_degfloat | tuple[float, float]Elbow flex angle or range (degrees)
wrist_flex_degfloat | tuple[float, float]Wrist flex angle or range (degrees)
wrist_roll_degfloat | tuple[float, float]Wrist roll angle or range (degrees)
gripper_degfloat | tuple[float, float]Gripper angle or range (degrees)

Methods

MethodReturnsDescription
sample(rng)tuple[float, ...]Sample concrete joint angles in degrees
sample_rad(rng)tuple[float, ...]Sample concrete joint angles in radians

Built-in Poses

NameConstantDescription
"rest"REST_POSEArm curled in rest position, free shoulder_pan/wrist_roll/gripper
"extended"EXTENDED_POSEArm extended forward, free shoulder_pan/wrist_roll/gripper

Access via the POSES dict or by constant:

from so101_nexus import POSES, REST_POSE

pose = POSES["rest"]   # same as REST_POSE

RobotConfig

Controls the arm rest pose, grasping thresholds, and the end-effector solver.

from so101_nexus import RobotConfig

robot = RobotConfig(rest_qpos_deg=(0.0, -90.0, 90.0, 37.82, 0.0, -63.03))

Constructor Parameters

ParameterTypeDefaultDescription
rest_qpos_degtuple[float, ...](0.0, -90.0, 90.0, 37.8152144786, 0.0, -63.0253574644)Rest joint positions in degrees, one per joint
init_posestr | Pose | NoneNoneInitial pose for resets. String looks up from POSES, Pose instance used directly, None uses legacy rest + noise.
grasp_force_thresholdfloat0.5Minimum contact normal force (N) for a finger contact to count toward grasp detection
grasp_opposing_normal_thresholdfloat0.3How strongly the two finger sets must oppose for GraspState to fire: the force-weighted mean inward contact normals must satisfy dot(n_gripper, n_jaw) <= -threshold. Must be in [-1, 1]; -1.0 disables the test and accepts any two-sided contact
static_vel_thresholdfloat0.2Maximum velocity to consider the arm stationary
ee_orientation_weightfloat0.01Relative weight of the rotational error in the end-effector modes' inverse kinematics. Must be in (0, 1]. See Control Modes.
ee_delta_action_scaletuple[float, ...](0.02, 0.02, 0.02, 0.1, 0.1, 0.1, 0.2)Physical scale of a +/-1 pd_ee_delta_pose action: (x, y, z) in meters, (wx, wy, wz) in radians, gripper in radians. Seven positive entries.

Properties and methods

PropertyTypeDescription
rest_qpos_radtuple[float, ...]rest_qpos_deg converted to radians
rest_qpostuple[float, ...]Alias for rest_qpos_rad
resolve_pose()Pose | NoneReturns the resolved Pose object, or None

RobotCameraPreset

Robot-specific camera and mounting parameters for SO-100 and SO-101. All fields are required (no defaults); wrist_cam_euler_center_rad and wrist_cam_euler_noise_rad expose the Euler angles in radians.

ParameterTypeDescription
base_quattuple[float, float, float, float]Base orientation quaternion (w, x, y, z)
sensor_cam_eye_postuple[float, float, float]Sensor camera eye position
sensor_cam_target_postuple[float, float, float]Sensor camera target position
human_cam_eye_postuple[float, float, float]Human camera eye position
human_cam_target_postuple[float, float, float]Human camera target position
wrist_camera_mount_linkstrLink name for wrist camera mounting
wrist_cam_pos_centertuple[float, float, float]Center position for the wrist camera
wrist_cam_pos_noisetuple[float, float, float]Position noise applied to the wrist camera
wrist_cam_euler_center_degtuple[float, float, float]Center wrist camera Euler angles in degrees
wrist_cam_euler_noise_degtuple[float, float, float]Euler angle noise applied to the wrist camera

RewardConfig

Defines reward component weights and computes the shaped reward.

from so101_nexus import RewardConfig

reward = RewardConfig(reaching=0.25, grasping=0.25, task_objective=0.25, completion_bonus=0.25)

Constructor Parameters

ParameterTypeDefaultDescription
reachingfloat0.25Weight for the reaching component. Potential-shaped delta for pick-lift/pick-and-place; raw progress value for Touch/Move/LookAt.
graspingfloat0.25Weight for the grasping component. Potential-shaped delta for pick-lift/pick-and-place.
task_objectivefloat0.25Weight for the task-specific objective. Potential-shaped delta for pick-lift/pick-and-place.
completion_bonusfloat0.25Weight for the task completion bonus
action_delta_penaltyfloat0.0Penalty coefficient on L2 norm of consecutive action deltas
energy_penaltyfloat0.0Penalty coefficient on L2 norm of the action vector
tanh_shaping_scalefloat5.0Scale factor for tanh reward shaping
velocity_shaping_scalefloat15.0Scale factor for tanh velocity shaping in PickAndPlace and StackCube task potentials

A "potential-shaped delta" pays the change in progress since the previous step rather than the raw value, so dwelling at a fixed state (e.g. holding an object without lifting it) earns no further reward. The implementation is so101_nexus.rewards.potential_shaping; see Rewards for which tasks use it.

Validation

The four base weights (reaching + grasping + task_objective + completion_bonus) must sum to 1.0. Construction raises an error if this constraint is violated. The penalty coefficients are separate additive terms and are not included in the sum constraint.

Methods

compute()

def compute(
    reach_progress: float,
    is_grasped: bool,
    task_progress: float,
    is_complete: bool,
    action_delta_norm: float = 0.0,
    energy_norm: float = 0.0,
) -> float

Computes the weighted reward from per-component progress values.

ParameterTypeDefaultDescription
reach_progressfloatrequiredProgress toward the target (0 to 1)
is_graspedboolrequiredWhether the object is currently grasped
task_progressfloatrequiredProgress on the task objective (0 to 1)
is_completeboolrequiredWhether the task is fully complete
action_delta_normfloat0.0Norm of the action delta for penalty
energy_normfloat0.0Norm of energy usage for penalty

Returns a single float reward value.


EnvironmentConfig

Base configuration for all environments. Task-specific configs extend this class.

from so101_nexus import EnvironmentConfig

config = EnvironmentConfig(obs_mode="state")

Constructor Parameters

ParameterTypeDefaultDescription
renderRenderConfig | NoneNoneVisualization camera geometry and inherited quality for all cameras. Uses RenderConfig() defaults when None.
rewardRewardConfig | NoneNoneReward weights. Uses RewardConfig() defaults when None.
robotRobotConfig | NoneNoneRobot settings. Uses RobotConfig() defaults when None.
ground_colorsColorConfig"gray"Ground plane color(s)
reset_settle_framesint5No-op environment frames advanced after reset before returning the first observation
goal_threshfloat0.025Distance threshold for goal completion (meters)
spawn_half_sizefloat0.05Half-size of object spawn region
spawn_centertuple[float, float](0.15, 0.0)Center of the spawn region (x, y)
spawn_min_radiusfloat0.10Minimum spawn distance from the robot base
spawn_max_radiusfloat0.30Maximum spawn distance from the robot base
spawn_angle_half_range_degfloat90.0Half-range of spawn angle in degrees
obs_modeObsMode"state"Observation mode. "state" returns all selected components. "visual" returns cameras and joint positions. info["privileged_state"] has the full selected state.
robot_colorsColorConfig"yellow"Plastic arm colors. MuJoCo samples on each seeded reset; Warp uses the first color because materials are shared across worlds.
robot_init_qpos_noisefloat0.02Noise added to initial joint positions
terminate_on_successboolTrueEnd the episode when the success predicate becomes true. When False, only the step cap truncates the episode.
observationslist[Observation] | NoneNoneObservation components to include. Task-specific configs provide defaults when None.

Validation

  • obs_mode must be "state" or "visual"
  • reset_settle_frames must be a nonnegative integer
  • When obs_mode="visual", observations must contain at least one camera component (WristCamera or OverheadCamera)
  • Duplicate camera component types are not allowed

Episode Length

max_episode_steps is owned by the Gymnasium registration, not the config. Set it per env at construction, the same way for every task:

import gymnasium as gym

# MuJoCo (single env): applied via the TimeLimit wrapper
env = gym.make("MuJoCoTouch-v1", config=config, max_episode_steps=256)

# Warp (batched): forwarded to the vector env, which truncates internally
envs = gym.make_vec("WarpTouch-v1", num_envs=4, device="cuda", config=config, max_episode_steps=256)

Registered defaults: PickLift, PickAndPlace, and StackCube 1024, Touch 512, LookAt and Move 256. They apply to both the MuJoCo*-v1 and Warp*-v1 ids.


Observation Components

Observation components are lightweight descriptor classes passed via the observations parameter. See Observations for a conceptual overview.

State Components

ClassSizeDescription
JointPositions()6Current joint angles
JointVelocities()6Current joint angular velocities (rad/s)
JointEfforts()6Actuator force on each joint (N*m)
GripperContactForce()3World-frame resultant contact force on the fingers (N)
EndEffectorPose()7TCP position + quaternion
TargetOffset()3Vector to the goal: goal minus object in manipulation tasks, goal minus TCP elsewhere
GazeDirection()3Unit vector from the wrist camera toward the target object
GazeState()1Binary in-frame flag (target object inside the wrist camera's field of view)
GraspState()1Binary grasp flag (two-sided, opposing-normal contact)
ObjectPose()7Object position + quaternion
ObjectVelocity()6Object linear (world frame) + angular (body frame) velocity
ObjectOffset()3Vector from gripper to object
TargetPosition()3Absolute goal position

Camera Components

Both camera classes accept the keyword-only modalities parameter. Use ("rgb",) for RGB, ("depth",) for depth, or ("rgb", "depth") for both. The default is ("rgb",). Empty, duplicate, and unknown modalities raise ValueError. See Depth observations for output keys, units, shapes, and clipping behavior. Both also accept quality for a rendering quality override.

WristCamera

from so101_nexus import WristCamera

cam = WristCamera(width=224, height=224)
ParameterTypeDefaultDescription
widthint640Image width in pixels
heightint480Image height in pixels
fov_deg_rangetuple[float, float](60.0, 90.0)FOV randomization range in degrees
pitch_deg_rangetuple[float, float](-34.4, 0.0)Pitch randomization range in degrees
pos_x_noisefloat0.005Position noise along x-axis
pos_y_centerfloat0.04Nominal y-offset from the wrist
pos_y_noisefloat0.01Position noise along y-axis
pos_z_centerfloat-0.04Nominal z-offset from the wrist
pos_z_noisefloat0.01Position noise along z-axis
modalitiestuple[Literal["rgb", "depth"], ...]("rgb",)Requested camera outputs
qualityRenderingQuality | NoneNoneOptional override; None inherits the environment's render quality

Properties: fov_rad_range, pitch_rad_range (converted to radians).

OverheadCamera

from so101_nexus import OverheadCamera

cam = OverheadCamera(width=320, height=240, fov_deg=45.0)
ParameterTypeDefaultDescription
widthint640Image width in pixels
heightint480Image height in pixels
fov_degfloat45.0Vertical field-of-view in degrees
modalitiestuple[Literal["rgb", "depth"], ...]("rgb",)Requested camera outputs
qualityRenderingQuality | NoneNoneOptional override; None inherits the environment's render quality

PickConfig

Extends EnvironmentConfig with parameters for pick and lift tasks.

from so101_nexus import PickConfig, CubeObject, YCBObject

config = PickConfig(
    objects=[CubeObject(color="blue"), YCBObject("009_gelatin_box")],
    min_distractors=0,
    max_distractors=1,
)

Additional Parameters

These are in addition to all EnvironmentConfig parameters.

ParameterTypeDefaultDescription
objectslist[SceneObject] | SceneObject | NoneNoneObjects to pick. A single object is auto-wrapped. Defaults to [CubeObject()] when None.
min_distractorsint | NoneNoneInclusive minimum number of distractors per episode
max_distractorsint | NoneNoneInclusive maximum number of distractors per episode
n_distractorsint | NoneNoneFixed-count shorthand; omitted bounds inherit this value, or zero
lift_thresholdfloat0.05Height above the table to count as lifted (meters)
max_goal_heightfloat0.08Maximum target lift height (meters)
min_object_separationfloat0.04Minimum distance between spawned objects (meters)

Each reset samples an integer uniformly from min_distractors through max_distractors, including both endpoints, using the environment's seeded RNG. Equal bounds give a fixed count. Both default to zero when no count is supplied. The object pool needs at least max_distractors + 1 entries because one is the target. Bounds must be nonnegative integers with the minimum no larger than the maximum. n_distractors=k remains a shorthand for two bounds of k; when combined with explicit bounds, it must equal the maximum. Its stored value is the resolved maximum, rather than an episode's sampled count.

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), GraspState(), GazeState(), ObjectPose(), ObjectOffset()] (31 dimensions).


PickReturnConfig

Extends PickConfig for the pick-and-return task.

from so101_nexus import CubeObject, PickReturnConfig, RobotConfig

config = PickReturnConfig(
    objects=[CubeObject(color="blue"), CubeObject(color="red")],
    return_threshold_deg=5.0,
    robot=RobotConfig(rest_qpos_deg=(0.0, -90.0, 90.0, 37.8, 0.0, 0.0)),
)
ParameterTypeDefaultDescription
return_threshold_degfloat5.0Maximum absolute error of each arm joint from the configured rest posture, in degrees

return_threshold_deg, lift_threshold, and max_goal_height must be finite and positive. The rest target uses the first five angles in robot.rest_qpos_deg. robot.static_vel_threshold controls final arm staticness. reward.velocity_shaping_scale shapes the final slowdown and is active for this task. The default reward weights are reaching=0.15, grasping=0.15, task_objective=0.3, and completion_bonus=0.4. Passing a RewardConfig replaces these weights with your configuration. max_goal_height controls the curvature of lift shaping in meters. Lift credit is normalized to one at lift_threshold and saturates there. See task details for reward stages and default observations. An explicit observations list is preserved, including an empty list.

PickAndPlaceConfig

Extends EnvironmentConfig with parameters for pick-and-place tasks. The carried object is chosen per episode from an object pool: by default one cube per color in cube_colors, or pass objects to carry YCBObject / MeshObject instead.

from so101_nexus import PickAndPlaceConfig, YCBObject

# Default cube path (carries a colored cube onto the disc):
config = PickAndPlaceConfig(cube_colors="blue", target_colors="green")

# Default cubes with a float millimeter side length:
millimeter_config = PickAndPlaceConfig(cube_side_length_mm=25.4)

# Object-pool path (carries a YCB object):
config = PickAndPlaceConfig(objects=[YCBObject("009_gelatin_box")], target_colors="green")

Additional Parameters

These are in addition to all EnvironmentConfig parameters.

ParameterTypeDefaultDescription
objectslist[SceneObject] | SceneObject | NoneNoneCarried-object pool. None derives a cube pool from the cube sugar below. Passing this together with any non-default cube sugar raises ValueError.
target_colorsColorConfig"blue"Color(s) for the target disc
target_disc_radiusfloat0.05Radius of the target disc (meters)
min_object_target_separationfloat | NoneNoneMinimum object/disc spawn separation (meters); None falls back to min_cube_target_separation
cube_colorsColorConfig"red"Color(s) for the default cube pool (compatibility sugar)
cube_half_sizefloat | NoneNoneLegacy half-size of the default cube(s) in meters. Defaults to 0.0125 when neither size input is set.
cube_side_length_mmfloat | NoneNoneFull side length of the default cube(s) in millimeters. Do not combine with cube_half_size.
cube_massfloat0.01Mass of the default cube(s) (kg)
min_cube_target_separationfloat0.0375Deprecated alias for min_object_target_separation
object_static_lin_thresholdfloat0.01Maximum carried-object linear speed (m/s) for success
object_static_ang_thresholdfloat0.5Maximum carried-object angular speed (rad/s) for success
distractorslist[SceneObject] | SceneObject | NoneNonePool of non-target objects distractors are drawn from. Defaults to green, yellow, and purple cubes sharing cube_half_size / cube_mass
min_distractors, max_distractorsint | NoneNoneInclusive per-episode bounds; see distractor sampling. The maximum cannot exceed the distractors pool size
n_distractorsint | NoneNoneFixed-count shorthand, as in PickConfig
min_object_separationfloat0.04Minimum spawn separation between the carried object and the distractors (meters), on top of their bounding radii

Distractor slots are only compiled when max_distractors > 0, so the default scene holds exactly one object. Distractors keep min_object_target_separation from the disc, so clutter never spawns on the goal. A distractor cube whose color is also a carried cube color warns, because the task description names the carried object by color alone.

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), GraspState(), GazeState(), TargetPosition(), ObjectPose(), ObjectVelocity(), ObjectOffset(), TargetOffset()] (43 dimensions).

PickAndPlaceV2Config

This config extends PickAndPlaceConfig for the v2 center-placement contract. Use it with the v2 environment IDs. V1 and v2 reject the other version's config.

from so101_nexus import PickAndPlaceV2Config, YCBObject

config = PickAndPlaceV2Config(
    objects=[YCBObject("011_banana")],
    target_disc_radius=0.05,
)
ParameterTypeDefaultDescription
placement_modeLiteral["center"]"center"Only center placement is supported
target_disc_radiusfloat0.05Visible and evaluated acceptance radius in meters
goal_threshfloatSame as radiusMust equal target_disc_radius, otherwise construction fails
footprint_scanlinesint256Midpoint rows for projected triangle-union integration
support_min_weight_fractionfloat0.9Required upward table force as a fraction of compiled object weight
support_force_tolerancefloat0.001Absolute floor for robot and other-object force tolerance, in newtons
support_relative_force_tolerancefloat0.01Relative floor for the same tolerance, as a fraction of object weight
placement_dwell_timefloat0.2Consecutive seconds of physical support and settled motion
object_static_lin_thresholdfloat0.01Maximum center-of-mass linear speed in m/s
object_static_ang_thresholdfloat0.5Maximum angular speed in rad/s

The resolved contact-force tolerance is the larger of the absolute and relative values. All numerical thresholds must be finite. The radius, dwell time, and minimum weight fraction must be positive. The minimum weight fraction must not exceed one. The relative force tolerance must be less than one. The scanline count must be a positive integer. placement_contract returns JSON-safe evaluator metadata without simulator state.


StackCubeConfig

Extends EnvironmentConfig with parameters for the stack-cube task. Two cubes are spawned every episode: cube A (picked up and stacked) and cube B (the stationary base). Colors default to disjoint values so the two cubes are never the same color out of the box.

from so101_nexus import StackCubeConfig

config = StackCubeConfig(cube_a_colors="green", cube_b_colors="purple")

Additional Parameters

These are in addition to all EnvironmentConfig parameters.

ParameterTypeDefaultDescription
cube_a_colorsColorConfig"red"Color(s) for cube A, the cube that gets picked up and stacked
cube_b_colorsColorConfig"blue"Color(s) for cube B, the stationary stacking base
cube_half_sizefloat | NoneNoneLegacy half-extent of each cube in meters. Defaults to 0.0125 when neither size input is set.
cube_side_length_mmfloat | NoneNoneFull side length of each cube in millimeters. Do not combine with cube_half_size.
cube_massfloat0.01Mass of each cube (kg); both cubes share the same mass
min_cube_separationfloat0.04Minimum spawn separation between the two cubes (meters), on top of their combined bounding radii
stack_alignment_marginfloat0.005Alignment tolerance (meters) for the stacked success check
cube_static_lin_thresholdfloat0.01Maximum linear speed (m/s) at which cube A still counts as static for success
cube_static_ang_thresholdfloat0.5Maximum angular speed (rad/s) at which cube A still counts as static for success
distractorslist[SceneObject] | SceneObject | NoneNonePool of non-target objects distractors are drawn from. Defaults to green, yellow, and purple cubes sharing cube_half_size / cube_mass
min_distractors, max_distractorsint | NoneNoneInclusive per-episode bounds; see distractor sampling. The maximum cannot exceed the distractors pool size
n_distractorsint | NoneNoneFixed-count shorthand, as in PickConfig

Passing overlapping cube_a_colors/cube_b_colors pools warns (the two cubes may be the same color in some episodes).

Distractor slots are only compiled when max_distractors > 0, so the default scene holds exactly two cubes. A distractor cube whose color is also a cube A/B color warns, because the task description names cubes by color alone.

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), GraspState(), GazeState(), ObjectPose(), ObjectVelocity(), ObjectOffset(), TargetPosition(), TargetOffset()] (43 dimensions).


TouchConfig

Extends PickConfig with the touch task. The touch target can be any cube, YCB object, or mesh from the object pool.

from so101_nexus import TouchConfig

config = TouchConfig(touch_margin=0.03)

Additional Parameters

This is in addition to all PickConfig object-pool parameters (objects, min_distractors, max_distractors, n_distractors, min_object_separation) and the inherited EnvironmentConfig parameters.

ParameterTypeDefaultDescription
touch_marginfloat0.03Clearance added to the target object bounding radius; success fires when the TCP is within bounding_radius + touch_margin of the object center

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), GraspState(), GazeState(), ObjectPose(), ObjectOffset()] (31 dimensions).


LookAtConfig

Extends EnvironmentConfig with parameters for the look-at primitive task.

from so101_nexus import LookAtConfig, CubeObject

config = LookAtConfig(
    objects=CubeObject(color="blue"),
    fov_deg=60.0,
)

Additional Parameters

ParameterTypeDefaultDescription
objectslist[SceneObject] | SceneObject | NoneNoneTarget object(s). CubeObject, CylinderObject, SphereObject, and PyramidObject are supported. Defaults to [CubeObject()].
fov_degfloat | NoneNoneWrist-camera vertical FOV in degrees; success = target within fov_deg / 2 of the optical axis. None reads the live camera FOV.

The MuJoCoLookAt-v1 and WarpLookAt-v1 registrations default max_episode_steps to 256; override it at construction via gym.make(..., max_episode_steps=N) (MuJoCo) or gym.make_vec(..., max_episode_steps=N) (Warp).

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), GazeDirection(), GazeState()] (23 dimensions).


MoveConfig

Extends EnvironmentConfig with parameters for the directional move primitive task.

from so101_nexus import MoveConfig

config = MoveConfig(direction="up", target_distance=0.10)

Additional Parameters

ParameterTypeDefaultDescription
directionMoveDirection"up"Cardinal direction: "up", "down", "left", "right", "forward", "backward"
target_distancefloat0.10Distance in meters to travel from the initial TCP position
success_thresholdfloat0.01Max residual distance (m) to count as success

The MuJoCoMove-v1 and WarpMove-v1 registrations default max_episode_steps to 256; override it at construction via gym.make(..., max_episode_steps=N) (MuJoCo) or gym.make_vec(..., max_episode_steps=N) (Warp).

Default Observations

When observations is not specified: [JointPositions(), JointVelocities(), EndEffectorPose(), TargetOffset()] (22 dimensions).

On this page