Environments
MuJoCoLookAt-v1
Orient the TCP toward an object using the MuJoCo backend.
MuJoCoLookAt-v1
Orient the robot's end-effector so that it points at a target object.
Overview
| Property | Value |
|---|---|
| Environment ID | MuJoCoLookAt-v1 |
| Backend | MuJoCo |
| Max episode steps | 256 |
| Config class | LookAtConfig (extends EnvironmentConfig) |
| Task description | "Look at the {object repr}." |
Configuration
LookAtConfig extends EnvironmentConfig with the following fields:
| Field | Type | Default | Description |
|---|---|---|---|
objects | list | -- | List of objects to look at (supports CubeObject) |
fov_deg | float | None | None | Vertical field of view (degrees) of the wrist camera. Succeeds when the target lies within the camera's field of view (angle between the wrist-camera optical axis and the direction to the object is at most fov_deg / 2). When None, the FOV is read from the actual wrist camera at runtime, so the success band adapts to any camera configuration or a wider FOV. |
Observation Space
Default observation includes only joint positions (configurable via observations parameter on LookAtConfig):
| Component | Class | Dimensions | Description |
|---|---|---|---|
| Joint positions | JointPositions | 6 | Current angle of each robot joint |
Objects
The environment uses CubeObject instances as gaze targets.
Success Condition
Succeeds when the target is within the wrist camera's field of view: the angle between the wrist-camera optical axis and the direction to the object is at most half the camera FOV (fov_deg / 2, or the live cam_fovy / 2 when fov_deg is None). The dense reward shapes toward dead-center (1.0 when perfectly aligned), rewarding the object being centered in the frame.
Example
import gymnasium as gym
import so101_nexus.mujoco
from so101_nexus import CubeObject, LookAtConfig
env = gym.make(
"MuJoCoLookAt-v1",
render_mode="human",
config=LookAtConfig(
objects=[CubeObject()],
),
)
obs, info = env.reset()
for _ in range(256):
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
obs, info = env.reset()
env.close()