Come usare
- Copia il comando e controlla il pacchetto prima di installarlo nel tuo ambiente agente.
- Controlla strumenti, credenziali e ambienti richiesti nel documento originale.
- Descrivi input e risultato desiderato; confronta l’output con le fonti.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill mujoco-playgroundPrima di iniziare
- L’installazione non include abbonamenti, quote API o accesso ai file.
- Stelle e installazioni indicano attività, non un test indipendente di qualità o sicurezza.
Contenuto dal repository con titoli, esempi, codice, tabelle, link e immagini preservati.
MuJoCo Playground Sim-to-Real Composition (`playground_sim2real.py`)
Compose live MuJoCo Studio visualization, perception, IK, collision-aware planning, and visual servoing on mujoco_playground's out-of-the-box so101 and yam 50 Hz DeviceLayer stacks with identical code in mode="sim" and mode="real":
- Workspace Setup:
- Clone and sync:
git clone https://github.com/google-deepmind/mujoco_playground.git && cd mujoco_playground && make sync-cpu(ormake sync/make sync-cu12on Linux CUDA; adduv sync --extra so101or--extra yam --default-index https://pypi.org/simpleon robot hosts, and pass--no-synctouv run). - Install composed skill packages into
.venv:uv pip install --python .venv/bin/python --default-index https://pypi.org/simple ruckig opencv-python-headless scipy mink daqp. SetMUJOCO_GL=osmesaonly on headless Linux (omit on macOS).
- Unified `sim` / `real` `DeviceLayer` Execution ([`PlaygroundSim2Real`](scripts/playground_sim2real.py)):
- Wrap
SO101Follower/SO101FollowerDevice(so101_aloha_table, arm inrad, gripper in[0, 100]) orYamFollower/YamFollowerDevice(yam_aloha_table, arm inrad, gripper in[0, 1]) viawith PlaygroundSim2Real(robot="yam", mode="sim", setup_scene=...) as env:, passingsetup_scene(spec)(mujoco-spec-editing/mujoco-python) to customize objects, containers, conveyors, or camera/base poses for any scene. - Never bypass the follower
DeviceLayeror read privileged objectmj_data.xposinside controllers;PlaygroundSim2Realsteps throughfollower/goal_position/*, readsfollower/joint_position/*+ camera streams, and cosine-interpolates safely toFOLLOWER_HOMEonclose().
- Live MuJoCo Studio Visualization & Hot-Reload (`mujoco-studio-sidecar` + `mujoco-studio`):
- Load `mujoco-studio-sidecar` and `mujoco-studio` whenever interactively building scenes, controllers, or Studio UI panels.
mujoco_playground/.venv/bin/pythonalready shipsmujoco.studio:
publish build_playground_spec(robot="yam", setup_scene=...) from a SimPlugin in <workspace_dir>/sim_plugin.py (on_load_model calling self.handle.set_model(spec) and returning True, plus if self in event.plugins: self.handle.reload() on HotReloadEvent), place viewer_plugin.py / messages.py alongside it, launch studiod.py with <playground_dir>/.venv/bin/python (<mujoco-studio-sidecar>/scripts/studiod.py --workspace_dir=<dir>, IsDaemon: true), include [MuJoCo Studio](sidecar://robotics/mujoco_studio/), and verify /_studio/status after edits (or stream PlaygroundSim2Real rollouts to a Studio ViewerHandle via env.attach_studio(handle)).
- Composing `robotics` Plugin Skills Zero-Shot (`sim` -> `real`):
- `rgb-object-localization`: Call
cam = env.camera_opencv("wrist_camera") (computes K, T_world_cam, and T_ee_cam from measured q_arm via mj_kinematics + mj_camlight with MuJoCo-to-OpenCV R @ diag([1, -1, -1]) applied) and pass (mask, cam["K"], cam["T_world_cam"], z_plane_m) to ray_plane_contour_pose or two views to triangulate_matched_points.
- `i2rt-yam-ik` & Arm IK: On
yam, map world targets into the arm base frame (computed dynamically fromenv.mj_modelviaenv.t_world_base) withp_b = env.world_to_base_point(p_w)beforeYamIk(xml_path=env.robot_xml_path).make_topdown_grasp_pose(*p_b)+ik.solve(...)(or transform full poses withenv.world_to_base_pose). On 5-DOFso101(or either arm), callenv.solve_arm_ik(pos_world, approach_dir_world=(0, 0, -1), jaw_yaw_rad=...). - `motion-planning` (`motion_planner` `mp`): Transform world obstacles into the robot MJCF root frame via
env.world_to_base_point/env.world_to_base_pose(tableatz = 0.0in base frame), plan free-space detours and linear Cartesian strokes withplanner = mp.RobotArmPlanner(mjcf_path=env.robot_xml_path, joint_names=env.arm_joint_names, gripper_cmd=env.gripper_normalized)(planner.plan_arm/planner.plan_cartesian_stroke,control_dt_s=env.control_period_s), and stream viaenv.execute_trajectory(res["q_trajectory"]). - `visual-servoing` (`Ibvs`): Compute unoccluded hover height with
Ibvs.principal_ray_hover_distance(cam["T_ee_cam"], env.approach_axis_ee), refine wrist alignment at hover viaIbvs.ray_plane_contour_pose(mask, cam["fx_px"], cam["fy_px"], cam["cx_px"], cam["cy_px"], cam["T_world_cam"], z_plane_m)(or 6-DOFIbvs.compute_step), then descend, close gripper (env.set_gripper(...)), and lift.