Mode d’emploi
- Copiez la commande et examinez le paquet avant de l’installer dans votre environnement agent.
- Consultez le document original pour les outils, identifiants et environnements requis.
- Décrivez les entrées et le livrable souhaité ; comparez la sortie aux sources.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill mujoco-playgroundAvant de commencer
- L’installation n’inclut ni abonnement au modèle, ni quota API, ni accès aux fichiers.
- Étoiles et installations reflètent l’activité, pas un test indépendant de qualité ou de sécurité.
Rendu depuis le dépôt source en conservant titres, exemples, code, tableaux, liens et images.
MuJoCo Playground Sim-to-Real Composition (`playground_sim2real.py`)
Compose live MuJoCo Studio visualization, perception, IK, collision-aware planning, and visual servoing on mujoco_playground's out-of-the-box so101 and yam 50 Hz DeviceLayer stacks with identical code in mode="sim" and mode="real":
- Workspace Setup:
- Clone and sync:
git clone https://github.com/google-deepmind/mujoco_playground.git && cd mujoco_playground && make sync-cpu(ormake sync/make sync-cu12on Linux CUDA; adduv sync --extra so101or--extra yam --default-index https://pypi.org/simpleon robot hosts, and pass--no-synctouv run). - Install composed skill packages into
.venv:uv pip install --python .venv/bin/python --default-index https://pypi.org/simple ruckig opencv-python-headless scipy mink daqp. SetMUJOCO_GL=osmesaonly on headless Linux (omit on macOS).
- Unified `sim` / `real` `DeviceLayer` Execution ([`PlaygroundSim2Real`](scripts/playground_sim2real.py)):
- Wrap
SO101Follower/SO101FollowerDevice(so101_aloha_table, arm inrad, gripper in[0, 100]) orYamFollower/YamFollowerDevice(yam_aloha_table, arm inrad, gripper in[0, 1]) viawith PlaygroundSim2Real(robot="yam", mode="sim", setup_scene=...) as env:, passingsetup_scene(spec)(mujoco-spec-editing/mujoco-python) to customize objects, containers, conveyors, or camera/base poses for any scene. - Never bypass the follower
DeviceLayeror read privileged objectmj_data.xposinside controllers;PlaygroundSim2Realsteps throughfollower/goal_position/*, readsfollower/joint_position/*+ camera streams, and cosine-interpolates safely toFOLLOWER_HOMEonclose().
- Live MuJoCo Studio Visualization & Hot-Reload (`mujoco-studio-sidecar` + `mujoco-studio`):
- Load `mujoco-studio-sidecar` and `mujoco-studio` whenever interactively building scenes, controllers, or Studio UI panels.
mujoco_playground/.venv/bin/pythonalready shipsmujoco.studio:
publish build_playground_spec(robot="yam", setup_scene=...) from a SimPlugin in <workspace_dir>/sim_plugin.py (on_load_model calling self.handle.set_model(spec) and returning True, plus if self in event.plugins: self.handle.reload() on HotReloadEvent), place viewer_plugin.py / messages.py alongside it, launch studiod.py with <playground_dir>/.venv/bin/python (<mujoco-studio-sidecar>/scripts/studiod.py --workspace_dir=<dir>, IsDaemon: true), include [MuJoCo Studio](sidecar://robotics/mujoco_studio/), and verify /_studio/status after edits (or stream PlaygroundSim2Real rollouts to a Studio ViewerHandle via env.attach_studio(handle)).
- Composing `robotics` Plugin Skills Zero-Shot (`sim` -> `real`):
- `rgb-object-localization`: Call
cam = env.camera_opencv("wrist_camera") (computes K, T_world_cam, and T_ee_cam from measured q_arm via mj_kinematics + mj_camlight with MuJoCo-to-OpenCV R @ diag([1, -1, -1]) applied) and pass (mask, cam["K"], cam["T_world_cam"], z_plane_m) to ray_plane_contour_pose or two views to triangulate_matched_points.
- `i2rt-yam-ik` & Arm IK: On
yam, map world targets into the arm base frame (computed dynamically fromenv.mj_modelviaenv.t_world_base) withp_b = env.world_to_base_point(p_w)beforeYamIk(xml_path=env.robot_xml_path).make_topdown_grasp_pose(*p_b)+ik.solve(...)(or transform full poses withenv.world_to_base_pose). On 5-DOFso101(or either arm), callenv.solve_arm_ik(pos_world, approach_dir_world=(0, 0, -1), jaw_yaw_rad=...). - `motion-planning` (`motion_planner` `mp`): Transform world obstacles into the robot MJCF root frame via
env.world_to_base_point/env.world_to_base_pose(tableatz = 0.0in base frame), plan free-space detours and linear Cartesian strokes withplanner = mp.RobotArmPlanner(mjcf_path=env.robot_xml_path, joint_names=env.arm_joint_names, gripper_cmd=env.gripper_normalized)(planner.plan_arm/planner.plan_cartesian_stroke,control_dt_s=env.control_period_s), and stream viaenv.execute_trajectory(res["q_trajectory"]). - `visual-servoing` (`Ibvs`): Compute unoccluded hover height with
Ibvs.principal_ray_hover_distance(cam["T_ee_cam"], env.approach_axis_ee), refine wrist alignment at hover viaIbvs.ray_plane_contour_pose(mask, cam["fx_px"], cam["fy_px"], cam["cx_px"], cam["cy_px"], cam["T_world_cam"], z_plane_m)(or 6-DOFIbvs.compute_step), then descend, close gripper (env.set_gripper(...)), and lift.