google-deepmind/robotics-antigravity-plugin

mujoco-playground

Runs and zero-shot transfers robot controllers across MuJoCo Playground sim and physical SO-101 or I2RT YAM DeviceLayers (plus sim G1 and Trossen WXAI) via PlaygroundSim2Real and setupscene(spec), composing Studio, perception, IK, motion-planning, and visua…

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill mujoco-playground

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

MuJoCo Playground Sim-to-Real Composition (`playground_sim2real.py`)

Compose live MuJoCo Studio visualization, perception, IK, collision-aware planning, and visual servoing on mujoco_playground's out-of-the-box so101 and yam 50 Hz DeviceLayer stacks with identical code in mode="sim" and mode="real":

  1. Workspace Setup:
  • Clone and sync: git clone https://github.com/google-deepmind/mujoco_playground.git && cd mujoco_playground && make sync-cpu (or make sync / make sync-cu12 on Linux CUDA; add uv sync --extra so101 or --extra yam --default-index https://pypi.org/simple on robot hosts, and pass --no-sync to uv run).
  • Install composed skill packages into .venv: uv pip install --python .venv/bin/python --default-index https://pypi.org/simple ruckig opencv-python-headless scipy mink daqp. Set MUJOCO_GL=osmesa only on headless Linux (omit on macOS).
  1. Unified `sim` / `real` `DeviceLayer` Execution ([`PlaygroundSim2Real`](scripts/playground_sim2real.py)):
  • Wrap SO101Follower / SO101FollowerDevice (so101_aloha_table, arm in rad, gripper in [0, 100]) or YamFollower / YamFollowerDevice (yam_aloha_table, arm in rad, gripper in [0, 1]) via with PlaygroundSim2Real(robot="yam", mode="sim", setup_scene=...) as env:, passing setup_scene(spec) (mujoco-spec-editing / mujoco-python) to customize objects, containers, conveyors, or camera/base poses for any scene.
  • Never bypass the follower DeviceLayer or read privileged object mj_data.xpos inside controllers; PlaygroundSim2Real steps through follower/goal_position/*, reads follower/joint_position/* + camera streams, and cosine-interpolates safely to FOLLOWER_HOME on close().
  1. Live MuJoCo Studio Visualization & Hot-Reload (`mujoco-studio-sidecar` + `mujoco-studio`):
  • Load `mujoco-studio-sidecar` and `mujoco-studio` whenever interactively building scenes, controllers, or Studio UI panels.
  • mujoco_playground/.venv/bin/python already ships mujoco.studio:

publish build_playground_spec(robot="yam", setup_scene=...) from a SimPlugin in <workspace_dir>/sim_plugin.py (on_load_model calling self.handle.set_model(spec) and returning True, plus if self in event.plugins: self.handle.reload() on HotReloadEvent), place viewer_plugin.py / messages.py alongside it, launch studiod.py with <playground_dir>/.venv/bin/python (<mujoco-studio-sidecar>/scripts/studiod.py --workspace_dir=<dir>, IsDaemon: true), include [MuJoCo Studio](sidecar://robotics/mujoco_studio/), and verify /_studio/status after edits (or stream PlaygroundSim2Real rollouts to a Studio ViewerHandle via env.attach_studio(handle)).

  1. Composing `robotics` Plugin Skills Zero-Shot (`sim` -> `real`):
  • `rgb-object-localization`: Call

cam = env.camera_opencv("wrist_camera") (computes K, T_world_cam, and T_ee_cam from measured q_arm via mj_kinematics + mj_camlight with MuJoCo-to-OpenCV R @ diag([1, -1, -1]) applied) and pass (mask, cam["K"], cam["T_world_cam"], z_plane_m) to ray_plane_contour_pose or two views to triangulate_matched_points.

  • `i2rt-yam-ik` & Arm IK: On yam, map world targets into the arm base frame (computed dynamically from env.mj_model via env.t_world_base) with p_b = env.world_to_base_point(p_w) before YamIk(xml_path=env.robot_xml_path).make_topdown_grasp_pose(*p_b) + ik.solve(...) (or transform full poses with env.world_to_base_pose). On 5-DOF so101 (or either arm), call env.solve_arm_ik(pos_world, approach_dir_world=(0, 0, -1), jaw_yaw_rad=...).
  • `motion-planning` (`motion_planner` `mp`): Transform world obstacles into the robot MJCF root frame via env.world_to_base_point / env.world_to_base_pose (table at z = 0.0 in base frame), plan free-space detours and linear Cartesian strokes with planner = mp.RobotArmPlanner(mjcf_path=env.robot_xml_path, joint_names=env.arm_joint_names, gripper_cmd=env.gripper_normalized) (planner.plan_arm / planner.plan_cartesian_stroke, control_dt_s=env.control_period_s), and stream via env.execute_trajectory(res["q_trajectory"]).
  • `visual-servoing` (`Ibvs`): Compute unoccluded hover height with Ibvs.principal_ray_hover_distance(cam["T_ee_cam"], env.approach_axis_ee), refine wrist alignment at hover via Ibvs.ray_plane_contour_pose(mask, cam["fx_px"], cam["fy_px"], cam["cx_px"], cam["cy_px"], cam["T_world_cam"], z_plane_m) (or 6-DOF Ibvs.compute_step), then descend, close gripper (env.set_gripper(...)), and lift.
같은 저장소의 Skills

더 많은 Skills

모든 Skills
google-deepmind
커뮤니티

blender-creation

Creates physically calibrated, articulated, PBR-textured 3D MuJoCo assets (MJCF model.xml, .mjz, mjSpec) from scratch using headless Blender (bpy, bmesh, blendermeshutils, assetbuildutils), convex collision synthesis, 10-scenario auditing, and Studio preview. Use only when asked to create, model, or build a 3D MuJoCo object from a URL, image, or text description, or when blender-creation is named. Don't use for non-MuJoCo Blender/Cycles/CAD modeling.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

fr3-duo-control

Franka Duo robot only (Franka Research 3 / FR3 Duo dual-arm cell with Robotiq 2F-85 grippers, ZED Mini and RealSense D405 cameras). Brings up, homes, and controls it via pylibfranka on hardware or in MuJoCo sim. Use only when the user names Franka Duo, FR3 Duo, or fr3duo.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-bringup

Checks and installs the I2RT YAM SDK (i2rt), configures Linux socketcan or macOS gsusb CAN bringup for I2RT YAM robot arms, and provides a wave smoke test (wave.py). Use only before bringing up or programming a physical or simulated I2RT YAM robot arm, or when asked to run a wave gesture on an I2RT YAM robot. Don't use for non-YAM CAN bus debugging (e.g. automotive OBD-II or industrial PLC CAN networks) or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-ik

Solves 6-DOF inverse kinematics for the I2RT YAM and YAM Pro robot arm using Mink + DAQP on combined YAM + gripper MJCF. Use only when computing joint targets or local Cartesian moves on an I2RT YAM or YAM Pro robot arm. Don't use for non-YAM inverse kinematics, general quadratic programming (QP) solvers, or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일