google-deepmind/robotics-antigravity-plugin

motion-planning

Plans collision-free 6-DoF grasped-object paths, coupled robot arm+object joint trajectories, and Cartesian contact strokes for manipulation arms (SO-101, I2RT YAM, ALOHA, or MuJoCo robots) with stereo triangulation, clearance diagnostics, and camera overla…

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill motion-planning

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

<!-- disableFinding(LINEOVER80) --> <!-- disableFinding(HEADING_FUNCTION) -->

Collision-Aware Motion Planning (`motion_planner.py`)

Import import motion_planner as mp (prepending <skill_dir>/scripts to sys.path if needed).

CRITICAL — Never `cat`, `head`, or `tail` `motion_planner.py` or raw robot XMLs! Check README.md/runtime specs for transport, action layout, control_dt_s, and camera intrinsics, write workspace .py scripts or python3 - << 'EOF' (avoid /tmp scripts or inline python3 -c quoting errors), and call mjcf_path = mp.resolve_mjcf_path(), print(mp.describe_robot(mjcf_path)["summary_text"]), and planner.gripper_frame_info(gripper_cmd) upfront for joints/limits, TCP axes (approach_axis_ee, closing_axis_ee, slim_axis_ee), jaw widths, and asymmetric_protrusions.

1. Calibration, Hollow Geometry & Immediate t = 0 Escape

  • MJCF Root Frame & Sparse Triangulation: Express all poses/primitives in the robot MJCF root frame (never assume z_support = 0.0 or mix external world extrinsics). Compute camera poses via mp.get_camera_poses_in_robot_frame(mjcf_path, joint_positions) ({"left": left_q, "right": right_q}, {joint_name: val}, or single-arm q) and triangulate 1–2 support points (z_support, max_ray_residual_m < 0.015) plus sparse landmarks across >= 2 views via mp.triangulate_pixels (standing height h = z_top - z_support, resting center z = z_support + 0.5 * h, or mp.intersect_ray_plane — never run dense stereo, RANSAC, or scipy.optimize silhouette fitters).
  • Safety-Margin Overlay Fit (`~1–1.5 cm`), Hollow Primitives & Door Padding: Persist scene_model.py (build_scene, build_object). Decompose hollow bowls, cups, racks, and cavities into thin wall, floor/sill, ceiling, and rim primitives (add_vertical_cylinder base + perimeter ring of vertical add_capsule walls for round vessels, or add_oriented_box panels for box cavities — never solid blocks sealing interior goals). Model support (group="table"), ungrasped target (group="target_init", removed via scene.without("target_init") once attached), and idle arms (planner.add_other_arm_to_scene). Pad fixed walls 3–5 mm and unlatched hinged doors `~18–25 mm` (`group="door"`) (>= 12–15 cm staging clearance). Calibrate to the safety margin (`~1–1.5 cm`), not subpixel perfection: once wireframes in mp.render_scene_overlay enclose obstacles/rims within ~1–1.5 cm (2–3 passes max), stop refining geometry and move to execution (avoid custom Canny, vanishing-point, or numerical solvers; use close-range wrist views at pre_grasp/pre_place if needed).
  • Immediate All-Arm `t = 0` Escape: Check every arm at t = 0 via planner.check_arm_config(scene, q_init). If any arm has min_clearance_m < 0.025, execute `planner.escape_initial_collision(scene, q_init, target_clearance_m=0.04)["q_trajectory"]` right away before planning pick/place, re-observe, update the arm's parked pose, and only re-fit scene_model.py if an object shifted.

2. Motion Planning, Lookahead Feasibility & Checkpoint Execution

  • Free-Space `plan_arm` vs. Linear `plan_cartesian_stroke`: Use planner.plan_arm (safety_margin_m=0.015) between pre-contact poses (5–8 cm clear), attaching held objects via planner.attach_object_at_world_pose(obj, q_grasp, obj_pose6d) and planner.set_gripper_opening(gripper_cmd). Always use `planner.plan_cartesian_stroke(..., max_joint_vel_rad_s=0.5)` (never 2-point joint interpolation) for pre_grasp -> grasp, vertical lift clear of rims, pre_place -> place -> retract, and linear cavity insertions/pushes (ignore_obstacles=("table",) or clearance_tol_m=0.003 at contact). Hold targets `~0.4–0.5 s` when closing or fully opening (0.0) the gripper before lifting/retracting. Center grasps on ee_site, align closing_axis_ee across the wider clearance axis so slim_axis_ee faces narrow jambs/neighbors (or along thin rims/rods), and flip wrist roll 180° (ee_rot @ np.diag([-1, -1, 1])) so asymmetric_protrusions face open space.
  • Single-Arm Reach First & Decoupled Regrasp Staging: First check whether a single arm can both pick and reach a valid goal feature directly. When reorientation or cross-workspace transfer is required, prefer placing stably on an unobstructed surface (clear of open doors/obstacles), releasing, re-observing, and regrasping to decouple the arms at a zero-risk checkpoint — reserve mid-air bimanual handover strictly for slender/non-standing objects that cannot rest stably on a surface, and if coupled two-arm or multi-stage planning fails once or twice offline, switch immediately to surface staging or an alternate reachable goal feature rather than tuning coupled IK in a loop.
  • Lookahead Endpoint Feasibility, Then Checkpoint-by-Checkpoint Execution: Reserve `>= 75%` of episode budget for execution. Before grasping, run a fast endpoint feasibility check (solve_ik + check_arm_config / check_se3_pose at grasp and goal placement) to avoid kinematic dead ends — but only synthesize `plan_arm` / `plan_cartesian_stroke` trajectories up to the next safe, reversible physical checkpoint (collision escape, lifted hover, surface staging, or single-item placement), then execute right away and re-observe real camera + joint feedback rather than pre-solving multi-stage trajectories or all N items offline. After each placement, update scene with placed items at their observed resting poses and triangulate 3D position to verify it lies deep inside the goal volume (following shallow doorway-sill releases with a closed-gripper linear centerline push, or using a secure upper support feature on tall stands), parking arms >= 10 cm clear when done.

3. motion_planner (mp) API (pose6d = [x, y, z, yaw, pitch, roll])

  • Introspection & SE(3): mp.resolve_mjcf_path() -> str, mp.describe_robot(mjcf_path=None) -> {"arms", "cameras", "summary_text"}, mp.rpy_to_matrix(r, p, y), mp.matrix_to_rpy(rot), mp.quat_wxyz_to_matrix(q_wxyz), mp.pose6d_from_pos_rot(pos, rot=None), mp.pos_rot_from_pose6d(pose6d) -> (pos, rot), mp.make_transform(pos, rot=None), mp.invert_transform(tf).
  • `Scene3D` & `MultiSphereObject`: scene = mp.Scene3D(): .add_sphere(name, center, radius, group=""), .add_aligned_box(name, center, half_extents, group=""), .add_oriented_box(name, center, half_extents, rotation, group=""), .add_capsule(name, point_a, point_b, radius, group=""), .add_vertical_cylinder(name, center, radius, height, group=""), .add_object_spheres(obj, pose6d, name_prefix="", group=""), .without(*names_or_groups), .clone(). obj = mp.MultiSphereObject(name="object"): .add_sphere(local_center, radius), .add_capsule_spheres(p0, p1, radius, num_spheres=5). mp.check_se3_pose(obj, scene, pose6d) -> {"valid", "collisions", "min_clearance_m"}, mp.plan_object_se3(obj, scene, start_pose6d, goal_pose6d, safety_margin_m=0.010) -> {"solved", "waypoints", "interpolated"}.
  • `RobotArmPlanner`: planner = mp.RobotArmPlanner(mjcf_path=None, arm="left", gripper_cmd=0.0) (.dof, .q_min, .q_max, .joint_names, .ee_site_name): .fk(q) -> (ee_pos, ee_rot), .gripper_frame_info(gripper_cmd=None) -> {"approach_axis_ee", "closing_axis_ee", "slim_axis_ee", "open_inner_gap_m", "closed_inner_gap_m", "open_outer_width_m", "slim_profile_width_m", "asymmetric_protrusions", "top_down_rot_hint"}, .set_gripper_opening(gripper_cmd), .attach_object_at_world_pose(obj, q_grasp, obj_world_pose6d) -> T_ee_obj, .ee_pose_for_object_pose(obj_world_pose6d) -> (ee_pos, ee_rot), .attached_object_pose(q) -> pose6d, .detach_object(), .add_other_arm_to_scene(scene, other_arm_q, other_gripper_cmd=0.0), .check_arm_config(scene, q, ignore_obstacles=(), clearance_tol_m=0.0) -> {"valid", "self_collisions", "collisions", "min_clearance_m", "nearest_pair"}, .solve_ik(target_pos, target_rot=None, q_seed=None, scene=None, num_restarts=16, ignore_obstacles=(), clearance_tol_m=0.0) -> {"success", "q", "pos_err_m", "rot_err_rad", "collision_check"}, .escape_initial_collision(scene, q_start, target_clearance_m=0.015, control_dt_s=0.04) -> {"escaped", "q_escaped", "q_trajectory"}, .plan_cartesian_stroke(q_start, target_pos, target_rot=None, num_waypoints=8, scene=None, ignore_obstacles=(), clearance_tol_m=0.0, max_joint_vel_rad_s=0.55, control_dt_s=0.04) -> {"solved", "waypoints", "q_trajectory", "min_clearance_m"}, .plan_arm(scene, q_start, q_goal, max_joint_vel_rad_s=1.25, control_dt_s=0.04, safety_margin_m=0.015) -> {"solved", "waypoints", "q_trajectory"}, mp.resample_waypoints_cosine(waypoints, max_joint_vel_rad_s=1.25, control_dt_s=0.04) -> np.ndarray.
  • Cameras & Overlay: mp.get_camera_poses_in_robot_frame(mjcf_path, joint_positions=None) -> {cam_key: {"pos", "rot", "mjcf_name"}}, mp.triangulate_pixels(uv_by_camera, cam_poses, cam_intrinsics) -> {"point_3d", "max_ray_residual_m"}, mp.intersect_ray_plane(u_col, v_row, cam_pose, cam_intrinsics, plane_z=0.0), mp.render_scene_overlay(mjcf_path, joint_positions, camera_image_paths, scene, output_path, objects=(), camera_intrinsics=None) -> str.
같은 저장소의 Skills

더 많은 Skills

모든 Skills
google-deepmind
커뮤니티

blender-creation

Creates physically calibrated, articulated, PBR-textured 3D MuJoCo assets (MJCF model.xml, .mjz, mjSpec) from scratch using headless Blender (bpy, bmesh, blendermeshutils, assetbuildutils), convex collision synthesis, 10-scenario auditing, and Studio preview. Use only when asked to create, model, or build a 3D MuJoCo object from a URL, image, or text description, or when blender-creation is named. Don't use for non-MuJoCo Blender/Cycles/CAD modeling.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

fr3-duo-control

Franka Duo robot only (Franka Research 3 / FR3 Duo dual-arm cell with Robotiq 2F-85 grippers, ZED Mini and RealSense D405 cameras). Brings up, homes, and controls it via pylibfranka on hardware or in MuJoCo sim. Use only when the user names Franka Duo, FR3 Duo, or fr3duo.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-bringup

Checks and installs the I2RT YAM SDK (i2rt), configures Linux socketcan or macOS gsusb CAN bringup for I2RT YAM robot arms, and provides a wave smoke test (wave.py). Use only before bringing up or programming a physical or simulated I2RT YAM robot arm, or when asked to run a wave gesture on an I2RT YAM robot. Don't use for non-YAM CAN bus debugging (e.g. automotive OBD-II or industrial PLC CAN networks) or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-ik

Solves 6-DOF inverse kinematics for the I2RT YAM and YAM Pro robot arm using Mink + DAQP on combined YAM + gripper MJCF. Use only when computing joint targets or local Cartesian moves on an I2RT YAM or YAM Pro robot arm. Don't use for non-YAM inverse kinematics, general quadratic programming (QP) solvers, or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일