google-deepmind/robotics-antigravity-plugin

motion-planning

Plans collision-free 6-DoF grasped-object paths, coupled robot arm+object joint trajectories, and Cartesian contact strokes for manipulation arms (SO-101, I2RT YAM, ALOHA, or MuJoCo robots) with stereo triangulation, clearance diagnostics, and camera overla…

ソースを見る

使い方

  1. インストールコマンドをコピーし、自分の環境へ導入する前にソースを確認します。
  2. 必要なツール、認証情報、対応環境を原文で確認します。
  3. 入力と成果物を指定し、出力を資料と照合します。
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill motion-planning

開始前の確認

  • 導入にはモデル契約、API枠、ファイルへのアクセス許可は含まれません。
  • スター数や導入数は活動の指標であり、独立した品質・安全性試験ではありません。
リポジトリの原文

見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。

<!-- disableFinding(LINEOVER80) --> <!-- disableFinding(HEADING_FUNCTION) -->

Collision-Aware Motion Planning (`motion_planner.py`)

Import import motion_planner as mp (prepending <skill_dir>/scripts to sys.path if needed).

CRITICAL — Never `cat`, `head`, or `tail` `motion_planner.py` or raw robot XMLs! Check README.md/runtime specs for transport, action layout, control_dt_s, and camera intrinsics, write workspace .py scripts or python3 - << 'EOF' (avoid /tmp scripts or inline python3 -c quoting errors), and call mjcf_path = mp.resolve_mjcf_path(), print(mp.describe_robot(mjcf_path)["summary_text"]), and planner.gripper_frame_info(gripper_cmd) upfront for joints/limits, TCP axes (approach_axis_ee, closing_axis_ee, slim_axis_ee), jaw widths, and asymmetric_protrusions.

1. Calibration, Hollow Geometry & Immediate t = 0 Escape

  • MJCF Root Frame & Sparse Triangulation: Express all poses/primitives in the robot MJCF root frame (never assume z_support = 0.0 or mix external world extrinsics). Compute camera poses via mp.get_camera_poses_in_robot_frame(mjcf_path, joint_positions) ({"left": left_q, "right": right_q}, {joint_name: val}, or single-arm q) and triangulate 1–2 support points (z_support, max_ray_residual_m < 0.015) plus sparse landmarks across >= 2 views via mp.triangulate_pixels (standing height h = z_top - z_support, resting center z = z_support + 0.5 * h, or mp.intersect_ray_plane — never run dense stereo, RANSAC, or scipy.optimize silhouette fitters).
  • Safety-Margin Overlay Fit (`~1–1.5 cm`), Hollow Primitives & Door Padding: Persist scene_model.py (build_scene, build_object). Decompose hollow bowls, cups, racks, and cavities into thin wall, floor/sill, ceiling, and rim primitives (add_vertical_cylinder base + perimeter ring of vertical add_capsule walls for round vessels, or add_oriented_box panels for box cavities — never solid blocks sealing interior goals). Model support (group="table"), ungrasped target (group="target_init", removed via scene.without("target_init") once attached), and idle arms (planner.add_other_arm_to_scene). Pad fixed walls 3–5 mm and unlatched hinged doors `~18–25 mm` (`group="door"`) (>= 12–15 cm staging clearance). Calibrate to the safety margin (`~1–1.5 cm`), not subpixel perfection: once wireframes in mp.render_scene_overlay enclose obstacles/rims within ~1–1.5 cm (2–3 passes max), stop refining geometry and move to execution (avoid custom Canny, vanishing-point, or numerical solvers; use close-range wrist views at pre_grasp/pre_place if needed).
  • Immediate All-Arm `t = 0` Escape: Check every arm at t = 0 via planner.check_arm_config(scene, q_init). If any arm has min_clearance_m < 0.025, execute `planner.escape_initial_collision(scene, q_init, target_clearance_m=0.04)["q_trajectory"]` right away before planning pick/place, re-observe, update the arm's parked pose, and only re-fit scene_model.py if an object shifted.

2. Motion Planning, Lookahead Feasibility & Checkpoint Execution

  • Free-Space `plan_arm` vs. Linear `plan_cartesian_stroke`: Use planner.plan_arm (safety_margin_m=0.015) between pre-contact poses (5–8 cm clear), attaching held objects via planner.attach_object_at_world_pose(obj, q_grasp, obj_pose6d) and planner.set_gripper_opening(gripper_cmd). Always use `planner.plan_cartesian_stroke(..., max_joint_vel_rad_s=0.5)` (never 2-point joint interpolation) for pre_grasp -> grasp, vertical lift clear of rims, pre_place -> place -> retract, and linear cavity insertions/pushes (ignore_obstacles=("table",) or clearance_tol_m=0.003 at contact). Hold targets `~0.4–0.5 s` when closing or fully opening (0.0) the gripper before lifting/retracting. Center grasps on ee_site, align closing_axis_ee across the wider clearance axis so slim_axis_ee faces narrow jambs/neighbors (or along thin rims/rods), and flip wrist roll 180° (ee_rot @ np.diag([-1, -1, 1])) so asymmetric_protrusions face open space.
  • Single-Arm Reach First & Decoupled Regrasp Staging: First check whether a single arm can both pick and reach a valid goal feature directly. When reorientation or cross-workspace transfer is required, prefer placing stably on an unobstructed surface (clear of open doors/obstacles), releasing, re-observing, and regrasping to decouple the arms at a zero-risk checkpoint — reserve mid-air bimanual handover strictly for slender/non-standing objects that cannot rest stably on a surface, and if coupled two-arm or multi-stage planning fails once or twice offline, switch immediately to surface staging or an alternate reachable goal feature rather than tuning coupled IK in a loop.
  • Lookahead Endpoint Feasibility, Then Checkpoint-by-Checkpoint Execution: Reserve `>= 75%` of episode budget for execution. Before grasping, run a fast endpoint feasibility check (solve_ik + check_arm_config / check_se3_pose at grasp and goal placement) to avoid kinematic dead ends — but only synthesize `plan_arm` / `plan_cartesian_stroke` trajectories up to the next safe, reversible physical checkpoint (collision escape, lifted hover, surface staging, or single-item placement), then execute right away and re-observe real camera + joint feedback rather than pre-solving multi-stage trajectories or all N items offline. After each placement, update scene with placed items at their observed resting poses and triangulate 3D position to verify it lies deep inside the goal volume (following shallow doorway-sill releases with a closed-gripper linear centerline push, or using a secure upper support feature on tall stands), parking arms >= 10 cm clear when done.

3. motion_planner (mp) API (pose6d = [x, y, z, yaw, pitch, roll])

  • Introspection & SE(3): mp.resolve_mjcf_path() -> str, mp.describe_robot(mjcf_path=None) -> {"arms", "cameras", "summary_text"}, mp.rpy_to_matrix(r, p, y), mp.matrix_to_rpy(rot), mp.quat_wxyz_to_matrix(q_wxyz), mp.pose6d_from_pos_rot(pos, rot=None), mp.pos_rot_from_pose6d(pose6d) -> (pos, rot), mp.make_transform(pos, rot=None), mp.invert_transform(tf).
  • `Scene3D` & `MultiSphereObject`: scene = mp.Scene3D(): .add_sphere(name, center, radius, group=""), .add_aligned_box(name, center, half_extents, group=""), .add_oriented_box(name, center, half_extents, rotation, group=""), .add_capsule(name, point_a, point_b, radius, group=""), .add_vertical_cylinder(name, center, radius, height, group=""), .add_object_spheres(obj, pose6d, name_prefix="", group=""), .without(*names_or_groups), .clone(). obj = mp.MultiSphereObject(name="object"): .add_sphere(local_center, radius), .add_capsule_spheres(p0, p1, radius, num_spheres=5). mp.check_se3_pose(obj, scene, pose6d) -> {"valid", "collisions", "min_clearance_m"}, mp.plan_object_se3(obj, scene, start_pose6d, goal_pose6d, safety_margin_m=0.010) -> {"solved", "waypoints", "interpolated"}.
  • `RobotArmPlanner`: planner = mp.RobotArmPlanner(mjcf_path=None, arm="left", gripper_cmd=0.0) (.dof, .q_min, .q_max, .joint_names, .ee_site_name): .fk(q) -> (ee_pos, ee_rot), .gripper_frame_info(gripper_cmd=None) -> {"approach_axis_ee", "closing_axis_ee", "slim_axis_ee", "open_inner_gap_m", "closed_inner_gap_m", "open_outer_width_m", "slim_profile_width_m", "asymmetric_protrusions", "top_down_rot_hint"}, .set_gripper_opening(gripper_cmd), .attach_object_at_world_pose(obj, q_grasp, obj_world_pose6d) -> T_ee_obj, .ee_pose_for_object_pose(obj_world_pose6d) -> (ee_pos, ee_rot), .attached_object_pose(q) -> pose6d, .detach_object(), .add_other_arm_to_scene(scene, other_arm_q, other_gripper_cmd=0.0), .check_arm_config(scene, q, ignore_obstacles=(), clearance_tol_m=0.0) -> {"valid", "self_collisions", "collisions", "min_clearance_m", "nearest_pair"}, .solve_ik(target_pos, target_rot=None, q_seed=None, scene=None, num_restarts=16, ignore_obstacles=(), clearance_tol_m=0.0) -> {"success", "q", "pos_err_m", "rot_err_rad", "collision_check"}, .escape_initial_collision(scene, q_start, target_clearance_m=0.015, control_dt_s=0.04) -> {"escaped", "q_escaped", "q_trajectory"}, .plan_cartesian_stroke(q_start, target_pos, target_rot=None, num_waypoints=8, scene=None, ignore_obstacles=(), clearance_tol_m=0.0, max_joint_vel_rad_s=0.55, control_dt_s=0.04) -> {"solved", "waypoints", "q_trajectory", "min_clearance_m"}, .plan_arm(scene, q_start, q_goal, max_joint_vel_rad_s=1.25, control_dt_s=0.04, safety_margin_m=0.015) -> {"solved", "waypoints", "q_trajectory"}, mp.resample_waypoints_cosine(waypoints, max_joint_vel_rad_s=1.25, control_dt_s=0.04) -> np.ndarray.
  • Cameras & Overlay: mp.get_camera_poses_in_robot_frame(mjcf_path, joint_positions=None) -> {cam_key: {"pos", "rot", "mjcf_name"}}, mp.triangulate_pixels(uv_by_camera, cam_poses, cam_intrinsics) -> {"point_3d", "max_ray_residual_m"}, mp.intersect_ray_plane(u_col, v_row, cam_pose, cam_intrinsics, plane_z=0.0), mp.render_scene_overlay(mjcf_path, joint_positions, camera_image_paths, scene, output_path, objects=(), camera_intrinsics=None) -> str.
同じリポジトリから

関連する Skills

すべての Skills
google-deepmind
コミュニティ

blender-creation

Creates physically calibrated, articulated, PBR-textured 3D MuJoCo assets (MJCF model.xml, .mjz, mjSpec) from scratch using headless Blender (bpy, bmesh, blendermeshutils, assetbuildutils), convex collision synthesis, 10-scenario auditing, and Studio preview. Use only when asked to create, model, or build a 3D MuJoCo object from a URL, image, or text description, or when blender-creation is named. Don't use for non-MuJoCo Blender/Cycles/CAD modeling.

1時間の導入数
2
GitHub Stars
80
更新日
11月17日
google-deepmind
コミュニティ

fr3-duo-control

Franka Duo robot only (Franka Research 3 / FR3 Duo dual-arm cell with Robotiq 2F-85 grippers, ZED Mini and RealSense D405 cameras). Brings up, homes, and controls it via pylibfranka on hardware or in MuJoCo sim. Use only when the user names Franka Duo, FR3 Duo, or fr3duo.

1時間の導入数
2
GitHub Stars
80
更新日
11月17日
google-deepmind
コミュニティ

i2rt-yam-bringup

Checks and installs the I2RT YAM SDK (i2rt), configures Linux socketcan or macOS gsusb CAN bringup for I2RT YAM robot arms, and provides a wave smoke test (wave.py). Use only before bringing up or programming a physical or simulated I2RT YAM robot arm, or when asked to run a wave gesture on an I2RT YAM robot. Don't use for non-YAM CAN bus debugging (e.g. automotive OBD-II or industrial PLC CAN networks) or non-robotics tasks.

1時間の導入数
2
GitHub Stars
80
更新日
11月17日
google-deepmind
コミュニティ

i2rt-yam-ik

Solves 6-DOF inverse kinematics for the I2RT YAM and YAM Pro robot arm using Mink + DAQP on combined YAM + gripper MJCF. Use only when computing joint targets or local Cartesian moves on an I2RT YAM or YAM Pro robot arm. Don't use for non-YAM inverse kinematics, general quadratic programming (QP) solvers, or non-robotics tasks.

1時間の導入数
2
GitHub Stars
80
更新日
11月17日