google-deepmind/robotics-antigravity-plugin

so101-calibration

Stage 5 3D hand-eye projection and tabletop surface height (ztable) calibration for the physical SO-101 6-DOF robot arm with a fixed/overhead workspace camera: fixed-camera check, two-cue gripper marker verification, guarded 3 mm vertical touch-down probing…

查看源码

使用方法

  1. 复制安装命令,在自己的智能体环境安装前先检查源码包。
  2. 从原始文档核对所需工具、凭据和支持的宿主。
  3. 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill so101-calibration

开始前需了解

  • 安装不包含模型订阅、API 额度或访问文件的权限。
  • 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
仓库原始内容

按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。

SO-101 Stage 5 (Optional · Requires Fixed Camera) — Tabletop Touch-Down & 3D Hand-Eye Calibration

After completing Stages 0–3 in ../so101-bringup/SKILL.md (including optional Stage 4: the USB bandwidth check and reference frames, which R10 drift checks require) and following all safety rules (R1–R16) in safety_rules.md, measure the physical tabletop height z_table and fit the 3D fixed/overhead camera projection matrix using calibration_notes.md:

  1. Step 0 — Verify Fixed/Overhead Camera Configuration:
  • Check NOTES.md and config/robot_config.json (get_fixed_camera_role_and_config in so101_common.py).
  • Stage 5 requires a stationary fixed/overhead camera viewing the arm and tabletop. If the human only set up a wrist camera (minimal 1-camera setup), stop with a clear message explaining that Stage 5 requires a fixed camera and ask whether they want to add one or finish at Stage 3 ("Hello World").
  1. Part A — Two-Cue Gripper Marker Verification ([detect_marker.py](scripts/detect_marker.py), `R9`, `R16`):
  • With motor torque OFF (R16), ask the human to place a unique high-contrast marker on the gripper after the wrist joint (e.g., a royal-blue triangle sticker or high-contrast Post-it triangle on the flat matte-white wrist camera mount plate) and verify 1 unique detection with 0 background false positives:
bash
     python3 scripts/detect_marker.py --config config/marker_detect.json --image data/stage4/reference_overhead.png
  • Cue 1 (marker colour & solidity): Filters on chromatic BGR excess ((B - G) >= min_b_minus_g, (G - R) >= min_g_minus_r, B >= min_b) or custom hsv_lower..hsv_upper with a 3×3 dilation, bounded area, and convex-hull solidity.
  • Cue 2 (mount surround contrast): Checks the concentric annulus r ∈ [10, 32] px around each candidate centroid for bright matte-white pixels (white_surround_fraction >= 0.35) or high surround luminance contrast, rejecting background specks.
  1. Part B & Part C Dry-Runs + Single `"go"` for Touch-Down & 3D Hand-Eye Sweep ([measure_table.py](scripts/measure_table.py) & [calibrate_handeye.py](scripts/calibrate_handeye.py), `R1`, `R2`, `R5`, `R8`, `R10`):
  • Automatically run both measure_table.py --dry-run and calibrate_handeye.py --dry-run (R2) without asking for confirmation to validate planned FK tip heights (SO101Follower.urdf, R8), the `-0.050 m` (`5 cm` below base plane) hard safety floor (R5), and the multi-height workspace grid (z_clearances_above_table_m = [0.035, 0.065] m).
  • Ask the human to clear the central tabletop target patch (patch_xy_m = [0.200, 0.000]) and obtain a single `"go"` (`R1`) covering both Part B (measure_table.py) and Part C (calibrate_handeye.py):
bash
     python3 scripts/measure_table.py --dry-run
     python3 scripts/calibrate_handeye.py --dry-run
     python3 scripts/measure_table.py
     python3 scripts/calibrate_handeye.py
     python3 scripts/calibrate_handeye.py --fit-from-log
  • Part B (`measure_table.py`): Descends vertically at 3.0 mm/step (<= 12°/s), establishes a free-air gravity sag & servo load baseline over the first 4 probes, and halts immediately when sag_excess, load_excess, or Z-stall indicates contact, recording z_table_m, saving annotated contact images, unwinding back to rest_pose (R3), and turning torque off (R4).
  • Part C (`calibrate_handeye.py`): Combines the multi-height grid samples above z_table_m with the guarded touch-down contact sample (Pose 0 at z_table_m) and fits a perspective-regularized (reg_lambda = 500.0) 3×4 projective matrix M (fit_projective_handeye) plus closed-form ray-plane unprojection pixel_to_world_at_z(M, u, v, z_plane_m).
  1. Verification, Line-of-Sight Gate & Phase 2 Readiness (`R10`, `R14`):
  • Visually inspect touchdown_contact_overhead.png, touchdown_contact_wrist.png, and handeye_calibration_grid.png to verify contact location and projected tabletop metric grid alignment.
  • Confirm rmse_reprojection_px (<= 8.0 px, hold-out < 15.0 px under R10) and rmse_localization_mm (< 3.0 mm at z_table_m), and confirm robot.bus Present_Position returned within 5° of rest_pose with Torque_Enable = 0.
  • Report Stage 5 completion using the 6-field R14 format before proceeding to Phase 2 vision-guided pick-and-place.
  1. Coordinate Conventions & Safety Invariants:
  • Robot base frame (gripper_frame_link): +X forward from base, +Y left, +Z upward (z_table_m typically ~-0.024 m below shoulder base origin).
  • Fixed/overhead camera pixels (u, v) follow top-left image origin (1280×720); pixel_to_world_at_z unprojects any tabletop pixel (u, v) onto horizontal plane Z = z_plane_m.
  • All shared calibration and trajectory helpers live in so101_common.py; never lower z_hard_floor_m below -0.050 m or modify SO101Follower gripper current/torque limits.
来自同一仓库

更多 Skills

全部 Skills
google-deepmind
社区

blender-creation

Creates physically calibrated, articulated, PBR-textured 3D MuJoCo assets (MJCF model.xml, .mjz, mjSpec) from scratch using headless Blender (bpy, bmesh, blendermeshutils, assetbuildutils), convex collision synthesis, 10-scenario auditing, and Studio preview. Use only when asked to create, model, or build a 3D MuJoCo object from a URL, image, or text description, or when blender-creation is named. Don't use for non-MuJoCo Blender/Cycles/CAD modeling.

1小时安装量
2
GitHub Stars
80
最近更新
11月17日
google-deepmind
社区

fr3-duo-control

Franka Duo robot only (Franka Research 3 / FR3 Duo dual-arm cell with Robotiq 2F-85 grippers, ZED Mini and RealSense D405 cameras). Brings up, homes, and controls it via pylibfranka on hardware or in MuJoCo sim. Use only when the user names Franka Duo, FR3 Duo, or fr3duo.

1小时安装量
2
GitHub Stars
80
最近更新
11月17日
google-deepmind
社区

i2rt-yam-bringup

Checks and installs the I2RT YAM SDK (i2rt), configures Linux socketcan or macOS gsusb CAN bringup for I2RT YAM robot arms, and provides a wave smoke test (wave.py). Use only before bringing up or programming a physical or simulated I2RT YAM robot arm, or when asked to run a wave gesture on an I2RT YAM robot. Don't use for non-YAM CAN bus debugging (e.g. automotive OBD-II or industrial PLC CAN networks) or non-robotics tasks.

1小时安装量
2
GitHub Stars
80
最近更新
11月17日
google-deepmind
社区

i2rt-yam-ik

Solves 6-DOF inverse kinematics for the I2RT YAM and YAM Pro robot arm using Mink + DAQP on combined YAM + gripper MJCF. Use only when computing joint targets or local Cartesian moves on an I2RT YAM or YAM Pro robot arm. Don't use for non-YAM inverse kinematics, general quadratic programming (QP) solvers, or non-robotics tasks.

1小时安装量
2
GitHub Stars
80
最近更新
11月17日