使用方法
- 复制安装命令,在自己的智能体环境安装前先检查源码包。
- 从原始文档核对所需工具、凭据和支持的宿主。
- 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill so101-calibration开始前需了解
- 安装不包含模型订阅、API 额度或访问文件的权限。
- 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。
SO-101 Stage 5 (Optional · Requires Fixed Camera) — Tabletop Touch-Down & 3D Hand-Eye Calibration
After completing Stages 0–3 in ../so101-bringup/SKILL.md (including optional Stage 4: the USB bandwidth check and reference frames, which R10 drift checks require) and following all safety rules (R1–R16) in safety_rules.md, measure the physical tabletop height z_table and fit the 3D fixed/overhead camera projection matrix using calibration_notes.md:
- Step 0 — Verify Fixed/Overhead Camera Configuration:
- Check
NOTES.mdandconfig/robot_config.json(get_fixed_camera_role_and_configin so101_common.py). - Stage 5 requires a stationary fixed/overhead camera viewing the arm and tabletop. If the human only set up a wrist camera (minimal 1-camera setup), stop with a clear message explaining that Stage 5 requires a fixed camera and ask whether they want to add one or finish at Stage 3 ("Hello World").
- Part A — Two-Cue Gripper Marker Verification ([detect_marker.py](scripts/detect_marker.py), `R9`, `R16`):
- With motor torque OFF (
R16), ask the human to place a unique high-contrast marker on the gripper after the wrist joint (e.g., a royal-blue triangle sticker or high-contrast Post-it triangle on the flat matte-white wrist camera mount plate) and verify1unique detection with0background false positives:
python3 scripts/detect_marker.py --config config/marker_detect.json --image data/stage4/reference_overhead.png- Cue 1 (marker colour & solidity): Filters on chromatic BGR excess (
(B - G) >= min_b_minus_g,(G - R) >= min_g_minus_r,B >= min_b) or customhsv_lower..hsv_upperwith a3×3dilation, bounded area, and convex-hull solidity. - Cue 2 (mount surround contrast): Checks the concentric annulus
r ∈ [10, 32] pxaround each candidate centroid for bright matte-white pixels (white_surround_fraction >= 0.35) or high surround luminance contrast, rejecting background specks.
- Part B & Part C Dry-Runs + Single `"go"` for Touch-Down & 3D Hand-Eye Sweep ([measure_table.py](scripts/measure_table.py) & [calibrate_handeye.py](scripts/calibrate_handeye.py), `R1`, `R2`, `R5`, `R8`, `R10`):
- Automatically run both
measure_table.py --dry-runandcalibrate_handeye.py --dry-run(R2) without asking for confirmation to validate planned FK tip heights (SO101Follower.urdf,R8), the `-0.050 m` (`5 cm` below base plane) hard safety floor (R5), and the multi-height workspace grid (z_clearances_above_table_m = [0.035, 0.065] m). - Ask the human to clear the central tabletop target patch (
patch_xy_m = [0.200, 0.000]) and obtain a single `"go"` (`R1`) covering both Part B (measure_table.py) and Part C (calibrate_handeye.py):
python3 scripts/measure_table.py --dry-run
python3 scripts/calibrate_handeye.py --dry-run
python3 scripts/measure_table.py
python3 scripts/calibrate_handeye.py
python3 scripts/calibrate_handeye.py --fit-from-log- Part B (`measure_table.py`): Descends vertically at
3.0 mm/step(<= 12°/s), establishes a free-air gravity sag & servo load baseline over the first 4 probes, and halts immediately whensag_excess,load_excess, or Z-stall indicates contact, recordingz_table_m, saving annotated contact images, unwinding back torest_pose(R3), and turning torque off (R4). - Part C (`calibrate_handeye.py`): Combines the multi-height grid samples above
z_table_mwith the guarded touch-down contact sample (Pose 0atz_table_m) and fits a perspective-regularized (reg_lambda = 500.0)3×4projective matrixM(fit_projective_handeye) plus closed-form ray-plane unprojectionpixel_to_world_at_z(M, u, v, z_plane_m).
- Verification, Line-of-Sight Gate & Phase 2 Readiness (`R10`, `R14`):
- Visually inspect
touchdown_contact_overhead.png,touchdown_contact_wrist.png, andhandeye_calibration_grid.pngto verify contact location and projected tabletop metric grid alignment. - Confirm
rmse_reprojection_px(<= 8.0 px, hold-out< 15.0 pxunderR10) andrmse_localization_mm(< 3.0 mmatz_table_m), and confirmrobot.busPresent_Positionreturned within5°ofrest_posewithTorque_Enable = 0. - Report Stage 5 completion using the 6-field
R14format before proceeding to Phase 2 vision-guided pick-and-place.
- Coordinate Conventions & Safety Invariants:
- Robot base frame (
gripper_frame_link):+Xforward from base,+Yleft,+Zupward (z_table_mtypically ~-0.024 mbelow shoulder base origin). - Fixed/overhead camera pixels
(u, v)follow top-left image origin (1280×720);pixel_to_world_at_zunprojects any tabletop pixel(u, v)onto horizontal planeZ = z_plane_m. - All shared calibration and trajectory helpers live in so101_common.py; never lower
z_hard_floor_mbelow-0.050 mor modifySO101Followergripper current/torque limits.