google-deepmind/robotics-antigravity-plugin

so101-calibration

Stage 5 3D hand-eye projection and tabletop surface height (ztable) calibration for the physical SO-101 6-DOF robot arm with a fixed/overhead workspace camera: fixed-camera check, two-cue gripper marker verification, guarded 3 mm vertical touch-down probing…

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill so101-calibration

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

SO-101 Stage 5 (Optional · Requires Fixed Camera) — Tabletop Touch-Down & 3D Hand-Eye Calibration

After completing Stages 0–3 in ../so101-bringup/SKILL.md (including optional Stage 4: the USB bandwidth check and reference frames, which R10 drift checks require) and following all safety rules (R1–R16) in safety_rules.md, measure the physical tabletop height z_table and fit the 3D fixed/overhead camera projection matrix using calibration_notes.md:

  1. Step 0 — Verify Fixed/Overhead Camera Configuration:
  • Check NOTES.md and config/robot_config.json (get_fixed_camera_role_and_config in so101_common.py).
  • Stage 5 requires a stationary fixed/overhead camera viewing the arm and tabletop. If the human only set up a wrist camera (minimal 1-camera setup), stop with a clear message explaining that Stage 5 requires a fixed camera and ask whether they want to add one or finish at Stage 3 ("Hello World").
  1. Part A — Two-Cue Gripper Marker Verification ([detect_marker.py](scripts/detect_marker.py), `R9`, `R16`):
  • With motor torque OFF (R16), ask the human to place a unique high-contrast marker on the gripper after the wrist joint (e.g., a royal-blue triangle sticker or high-contrast Post-it triangle on the flat matte-white wrist camera mount plate) and verify 1 unique detection with 0 background false positives:
bash
     python3 scripts/detect_marker.py --config config/marker_detect.json --image data/stage4/reference_overhead.png
  • Cue 1 (marker colour & solidity): Filters on chromatic BGR excess ((B - G) >= min_b_minus_g, (G - R) >= min_g_minus_r, B >= min_b) or custom hsv_lower..hsv_upper with a 3×3 dilation, bounded area, and convex-hull solidity.
  • Cue 2 (mount surround contrast): Checks the concentric annulus r ∈ [10, 32] px around each candidate centroid for bright matte-white pixels (white_surround_fraction >= 0.35) or high surround luminance contrast, rejecting background specks.
  1. Part B & Part C Dry-Runs + Single `"go"` for Touch-Down & 3D Hand-Eye Sweep ([measure_table.py](scripts/measure_table.py) & [calibrate_handeye.py](scripts/calibrate_handeye.py), `R1`, `R2`, `R5`, `R8`, `R10`):
  • Automatically run both measure_table.py --dry-run and calibrate_handeye.py --dry-run (R2) without asking for confirmation to validate planned FK tip heights (SO101Follower.urdf, R8), the `-0.050 m` (`5 cm` below base plane) hard safety floor (R5), and the multi-height workspace grid (z_clearances_above_table_m = [0.035, 0.065] m).
  • Ask the human to clear the central tabletop target patch (patch_xy_m = [0.200, 0.000]) and obtain a single `"go"` (`R1`) covering both Part B (measure_table.py) and Part C (calibrate_handeye.py):
bash
     python3 scripts/measure_table.py --dry-run
     python3 scripts/calibrate_handeye.py --dry-run
     python3 scripts/measure_table.py
     python3 scripts/calibrate_handeye.py
     python3 scripts/calibrate_handeye.py --fit-from-log
  • Part B (`measure_table.py`): Descends vertically at 3.0 mm/step (<= 12°/s), establishes a free-air gravity sag & servo load baseline over the first 4 probes, and halts immediately when sag_excess, load_excess, or Z-stall indicates contact, recording z_table_m, saving annotated contact images, unwinding back to rest_pose (R3), and turning torque off (R4).
  • Part C (`calibrate_handeye.py`): Combines the multi-height grid samples above z_table_m with the guarded touch-down contact sample (Pose 0 at z_table_m) and fits a perspective-regularized (reg_lambda = 500.0) 3×4 projective matrix M (fit_projective_handeye) plus closed-form ray-plane unprojection pixel_to_world_at_z(M, u, v, z_plane_m).
  1. Verification, Line-of-Sight Gate & Phase 2 Readiness (`R10`, `R14`):
  • Visually inspect touchdown_contact_overhead.png, touchdown_contact_wrist.png, and handeye_calibration_grid.png to verify contact location and projected tabletop metric grid alignment.
  • Confirm rmse_reprojection_px (<= 8.0 px, hold-out < 15.0 px under R10) and rmse_localization_mm (< 3.0 mm at z_table_m), and confirm robot.bus Present_Position returned within 5° of rest_pose with Torque_Enable = 0.
  • Report Stage 5 completion using the 6-field R14 format before proceeding to Phase 2 vision-guided pick-and-place.
  1. Coordinate Conventions & Safety Invariants:
  • Robot base frame (gripper_frame_link): +X forward from base, +Y left, +Z upward (z_table_m typically ~-0.024 m below shoulder base origin).
  • Fixed/overhead camera pixels (u, v) follow top-left image origin (1280×720); pixel_to_world_at_z unprojects any tabletop pixel (u, v) onto horizontal plane Z = z_plane_m.
  • All shared calibration and trajectory helpers live in so101_common.py; never lower z_hard_floor_m below -0.050 m or modify SO101Follower gripper current/torque limits.
같은 저장소의 Skills

더 많은 Skills

모든 Skills
google-deepmind
커뮤니티

blender-creation

Creates physically calibrated, articulated, PBR-textured 3D MuJoCo assets (MJCF model.xml, .mjz, mjSpec) from scratch using headless Blender (bpy, bmesh, blendermeshutils, assetbuildutils), convex collision synthesis, 10-scenario auditing, and Studio preview. Use only when asked to create, model, or build a 3D MuJoCo object from a URL, image, or text description, or when blender-creation is named. Don't use for non-MuJoCo Blender/Cycles/CAD modeling.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

fr3-duo-control

Franka Duo robot only (Franka Research 3 / FR3 Duo dual-arm cell with Robotiq 2F-85 grippers, ZED Mini and RealSense D405 cameras). Brings up, homes, and controls it via pylibfranka on hardware or in MuJoCo sim. Use only when the user names Franka Duo, FR3 Duo, or fr3duo.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-bringup

Checks and installs the I2RT YAM SDK (i2rt), configures Linux socketcan or macOS gsusb CAN bringup for I2RT YAM robot arms, and provides a wave smoke test (wave.py). Use only before bringing up or programming a physical or simulated I2RT YAM robot arm, or when asked to run a wave gesture on an I2RT YAM robot. Don't use for non-YAM CAN bus debugging (e.g. automotive OBD-II or industrial PLC CAN networks) or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일
google-deepmind
커뮤니티

i2rt-yam-ik

Solves 6-DOF inverse kinematics for the I2RT YAM and YAM Pro robot arm using Mink + DAQP on combined YAM + gripper MJCF. Use only when computing joint targets or local Cartesian moves on an I2RT YAM or YAM Pro robot arm. Don't use for non-YAM inverse kinematics, general quadratic programming (QP) solvers, or non-robotics tasks.

1시간 설치 수
2
GitHub Stars
80
업데이트
11월 17일