사용 방법
- 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
- 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
- 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/google-deepmind/robotics-antigravity-plugin --skill rgb-object-localization시작 전 확인
- 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
- 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.
RGB-Only 3D Object Localization (`rgb_localization.py`)
Localize metric 3D coordinates and principal axes from RGB-only cameras using bundled `rgb_localization.py` (opencv-python numpy, plus google-genai and kornia torch when applicable):
- Camera Verification & 2D Target Detection (`gemini-4-hydrogen` or Agent VLM + Local Mask Fallback):
- Flush stale OS buffer frames before static captures and visually verify camera stream roles once on startup (never assume OS device index order).
- With `GEMINI_API_KEY`: Query
client.models.generate_content(model="gemini-4-hydrogen", ...)(thinking_level="low","high"in clutter) for0–1000JSON[{"label": ..., "point": [y, x], "box_2d": [ymin, xmin, ymax, xmax]}](u = x * W / 1000,v = y * H / 1000). - Fallback when `GEMINI_API_KEY` is unset: Save one snapshot per camera, inspect once with native VLM (
read_file) to confirm camera roles and target appearance, and segment the target foreground mask locally with OpenCV color/contrast thresholding (passing an optionalroi_maskto exclude any robot/gripper geoms visible in the FOV).
- Metric 3D Mapping (Choose by Camera & Scene Geometry):
- Calibrated Camera + Known Support Plane (`ray_plane_contour_pose`): Given calibrated intrinsics
K, camera poseT_world_cam = T_world_ee(q_actual) @ T_ee_cam, and target plane heightz_plane_m(accounting for measured support height and visible object face/centroid height), callray_plane_contour_pose(mask, K, T_world_cam, z_plane_m, roi_mask=roi_mask). Only align wrist yaw toyaw_radwhenanisotropy >= 2.0; keep neutral wrist orientation on symmetric objects. - Uncalibrated Static Camera + Planar Workspace (`fit_plane_homography` & `map_homography_xy_and_yaw`): Move the end-effector to
>= 4non-collinear(x_fk, y_fk)points onz_plane_m, detect fingertip motion centroids viacv2.absdiff(img_open, img_closed), fitH = fit_plane_homography(pts_uv, pts_xy), and map target(u_px, v_px)+ tangent(vx_px, vy_px)viamap_homography_xy_and_yaw. - Arbitrary 3D / Unknown-Plane Targets (`triangulate_matched_points`): Capture 2 calibrated views across a sufficient baseline (
>= 8 cm) using measuredq_actualFK. Match textured crops withkornia >= 0.8KF.DISK.from_pretrained("depth")+KF.LightGlueMatcher("disk")viaK.image.image_to_tensor(KF.SuperPointwas removed in Kornia 0.8) — or pass matched 2D centroids directly for smooth/textureless targets — intotriangulate_matched_points(K1, T1, pts1_uv, K2, T2, pts2_uv).