aws/agent-toolkit-for-aws

storing-and-querying-vectors

Store and query vector embeddings using Amazon S3 Vectors, a cost-effective long-term vector storage service with its own API namespace (s3vectors).

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill storing-and-querying-vectors

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

Store and Query Vectors with Amazon S3 Vectors

Overview

Amazon S3 Vectors is a cost-effective AWS service for storing and querying vector embeddings at scale. Optimized for long-term storage with subsecond latency for cold queries, as low as 100ms for warm queries.

Decision Guide

  • Hundreds/thousands of sustained queries per second (QPS): Wrong tool. Recommend OpenSearch.
  • Hybrid search, aggregations, faceted search: Recommend OpenSearch with S3 Vectors as storage engine. For OpenSearch integration, search AWS docs for "Using S3 Vectors with OpenSearch Service".
  • Tiered (bulk + hot): S3 Vectors for storage + OpenSearch Serverless for real-time. See references/limits-and-patterns.md.
  • Cost-effective storage, infrequent queries, RAG: S3 Vectors is the right fit. Proceed.

For latest guidance, search AWS docs for "S3 Vectors best practices".

Common Tasks

Classify the request before starting:

  • Simple query: Existing index, skip to Step 6
  • Standard: You MUST list existing indexes first and suggest reusing if relevant. Else, new index + store vectors, follow Steps 2-6
  • Migration or multi-tenant: Read references/limits-and-patterns.md first, then Steps 2-6

You MUST execute commands using AWS MCP server tools when connected. Fall back to AWS CLI only if AWS MCP is unavailable. You MUST explain each step to the user before executing.

1. Verify Dependencies

Constraints:

  • You MUST check whether AWS MCP tools or AWS CLI is available and inform user if missing
  • You MUST confirm target AWS region

2. Create a Vector Bucket

You MUST confirm bucket name with user. Names: 3-63 chars, lowercase letters, numbers, hyphens only. Encryption (SSE-S3 default or SSE-KMS for compliance) is immutable after creation.

bash
aws s3vectors create-vector-bucket \
  --vector-bucket-name <BUCKET_NAME>

Constraints:

  • You MUST explain encryption cannot be changed after creation
  • For SSE-KMS, KMS key policy MUST grant kms:GenerateDataKey and kms:Decrypt to the S3 Vectors service principal indexing.s3vectors.amazonaws.com. You MUST use full KMS key ARN (not alias). See references/limits-and-patterns.md for command example.

3. Create a Vector Index

Every parameter is immutable after creation.

Pre-flight checklist (confirm ALL with user):

  1. Dimension (required, integer 1-4096) -- MUST match embedding model output
  2. Distance metric (required) -- cosine or euclidean. Use embedding model's recommended metric;
  3. Non-filterable metadata keys (optional, max 10, 1-63 chars) -- Declare at creation or lose forever. For Bedrock Knowledge Bases integration, search AWS docs for "S3 Vectors Bedrock Knowledge Bases prerequisites" to get the required key names.
  4. Encryption (optional) -- Inherits from bucket. Override per-index if needed.
bash
aws s3vectors create-index \
  --vector-bucket-name <BUCKET_NAME> \
  --index-name <INDEX_NAME> \
  --dimension <DIM> \
  --distance-metric <cosine|euclidean> \
  --data-type float32 \
  --metadata-configuration '{"nonFilterableMetadataKeys":["<KEY1>","<KEY2>"]}'

Omit --metadata-configuration if no non-filterable keys are needed.

Index names: 3-63 chars, lowercase, numbers, hyphens, dots. Unique within bucket. Filterable metadata: 2 KB limit. Total metadata (filterable + non-filterable combined): 40 KB. See references/metadata-filtering.md.

4. Generate Embeddings (if needed)

Skip to Step 5 (store) or Step 6 (query) if user already has embeddings.

Constraints:

  • You MUST ask which embedding model to use if not specified
  • You MUST NOT assume a default model
  • Dimension MUST match Step 3
  • You MUST use the same model for both storing and querying

Generate embeddings with Bedrock invoke-model:

bash
aws bedrock-runtime invoke-model \
  --model-id <MODEL_ID> \
  --content-type application/json \
  --cli-binary-format raw-in-base64-out \
  --body '{"inputText": "your text"}' \
  invoke-model-output.json

You MUST use --cli-binary-format raw-in-base64-out for CLI v2. Output file is required for CLI. The response key is model-dependent (e.g., embedding for Titan, embeddings for Cohere). For Titan, parse with json.load(open('invoke-model-output.json'))['embedding']. Use embedding array as float32 in put-vectors or query-vectors. For batch embedding generation, use AWS SDK or CLI.

5. Put Vectors

bash
aws s3vectors put-vectors \
  --vector-bucket-name <BUCKET_NAME> \
  --index-name <INDEX_NAME> \
  --vectors '[{"key":"<ID>","data":{"float32":[<EMBEDDING>]},"metadata":{"topic":"science"}}]'

Constraints:

  • You MUST NOT exceed 500 vectors per call
  • You SHOULD batch vectors for cost optimization
  • For bulk operations, You SHOULD use an SDK instead of CLI -- vector payloads may be too large for shell arguments
  • You MUST implement retry with backoff on 429 TooManyRequestsException
  • See references/limits-and-patterns.md for batch patterns

6. Query Vectors

Generate embedding if needed (Step 4), then query:

bash
aws s3vectors query-vectors \
  --vector-bucket-name <BUCKET_NAME> \
  --index-name <INDEX_NAME> \
  --query-vector '{"float32":[<EMBEDDING>]}' \
  --top-k 10 \
  --return-distance

Optional: add --return-metadata and/or --filter '{"topic":{"$eq":"science"}}' (both require GetVectors permission). See references/metadata-filtering.md.

Example response body: {"vectors": [{"key": "id1", "distance": 0.45, "metadata": {"topic": "science"}}, ...], "distanceMetric": "cosine"}

Constraints:

  • Using --filter or --return-metadata requires both s3vectors:QueryVectors AND s3vectors:GetVectors IAM permissions. Without GetVectors, these options return 403.

Troubleshooting

ErrorCauseFix
DimensionMismatchDims don't match indexUse matching model, or delete/recreate index (confirm with user -- destroys all vectors).
403 Forbidden with --filter or --return-metadataMissing s3vectors:GetVectorsAdd s3vectors:GetVectors to IAM policy.
Fewer results than --top-kFew vectors match filterExpected -- filtering is inline. Broaden filter.
429 TooManyRequestsExceptionExceeded per-index rate limitsRetry with backoff. Shard across indexes for sustained throughput. Search AWS docs for "S3 Vectors limitations and restrictions" for current limits.
AccessDeniedExceptionMissing s3vectors:* IAM actionsS3 Vectors uses s3vectors:* namespace, not s3:*. Update IAM policy.
RequestTimeoutException or service unavailableRequest timeout or region not supportedRetry request. For regional availability, search AWS docs for "S3 Vectors limitations and restrictions".

Additional Resources

관련 작업

같은 저장소의 Skills

더 많은 Skills

모든 Skills
aws
공식

amazon-aurora-mysql

Amazon Aurora MySQL — creates, modifies, and advises on Aurora MySQL clusters specifically (MySQL-compatible engine, Aurora serverless, parallel query). Trigger for Aurora MySQL cluster operations, ACU sizing, I/O-Optimized storage, commitment pricing, or MySQL upgrade planning. Aurora MySQL uses full (VPC-based) configuration — express configuration is PostgreSQL-only. For Aurora PostgreSQL, use amazon-aurora-postgresql instead. Contains safety guardrails and response templates that override de

1시간 설치 수
2
GitHub Stars
3.5천
업데이트
11월 20일
aws
공식

amazon-aurora-postgresql

Amazon Aurora PostgreSQL — creates, modifies, and advises on Aurora PostgreSQL clusters specifically (PostgreSQL-compatible engine, Aurora serverless, express configuration, pgvector, Babelfish). Trigger for Aurora PostgreSQL cluster operations, express-configuration quick-start, ACU sizing, I/O-Optimized storage, commitment pricing, or PostgreSQL upgrade planning. For Aurora MySQL, use amazon-aurora-mysql instead. Contains safety guardrails, express-first routing, and response templates that ov

1시간 설치 수
2
GitHub Stars
3.5천
업데이트
11월 20일
aws
공식

amazon-bedrock

Builds generative AI applications on Amazon Bedrock. Covers model invocation (Converse API, InvokeModel), RAG with Knowledge Bases, Bedrock Agents, Guardrails, and AgentCore (including the Harness managed agent loop). Applies when invoking models, setting up or migrating Knowledge Bases to Managed, deploying to AgentCore, migrating/porting/converting a Bedrock Agent (including inline agents) to an AgentCore Harness, troubleshooting Bedrock errors (ThrottlingException, AccessDeniedException), or

1시간 설치 수
2
GitHub Stars
3.5천
업데이트
11월 20일
aws
공식

amazon-braket

Runs quantum workflows on AWS / Amazon Braket: circuits and algorithms, QPU/device and ARN discovery, LocalSimulator, tasks, program sets, Hybrid Jobs, S3 results, pricing, account setup, and spending limits. Applies to quantum programs and execution, AHS, OpenQASM, pulse control, reservations, notebooks, or Qiskit or PennyLane integrations.

1시간 설치 수
2
GitHub Stars
3.5천
업데이트
11월 20일