duckdb/duckdb-skills

s3-explore

Explore and query data on S3, Cloudflare R2, GCS, MinIO, or any S3-compatible storage.

ソースを見る

使い方

  1. インストールコマンドをコピーし、自分の環境へ導入する前にソースを確認します。
  2. 必要なツール、認証情報、対応環境を原文で確認します。
  3. 入力と成果物を指定し、出力を資料と照合します。
npx skills add https://github.com/duckdb/duckdb-skills --skill s3-explore

開始前の確認

  • 導入にはモデル契約、API枠、ファイルへのアクセス許可は含まれません。
  • スター数や導入数は活動の指標であり、独立した品質・安全性試験ではありません。
リポジトリの原文

見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。

You are helping the user explore data on remote object storage using DuckDB.

URL: $0 Question: ${1:-list and describe what's there}

Step 1 — Detect provider and set up credentials

Based on the URL or user context, prepend the appropriate secret configuration:

ProviderURL patternsSecret setup
AWS S3s3://CREATE SECRET (TYPE S3, PROVIDER credential_chain);
Cloudflare R2r2://, s3:// with R2 endpointCREATE SECRET (TYPE R2, PROVIDER credential_chain);
GCSgs://, gcs://CREATE SECRET (TYPE GCS, PROVIDER credential_chain);
MinIO / customs3:// with custom endpointCREATE SECRET (TYPE S3, KEY_ID '...', SECRET '...', ENDPOINT '...', USE_SSL true);

For R2, if the user provides an account ID, the endpoint is <account_id>.r2.cloudflarestorage.com. R2 URLs like r2://bucket/path should be rewritten to s3://bucket/path with the R2 secret.

For public buckets (e.g., Overture Maps, AWS open data), no secret is needed — skip this step.

Always prepend:

sql
LOAD httpfs;

Step 2 — Determine what the URL points to

If the URL looks like a directory or bucket (no file extension, or ends with /), list its contents with sizes:

bash
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
SELECT filename, (size / 1024 / 1024)::DECIMAL(10,1) AS size_mb, last_modified
FROM read_blob('<URL>/*')
ORDER BY filename
LIMIT 50;
"

Note: only select filename, size, last_modified — never select content, which would download the actual files.

If the URL points to a specific file or glob pattern (has a file extension or contains *), preview it:

bash
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
DESCRIBE FROM '<URL>';
SELECT count(*) AS row_count FROM '<URL>';
FROM '<URL>' LIMIT 20;
"

For Parquet files, get row counts and sizes from metadata (no data download):

bash
duckdb -c "
LOAD httpfs;
<SECRET_SETUP>
SELECT file_name,
       sum(row_group_num_rows) AS total_rows,
       (sum(row_group_compressed_bytes) / 1024 / 1024)::DECIMAL(10,1) AS compressed_mb
FROM parquet_metadata('<URL>')
GROUP BY file_name;
"

Step 3 — Answer the question

Using the listing, schema, or sample data, answer:

${1:-list and describe what's there}

If the user asks an analytical question (e.g., "how many rows match X"), write and run the appropriate SQL query. DuckDB pushes predicates down into Parquet on S3, so filtering is efficient even on large remote datasets.

Error handling

  • `duckdb: command not found` → delegate to /duckdb-skills:install-duckdb
  • Access denied / 403 → suggest the user check credentials: aws configure, environment variables, or provide explicit key/secret
  • Bucket not found / 404 → check the URL and region
  • Timeout on large listing → suggest narrowing the glob pattern or adding a prefix

タスクに関連する項目

同じリポジトリから

関連する Skills

すべての Skills
duckdb
コミュニティ

query

Run SQL queries against the attached DuckDB database or ad-hoc against files. Accepts raw SQL or natural language questions. Uses DuckDB Friendly SQL idioms.

1時間の導入数
2
GitHub Stars
610
更新日
4月23日
duckdb
コミュニティ

attach-db

Attach a DuckDB database file for use with /duckdb-skills:query. Explores the schema (tables, columns, row counts) and writes a SQL state file so subsequent queries can restore this session automatically via duckdb -init.

1時間の導入数
1
GitHub Stars
609
更新日
4月23日
duckdb
コミュニティ

convert-file

Convert any data file to another format: CSV, Parquet, JSON, Excel, GeoJSON, and more. Use when the user says "convert to parquet", "save as xlsx", "export as JSON", "make this a CSV", "turn into parquet", or any variation of format-to-format conversion for data files. Also triggers when the user wants to write Parquet, Excel, or other binary formats that Claude cannot produce natively.

1時間の導入数
1
GitHub Stars
609
更新日
4月23日
duckdb
コミュニティ

install-duckdb

Install or update DuckDB extensions. Each argument is either a plain extension name (installs from core) or name@repo (e.g. magic@community). Pass --update to update extensions instead of installing.

1時間の導入数
1
GitHub Stars
609
更新日
4月23日