magnus919/agent-skills

data-engineering

Design and operate data infrastructure — database operations (vector, relational, graph, time-series), ETL/ELT pipeline design (dbt patterns, incremental loading), SQL analytical patterns, data quality monitoring, schema migration, and storage infrastructur…

소스 보기

사용 방법

  1. 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
  2. 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
  3. 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/magnus919/agent-skills --skill data-engineering

시작 전 확인

  • 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
  • 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 Skill 문서

원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.

Data Engineering Methodology

Data engineering is the operational backbone of data-driven systems. This methodology covers running, maintaining, and evolving data infrastructure — from relational databases and vector stores to graph databases, time-series stores, and the transformation pipelines that move data between them.

When a model proposes data transformations, load the AI boundary workflow and use the companion record.

The Data Engineer's Domain

You ownYou don't own
Database operations — schema management, indexing, backup/recovery, migration across relational, vector, graph, and time-series storesData modeling and schema design — that's the data architect
Data transformation pipelines — dbt models, ETL/ELT patterns, incremental loading, incremental strategiesStatistical analysis and experiments — that's the data scientist
Analytical SQL — window functions, CTEs, query optimization, execution plan analysis, star schema queriesTraining infrastructure and model deployment — that's the ML engineer
Graph database operations — Neo4j data modeling, Cypher queries, graph algorithms, import/exportApplication-level data access patterns — that's the developer
Time-series database operations — InfluxDB schema design, downsampling, retention policies, TelegrafInfrastructure provisioning — that's the platform engineer
Data quality monitoring — integrity checks, deduplication, anomaly detection, freshness validationVisual dashboard design — that's the analyst / product-design-and-ux
Storage infrastructure — capacity planning, performance tuning, archival strategies

Valkey and Redis decisions

For in-memory event pipelines, vector/search indexes, compatibility and sink reconciliation, read the focused guide. For open-source-oriented choices, prefer Valkey's BSD licensing and community governance; preserve explicit engine requirements and verify version-specific compatibility. Redis's AGPL option does not erase licensing or trust preferences, and this guidance makes no claim about business-community preference.

Reference Files

ReferenceWhen to load
references/sql-analytical-patterns.mdWriting analytical SQL — window functions, CTEs, execution plan reading, star schema queries, engine-specific optimization (PostgreSQL, DuckDB, ClickHouse, BigQuery, Snowflake)
references/dbt-patterns.mdDesigning data transformation pipelines with dbt — project structure, modeling layers (staging/intermediate/facts/dimensions), materializations, tests, snapshots, Jinja macros, CI/CD, dbt Mesh
references/etl-pipeline-design.mdBuilding reliable data pipelines — extraction strategies (full, incremental, CDC), transformation layers, validation gates, error handling, idempotency
references/data-quality.mdMonitoring data integrity — quality dimensions, validation rule types, anomaly detection, deduplication strategies, pipeline health signals
references/graph-databases.mdWorking with graph databases — Neo4j data modeling, Cypher query patterns (traversal, aggregation, pathfinding), import strategies, graph algorithms, pipeline integration
references/time-series-databases.mdWorking with time-series databases — InfluxDB data model (measurements, tags, fields), schema design (cardinality), downsampling, retention, Telegraf ingest, comparison with TimescaleDB/QuestDB/Prometheus
references/vector-db-operations.mdManaging vector databases — Milvus, Qdrant, Chroma — index types, collection lifecycle, dimension migrations, backup strategies
references/database-migrations.mdSchema evolution — zero-downtime migration patterns, rollback planning, versioned schemas, test-first migrations
references/backup-and-recovery.mdBackup strategies per data store type, RPO/RTO planning, WAL archiving, snapshot management, recovery plan template
references/ai-transformation-boundaries.mdDefining acceptance, retry, cost, and publication boundaries for model-assisted transformations

Related Skills

  • postgres — operating a PostgreSQL server itself: configuration review, index and query-plan diagnosis, vacuum/bloat management, WAL archiving and point-in-time recovery, replication and failover, upgrades. This skill owns the engine-specific runbooks; data-engineering owns the engine-neutral methodology.
  • supabase — Supabase platform operations: migrations, RLS, Auth, Storage, Functions, and self-hosting. To measure an agent's Supabase task competence, use its agent evals harness reference.

Core Principles

Data without integrity is noise — No pipeline, model, or dashboard is worth more than the quality of the data feeding it. Validate at every boundary.

Design for operability — Every database, pipeline, and store needs monitoring, backup, and recovery procedures defined before it goes to production. If you can't detect failure, you can't recover from it.

Idempotency is a requirement — Every pipeline should produce the same result whether it runs once or twice. Duplicate handling is not optional.

Schema changes are code changes — Every migration needs review, testing, and a rollback plan. Schema drift is technical debt with compounding interest.

Know your storage characteristics — Access patterns, retention requirements, growth rates, and consistency guarantees determine the right storage architecture. Choose based on data, not familiarity.

같은 저장소의 Skills

더 많은 Skills

모든 Skills
magnus919
커뮤니티

ai-governance

Design and operate AI governance: principles, decision rights, risk tiers, lifecycle gates, fairness, transparency, privacy, security, compliance mapping, maturity and board reporting across SaaS, API, self-hosted and agentic systems. Use to admit, update or retire Agent Skills, MCP servers and A2A capabilities; approve an agent for production actions; or decide earned-autonomy promotion, reduction or revocation. Cover GxP, ALCOA+, data integrity, electronic records, assurance and QMS interfaces

1시간 설치 수
1
GitHub Stars
172
업데이트
11월 8일
magnus919
커뮤니티

amazon-memo

Create, adapt, or revise Amazon-style narrative decision memos, six-pagers, PR/FAQs, and short decision briefs, including reading meetings and async review. Use for choosing the right memo form, evidence-led reasoning, and revising mechanical AI prose for human readers. Do not use for general copywriting, document rendering, or replacing incident reports and technical design records.

1시간 설치 수
1
GitHub Stars
172
업데이트
11월 8일
magnus919
커뮤니티

analog-occultism

Use this skill to create subject-neutral images and videos in an analog occultism / industrial CRT noir aesthetic: near-monochrome archival technical atmosphere, severe low-key lighting, tactile signal degradation, industrial geometry, and quiet unresolved tension. Use for portraits, products, environments, interfaces, diagrams, abstract forms, stills, and motion. Do not use for colorful cyberpunk, vaporwave, glossy advertising, conventional horror, or clean digital renders.

1시간 설치 수
1
GitHub Stars
172
업데이트
11월 8일
magnus919
커뮤니티

binary-analysis

Plan static inspection of unknown binary files with a deterministic CLI whose Ghidra operations are currently unimplemented and fail closed. Use when you need to inspect a PE, ELF, or Mach-O file — triage suspicious binaries, map imported APIs, decompile functions, trace call paths, or produce structured evidence reports. Do not use for runtime analysis (debugging, dynamic tracing, sandbox execution), for modifying or patching binaries, or for binaries you already know everything about. The skil

1시간 설치 수
1
GitHub Stars
172
업데이트
11월 8일