goldsky-io/goldsky-agent

turbo-transforms

Write SQL, TypeScript, and dynamic table transforms for Turbo pipelines.

View source

How to use

  1. Copy the installation command and review the source package before installing it in your own agent environment.
  2. Check the source document for required tools, credentials and supported hosts.
  3. Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/goldsky-io/goldsky-agent --skill turbo-transforms

Before you start

  • Installation does not include a model subscription, API allowance or permission to access your files.
  • Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

Turbo Transforms

Write, understand, and debug SQL, TypeScript, and dynamic table transforms for Turbo pipeline configurations.

Identify what the user needs (decode, filter, reshape, combine, custom logic, or lookup joins), then use the relevant section below. If generating a complete pipeline YAML (not just a transform snippet), always validate with goldsky turbo validate before presenting it to the user.

Reference files for specialized topics:

  • references/evm-patterns.md — Advanced EVM patterns (decode-once-filter-many, UNION ALL, schema normalization)
  • references/typescript-transforms.md — TypeScript/WASM script transforms and handler transforms
  • references/dynamic-tables.md — Dynamic table transforms (allowlists, lookup joins)
  • references/solana-patterns.md — Solana instruction/log decoding and function examples

Transform Basics

YAML Structure

yaml
transforms:
  my_transform:
    type: sql
    primary_key: id
    sql: |
      SELECT id, block_number, address
      FROM my_source
      WHERE address = '0xabc...'

Required Fields

FieldRequiredDescription
typeYessql, script, handler, dynamic_table, or throttle
primary_keyYes\*Column used for uniqueness and ordering (\*not for throttle)
sqlYesSQL query (for sql type transforms)

Referencing Data

  • Reference sources by their YAML key name: FROM my_source
  • Reference other transforms by their YAML key name: FROM my_transform
  • No need for a from field in SQL transforms — the FROM clause in SQL handles it

SQL Streaming Limitations

Turbo SQL is powered by Apache DataFusion in streaming mode. The following are NOT supported:

  • Joins — use dynamic_table transforms for lookup-style joins instead
  • Aggregations (GROUP BY, COUNT, SUM, AVG) — use postgres_aggregate sink instead
  • Window functions (OVER, PARTITION BY, ROW_NUMBER)
  • Subqueries — use transform chaining instead
  • CTEs (WITH...AS) are supported

The _gs_op Column

Every record includes a _gs_op column that tracks the operation type: 'i' (insert), 'u' (update), 'd' (delete). Preserve this column through transforms for correct upsert semantics in database sinks.


Key SQL Functions

Quick reference for the most-used functions. For the complete function reference, see SQL transforms in the docs.

evm_log_decode() — Decode Raw EVM Logs

sql
evm_log_decode(abi_json, topics, data) -> STRUCT<name, event_params>
-- Aliases: _gs_log_decode, evm_decode_log

Returns decoded.event_signature (event name) and decoded.event_params[N] (1-indexed parameters as strings).

Example:

sql
SELECT
  _gs_log_decode('[{"anonymous":false,"inputs":[{"indexed":true,"name":"from","type":"address"},{"indexed":true,"name":"to","type":"address"},{"indexed":false,"name":"value","type":"uint256"}],"name":"Transfer","type":"event"}]',
    topics, data) AS decoded,
  id, block_number, transaction_hash, address, block_timestamp
FROM my_raw_logs

Tips: Backtick-escape reserved words (` data , decoded `). Include all event ABIs in one array. Pre-filter by contract address at the source level for efficiency.

Other Key Functions

FunctionPurpose
fetch_abi(url, format)Fetch ABI from URL ('raw' or 'etherscan' format)
_gs_keccak256(string)Keccak256 hash (returns hex with 0x prefix)
xxhash(string)Fast non-cryptographic hash for composite keys
to_u256(value) / u256_to_string(u256)256-bit integer math (avoids precision loss)
dynamic_table_check(table, value)Check if value exists in a dynamic lookup table
to_timestamp(seconds)Convert unix seconds to timestamp

Common Transform Patterns

Table Aliasing

sql
SELECT balance.token_type, balance.owner_address,
  CAST(`balance` AS STRING) AS balance_amount
FROM my_balances_source balance
WHERE balance.token_type = 'ERC_20' AND balance.balance IS NOT NULL

Simple Filtering

yaml
transforms:
  usdc_transfers:
    type: sql
    primary_key: id
    sql: |
      SELECT * FROM base_transfers
      WHERE address = lower('0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913')

Column Projection and Aliasing

yaml
transforms:
  clean_transfers:
    type: sql
    primary_key: id
    sql: |
      SELECT id, block_number, to_timestamp(block_timestamp) AS block_time,
        address AS token_address, sender, recipient, amount
      FROM my_source

Type Casting and Numeric Scaling

sql
-- Token amount conversions
(CAST(decoded.event_params[3] AS DOUBLE) / 1e6) AS amount_usdc    -- 6 decimals
(CAST(decoded.event_params[3] AS DOUBLE) / 1e18) AS amount_eth    -- 18 decimals
CAST(`balance` AS STRING) AS balance_amount                        -- preserve precision
TokenDecimalsDivisor
USDC, USDT61e6
WBTC81e8
ETH, most ERC-20181e18

Conditional Logic with CASE WHEN

sql
CASE WHEN decoded.event_params[4] = '0' THEN 'BUY' ELSE 'SELL' END AS side

Exclusion Filters

sql
WHERE decoded.event_params[1] NOT IN ('0x4bfb...', '0xc5d5...', '0x4d97...')

Transform Chaining

Build multi-step pipelines where each transform reads from the previous:

yaml
transforms:
  decoded:
    type: sql
    primary_key: id
    sql: |
      SELECT _gs_log_decode('[...]', topics, data) AS decoded,
        id, block_number, transaction_hash, address, block_timestamp
      FROM raw_logs_source
  transfers:
    type: sql
    primary_key: id
    sql: |
      SELECT id, block_number, decoded.event_params[1] AS from_address,
        decoded.event_params[2] AS to_address,
        (CAST(decoded.event_params[3] AS DOUBLE) / 1e18) AS amount
      FROM decoded
      WHERE decoded.event_signature = 'Transfer'

UNION ALL — Combining Multiple Event Types

Combine transforms with identical schemas into a single output:

yaml
  all_events:
    type: sql
    primary_key: id
    sql: |
      SELECT * FROM transfers
      UNION ALL
      SELECT * FROM approvals

All branches must have the same columns in the same order. Use '' or 0 as placeholders.

For advanced EVM patterns (decode-once-filter-many, normalizing disparate events, multi-event activity feeds), read references/evm-patterns.md.


Source-Level Filtering

Filter data at the source before it reaches transforms for efficiency:

yaml
sources:
  my_logs:
    type: dataset
    dataset_name: matic.raw_logs
    version: 1.0.0
    start_at: earliest
    filter: >-
      address IN ('0xabc...', '0xdef...')
      AND block_number >= 82422949
  • Source filter: → coarse pre-filtering (contract addresses, block ranges)
  • Transform WHERE → fine-grained filtering (event types, parameter values, exclusions)

Sink Batching Configuration

yaml
sinks:
  my_sink:
    type: clickhouse
    from: my_transform
    table: my_table
    secret_name: MY_SECRET
    primary_key: id
    batch_size: 1000
    batch_flush_interval: 300ms
Use Casebatch_flush_intervalbatch_size
Real-time dashboards300ms1000
Moderate throughput1000ms1000
High-volume streaming10s100000

Debugging Transforms

"Unknown source reference" — FROM clause name doesn't match any source/transform key. Check for typos.

"Missing primary_key" — Every transform needs primary_key. Almost always use id.

"Column not found" — Use goldsky turbo inspect <pipeline> -n <source_node> -p to see actual columns.

Empty results from decoded events:

  • Verify ABI JSON matches actual contract events
  • Check decoded.event_signature matches event name exactly (case-sensitive)
  • Ensure address filter matches correct contract
  • Parameters are 1-indexed: decoded.event_params[1], not [0]

Type mismatch in UNION ALL — All branches need identical column counts and compatible types.

bash
goldsky turbo inspect <pipeline-name> -n <transform_name> -p

TypeScript / WASM Script Transforms

For logic SQL can't express: custom parsing, BigInt arithmetic, stateful processing. Schema types: string, uint64, int64, float64, boolean, bytes.

Key rules: define function invoke(data), return null to filter, return object matching schema, no async/await or external imports.

See references/typescript-transforms.md for full docs, examples, and the SQL-vs-script decision table. Also includes Handler transforms for HTTP enrichment.


Dynamic Table Transforms

Updatable lookup tables for allowlists, blocklists, and join-style enrichment — without redeploying. Backed by PostgreSQL (durable) or in-memory (fast). Use dynamic_table_check('table_name', column) in SQL.

See references/dynamic-tables.md for full config, backend options, REST API updates, and wallet-tracking example.


Throttle Transform (Flow Control)

Caps the throughput of a stream by buffering records into batches and emitting each batch on a fixed minimum interval. Throttle does not modify data — every input record passes through unchanged. Use it to:

  • Stay under rate limits of downstream sinks or external APIs
  • Smooth bursty sources into a steady, predictable rate
  • Pace records into an HTTP handler so the receiving service is not overwhelmed
  • Slow down processing during development or testing
yaml
transforms:
  throttled:
    type: throttle
    from: my_source       # source or transform
    max_batch_size: 100
    min_batch_interval: 10s
FieldRequiredDescription
typeYesthrottle
fromYesSource or transform to read from
max_batch_sizeNoMax records per batch
min_batch_intervalNoMinimum time between batches (e.g. 10s, 500ms, 1m)

A batch is flushed downstream when both max_batch_size records have accumulated and min_batch_interval has elapsed since the previous batch. Effective max throughput ≈ max_batch_size / min_batch_interval records per second.

Best practices:

  • Place throttle close to the bottleneck (just before the rate-limited sink/handler) so upstream transforms still process at full speed.
  • Tune max_batch_size to match the downstream sink's preferred batch size.
  • Throttle is a transform — chain other transforms downstream from it the same way you would any other transform (e.g. from: throttled in a sink).
  • Throttle does not have a primary_key field (records pass through unchanged).
  • Remove throttle in production where possible; it caps throughput by design.

Solana Transform Patterns

IDL-based instruction decoding, program-specific decoders (Token, System, Stake, Vote), SPL token tracking.

See references/solana-patterns.md for all built-in decoders, full example pipeline, and array/JSON/hex function usage.


Related

  • `/turbo-builder` — Build and deploy pipelines interactively using these transforms
  • `/turbo-doctor` — Diagnose and fix pipeline issues (including transform errors)
  • `/turbo-pipelines` — Pipeline YAML configuration and architecture reference
  • `/turbo-operations` — Lifecycle commands and monitoring reference
  • `/datasets` — Blockchain dataset and chain prefix reference

Related by task

from this repository

More skills

All skills
goldsky-io
Community

auth-setup

Set up Goldsky CLI authentication and project configuration. Use this skill when the user needs to: install the goldsky CLI (what's the official install command?), run goldsky login (including when the browser opens but 'authentication failed'), run goldsky project list and see 'not logged in' or 'unauthorized', switch between Goldsky projects, check which project they're currently authenticated to, or fix 'unauthorized' errors when running goldsky turbo commands. Also use for 'walk me through s

1h installs
4
GitHub stars
12
Updated
Nov 2
goldsky-io
Community

boost

Configure and troubleshoot Goldsky Boost, the free RPC caching CDN in front of a provider the user already pays for. Use for cutting RPC bills without migrating, connecting Alchemy/QuickNode/Infura upstreams, goldsky boost, edge.goldsky.com/boost, verification, cache hit rates, ethgetLogs forwarding, and x-cache / x-edge-billable / x-edge-source headers. Boost keeps the existing provider and forwards uncached requests. To replace a provider with Goldsky's managed RPC, use /edge. Pipeline, Subgra

1h installs
4
GitHub stars
12
Updated
Nov 2
goldsky-io
Community

datasets

Use this skill when the user needs to look up or verify Goldsky blockchain dataset names, chain prefixes, dataset types, or versions. Triggers on questions like 'what\\'s the dataset name for X?', 'what prefix does Goldsky use for chain Y?', 'what version should I use for Z?', or 'what datasets are available for Solana/Stellar/Arbitrum/etc?'. Also use for chain-specific dataset questions (e.g., polygon vs matic prefix, stellarnet balance datasets, solana token transfer dataset names), and to ans

1h installs
4
GitHub stars
12
Updated
Nov 2
goldsky-io
Community

edge

Configure and troubleshoot Goldsky Edge, Goldsky's managed EVM RPC endpoint service. Use for Edge RPC, Goldsky RPC endpoints, edge.goldsky.com, slow ethgetLogs, rate limits, hedged requests, flashblocks, HyperEVM system transactions, x402 pay-per-request RPC, pricing, the Edge dashboard, gsedge keys, rpc-edge, and errors -32005/-32012/-32014/-32015/-32016. Covers resilient low-latency EVM endpoints and provider comparisons. For self-hosted eRPC configuration, consult docs.erpc.cloud/llms.txt. To

1h installs
4
GitHub stars
12
Updated
Nov 2