databricks-data-discovery

Databricks Data Discovery

This skill routes data work — decide first:

• the request is about the data — finding it, asking questions of it, or

generating SQL → delegate to Genie One: databricks genie ask -s <session-label> "..." (see Routing below).

• writing files or anything else → use your own coding-agent tools.

Genie One just needs an authenticated CLI profile (the parent databricks-core skill covers auth/profiles if you need it) — but route the data request to Genie first; don't detour into manual catalog browsing.

Routing — Genie vs your coding agent

Route to Genie when the request is about the data:

• Finding or locating data — "what tables are in wanderbricks?", "where does X

live?", which catalog/schema holds something. Let Genie resolve a fuzzy or partial name across all your data instead of guessing or grepping catalogs yourself.

• Questions about the data — "how many…", "what's the average…", "which … the

most…", trends, breakdowns.

• Generating SQL from a question — a first-cut query you can run, adapt, or drop

into a file.

• Understanding or profiling data to inform a query, dashboard, or app.

Route to your own coding agent (do NOT use Genie) for everything else:

• Writing or editing files — a .sql file, notebook, dashboard, app, config.

Genie finds the data and produces the SQL; you write the file.

• Authoring jobs, pipelines, apps, or DABs — use those product skills.
• DDL or writes — CREATE / INSERT / UPDATE / DELETE.
• General programming, git, infra, and any non-data task.

Key principle: data discovery, data questions, and query generation → Genie One. Everything else → your coding agent.

Why prefer Genie for data questions

Genie runs inside the Databricks data plane with governed, first-hand access to the org's Unity Catalog metadata, metric views, and curated semantic context — context you do not have when reverse-engineering schemas with ad-hoc SQL. For data questions it is often higher-quality and more performant than doing the discovery yourself, and it keeps improving as a managed Databricks capability. Don't default to writing your own discovery SQL just because you can.

How to ask Genie

Always pass a session label with -s, and prefer reusing the same one: a follow-up can only continue a conversation if the first ask set the session label, and reusing it lets later questions build on everything asked so far ("summarize all of the above"). Use a fresh session label only to start a deliberately separate session, or distinct session labels to run several in parallel.

The command is databricks genie ask (CLI >= v1.9.0). On an older CLI it lives under databricks experimental genie ask — same flags and behavior; use that exact fallback if databricks genie ask is not found.

# Always pass a session label, and reuse the SAME one so follow-ups build on each other
databricks genie ask -s trips "How many bookings were there last week?"
databricks genie ask -s trips "Break that down by destination"
databricks genie ask -s trips "Summarize all of the above"

# --include-sql also prints the SQL Genie ran (use it to generate a query, too)
databricks genie ask -s trips "Write SQL for the top 5 destinations by revenue" --include-sql

# --output json gives a parseable result
databricks genie ask -s trips "Top 5 destinations by revenue" --output json
# → {"status":"completed","conversation_id":"…","text":"…","tool_calls":[{"name":"execute_sql","sql":"…","title":"…"}]}

# Older CLI (< v1.9.0) — same command under the deprecated experimental alias:
#   databricks experimental genie ask -s trips "How many bookings were there last week?"

Genie searches across all the data you can see, runs SQL, and streams a grounded answer — rendered with the executed SQL and, where it helps, a terminal chart. It auto-resolves a SQL warehouse (override with --warehouse-id); nothing to pick or set up.

• Streams live: the answer, the agent's steps, and any SQL/results appear as

they arrive. Answers usually take ~5–30s; a stalled stream (no data for ~10 min) fails with a clear message, and Ctrl-C or kill (SIGTERM) cancels cleanly.

• Picking a session label: any string works — a topic like trips, or $$ for a

per-shell session. Default to reusing one session label so follow-ups keep full context; use a fresh one only for a deliberately separate session. An expired session label just starts fresh on the next ask. No id to copy around.

• Parallelism: to run sessions at the same time, give each its own session label

(-s q1, -s q2, …) — independent session labels don't interfere. Within a single session label keep calls sequential: send a follow-up after the previous turn returns, and never fire two asks at once on the same session label (they'd split into two conversations and only one mapping would survive).

• Structured output: --output json gives `{status, conversation_id, text,

toolcalls[]}`, where `toolcalls includes the SQL Genie executed; --raw dumps the raw event stream. Note --output json` buffers and prints once at the end (no live streaming) — use it for parsing, the default text output for interactive use.

• Generating a query: ask Genie to "write SQL for …" and read the SQL from the

response (it's in the answer text, and --include-sql also shows the query Genie ran to verify it — so the SQL is known-good). Genie resolves the schema and joins for you, so this beats hand-writing SQL against unfamiliar tables.

• For exact/full rows: Genie shows a preview inline; to pull the complete result

set locally, copy its SQL (--include-sql or the JSON tool_calls) into the parent's ... aitools tools query "<SQL>".

• A non-answer is a message, not an error: if Genie refuses or "couldn't find

relevant data," don't retry — use the manual fallback below.

Naming: "Genie One" is the current name for this cross-data chat — formerly "Databricks One", then "OneChat" (the backend tool is still literally named onechat). All the same thing.

If Genie One isn't available — manual fallback

Only fall back if Genie One is genuinely unavailable — first verify with databricks genie ask --help (or databricks experimental genie ask --help on a CLI older than v1.9.0); don't assume the command is missing. When Genie One isn't enabled, the CLI is too old to have either form of genie ask, or Genie can't cover the question, do the discovery yourself with the parent skill's commands — see [Manual Data Exploration](../databricks-core/manual-data-exploration.md) (keyword search via information_schema, discover-schema, and tools query). Running known SQL or profiling a known table that way is perfectly fine on its own. Do not default to databricks tables list or raw UC REST for data-location questions — invoke this skill and ask Genie first.

Relationship to the Genie One MCP

Databricks also offers this capability as a managed MCP server — the Genie One MCP. This skill delivers the same functionality through the Databricks CLI, with no MCP server to configure or host. More broadly, the Databricks Agent Skills cover the same ground as Databricks' managed MCP servers, so you don't need any MCP wired up to use this. If you already run the Genie One MCP, use whichever you prefer — they hit the same Genie backend.

Related Skills

• databricks-genie-agents — build and manage Genie Agents: curated

agents that let you or a group ask questions of specific data (create, configure, import/export).

• databricks-core (parent) — CLI auth, profiles, and the manual data exploration

reference used as the fallback.