grafana/skills

adaptive-metrics

Cut Grafana Cloud Metrics cost by shrinking active-series count with Adaptive Metrics aggregation rules — auto-recommendations from query history, custom exact/regex rules, label-drop config, unused-metric detection, and Alloy remotewrite fallback.

查看源码
仓库原始内容

按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。

Grafana Cloud Adaptive Metrics

Docs: https://grafana.com/docs/grafana-cloud/cost-management-and-billing/reduce-costs/metrics-costs/adaptive-metrics.md

Aggregation rules that pre-shrink high-cardinality metrics before storage — directly reduces active-series billing.

Prerequisites

  • Grafana Cloud Metrics plan (any paid tier)
  • API key with metrics:write (for the Adaptive Metrics API — adaptive-metrics.grafana.net, Bearer auth)
  • For the verification queries: the metrics query endpoint (prometheus-prod-XX.grafana.net) uses HTTP basic auth — <metrics_user> (numeric stack/instance ID) plus a token with metrics:read — not the Bearer key
  • Access to Home → Adaptive Metrics in the Cloud portal

Common Workflows

1. Review + apply auto-recommendations

bash
# 1. Pull the recommendation list (sorted by series-reduction impact)
curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/recommendations" \
  | jq '.recommendations[] | {metric_name, current_series, projected_series, estimated_reduction_percent}'

# 2. Capture the baseline series count for the target metric
#    (metrics query endpoint = basic auth, not the Bearer key)
curl -s -u "<metrics_user>:<metrics_token>" \
  "https://prometheus-prod-XX.grafana.net/api/prom/api/v1/query?query=count({__name__=\"process_cpu_seconds_total\"})" \
  | jq '.data.result[0].value[1]'   # → e.g. "12480"

# 3. Apply the recommendation (or click Apply in the UI)
curl -s -X POST -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/recommendations/<ID>/apply"

# 4. Wait ~5 min. Verify — re-run the count query; expect a large drop.
#    Also check the saving metric:
#      grafanacloud_instance_active_series_dropped_by_aggregation_rules

Rollback — delete the rule:

bash
curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/rules" | jq '.rules[] | {id, metric_name}'
curl -s -X DELETE -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/rules/<RULE_ID>"
# Or in the UI: Rules → row → Disable

2. Hand-write a custom rule

bash
# 1. Sanity-check the metric is not used WITH that label in dashboards/alerts
grep -r 'process_cpu_seconds_total' dashboards/ alerts/ | grep -E 'version|go_version'
# Expect no hits → safe to drop.

# 2. Create the rule
curl -s -X POST -H "Authorization: Bearer <KEY>" -H "Content-Type: application/json" \
  "https://adaptive-metrics.grafana.net/api/v1/rules" \
  -d '{"rules":[{"metric_name":"process_cpu_seconds_total","match_type":"MATCH_TYPE_EXACT",
                 "drop_labels":["version","go_version"],
                 "aggregations":[{"type":"AGGREGATION_TYPE_SUM"}]}]}'

# 3. Verify — same count() query as above; series count should drop within 5 min.

Full payloads (regex match, aggregation types, all caveats): `references/api.md`.

3. Drop unused metrics entirely

bash
# 1. List unused metrics
curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/usage-analysis?filter=unused" | \
  jq '.metrics[] | {metric_name, series_count, last_queried}'

# 2. Confirm not referenced in dashboards / alerts / recording rules
grep -r '<METRIC_NAME>' dashboards/ alerts/ recording-rules/

# 3. Add a write_relabel_config drop in Alloy (full block in references/api.md)
#    Reload Alloy: curl -X POST http://localhost:12345/-/reload

# 4. Verify — the metric should no longer appear in series counts after ~10 min
curl -s -u "<metrics_user>:<metrics_token>" \
  'https://prometheus-prod-XX.grafana.net/api/prom/api/v1/label/__name__/values' | jq '.data | index("<METRIC_NAME>")'  # → null

Measure the impact

promql
# Total active series (billed unit)
grafanacloud_instance_active_series

# Series specifically dropped by Adaptive Metrics rules
grafanacloud_instance_active_series_dropped_by_aggregation_rules

Rules take effect within ~5 minutes; full billing impact appears within an hour. The original high-cardinality samples keep flowing but the dropped labels no longer count toward billing.

Resources

来自同一仓库

更多 Skills

全部 Skills
grafana
社区

alerting-irm

Configure Grafana Alerting, Incident Response Management (IRM), and SLOs end-to-end — provisions Grafana-managed and data-source-managed alert rules, contact points (Slack/PagerDuty/email/webhook), notification policies with hierarchical matchers, silences, mute timings, on-call schedules and escalation chains, incident-management integrations, and SLOs with multi-window burn-rate alerts. Use when configuring alerts, debugging notification routing, setting up on-call rotations, declaring or managing incidents, defining SLOs, provisioning alerting via YAML or API, picking matchers for a notification policy, building a PagerDuty/Slack webhook receiver, or troubleshooting why an alert isn't firing — even when the user says "page me on errors", "alert me when X happens", "route this to the platform team", or "set up an SLO" without naming Alerting or IRM.

安装量
5
GitHub Stars
246
最近更新
9月8日
grafana
社区

alloy

Build a unified telemetry pipeline with Grafana Alloy — one OpenTelemetry-compatible binary that collects metrics, logs, traces, and profiles and ships to Grafana Cloud / Prometheus / Loki / Tempo / Pyroscope. Covers the Alloy config language (blocks, sys.env, component refs), prometheus.scrape → remotewrite, loki.source.file + loki.process → loki.write, otelcol.receiver.otlp → otelcol.exporter.otlp, pyroscope.scrape, K8s / Docker / EC2 discovery, relabeling, modules (import.file/git/http), clustering, Fleet Management remotecfg, the Alloy UI at :12345, and alloy fmt / alloy validate. Use when writing a config.alloy, replacing Grafana Agent / OTel Collector, scraping K8s pods, parsing logs, ingesting OTLP, or debugging "Alloy isn't sending anything" — even when the user says "set up the agent", "write me a scrape config", "drop these logs before sending", or "OTel collector config" without naming Alloy.

安装量
5
GitHub Stars
246
最近更新
9月8日
grafana
社区

beyla

Auto-instrument an application's HTTP / gRPC / DB traffic with Grafana Beyla eBPF — no code changes, no SDK, no restart. Covers requirements (Linux 5.8+ with BTF, CAPSYSADMIN, host PID), language matrix (Go / Java / Python / Ruby / Node / .NET / Rust / C++ / PHP), Docker + Helm + DaemonSet install, port- / process- / Kubernetes-metadata discovery, OTLP traces + Prometheus metrics export, routes decorator (cardinality control), trace sampling, and Grafana Cloud via Alloy. Use when adding observability to a service you can't recompile, instrumenting a closed-source binary, getting RED metrics + spans onto Tempo/Mimir without touching the app, or rolling Beyla as a cluster-wide DaemonSet — even when the user says "zero-code APM", "instrument legacy app", "trace this binary", "eBPF observability", or "no SDK" without naming Beyla.

安装量
5
GitHub Stars
246
最近更新
9月8日
grafana
社区

dashboarding

Build, modify, and ship Grafana dashboards as JSON via the HTTP API — panel types (timeseries / stat / gauge / table / heatmap / logs / traces / node-graph), gridPos 24-column layout, units, thresholds, template + datasource + chained variables, transformations (organize / calculateField / filterByValue), panel + dashboard links with ${field.labels.x} / ${from}, and Loki/Prometheus annotations. Use when scripting dashboard creation, writing the dashboard JSON for a new service, adding a $job dropdown variable, computing an "Error %" column with a transformation, overlaying deploys as annotations, or pushing a dashboard via POST /api/dashboards/db — even when the user says "create a dashboard for this metric", "add a service dropdown", "show errors as percentage", "overlay our deploys", or "export the dashboard JSON" without naming the API or schema. After every API push, verify with the returned version plus a GET on the dashboard UID.

安装量
5
GitHub Stars
246
最近更新
9月8日