website-monitoring

Website change monitoring

Create, list, update, and delete monitors that watch a URL (exact content diff) or a sitemap (added/removed URL tracking) on a schedule, with signed webhook delivery on change — all as normalized JSON from the Crawlora API, with no polling loop or diffing logic of your own to write.

When to use this skill

• "Watch this page and tell me when it changes." / "Track this competitor's

pricing page."

• "Alert me when new pages get added to this site" (sitemap mode).
• "List my active monitors." / "Pause/update/delete this monitor."
• "Show me this monitor's recent check history" — did it run, did it find a

change, did the webhook deliver.

• Anything that would otherwise mean writing your own cron job + diffing +

notification pipeline for "let me know when X changes."

Setup (one-time)

• Get a free Crawlora API key (2,000 credits/mo, no card) at https://crawlora.net.
• Set CRAWLORA_API_KEY in the environment before running the helper.
• The helper reads CRAWLORA_API_KEY from the environment and sends requests to https://api.crawlora.net/api/v1. Missing/invalid key → 401.

How it works

1. Create — POST /monitors with url, cadence_minutes (5 to

10080, i.e. 5 minutes to 7 days), and optionally target_type (page, the default — exact SHA-256 content-fingerprint diff — or sitemap, which tracks a sitemap's <loc> entries for additions and removals, with optional sitemap.include_patterns/exclude_patterns glob filters and sitemap.max_urls). Add notification.webhook_url (must be https and resolve to a public address) and an optional notification.webhook_secret to get a signed delivery on change. notification.events opts into change.detected (default — fires only when a check finds a difference) and/or run.completed (fires on every completed run, a heartbeat for a "last checked" UI).

1. List / get — GET /monitors lists the caller's own monitors (newest

first, capped at 100, free to call); GET /monitors/{id} fetches one.

1. Update — PATCH /monitors/{id} partially updates a monitor (free to

call). Changing target_type or sitemap resets the stored diff baseline (fingerprint, snapshot, or URL set), so the next check starts fresh rather than comparing against the old target's state.

1. Delete — DELETE /monitors/{id} removes a monitor.
2. Check history — GET /monitors/{id}/checks lists the monitor's most

recent check runs (newest first, capped at 50), including per-run webhook delivery status — use this to debug "why didn't I get notified."

1. Verify webhook deliveries — every delivery carries an

X-Crawlora-Signature: t=<unix>,v1=<hmac-hex> header, an HMAC-SHA256 digest of "{t}.{rawBody}" keyed by your webhook_secret. Recompute it and reject stale timestamps to guard against replay.

Full endpoint list, methods, and params: `reference/endpoints.md`.

Calling the API

# Create a page monitor with a webhook:
scripts/crawlora.sh -X POST /monitors '{
  "url": "https://example.com/pricing",
  "cadence_minutes": 60,
  "name": "Pricing page",
  "notification": {"webhook_url": "https://example.com/webhooks/crawlora"}
}' | jq '.'

# Create a sitemap monitor (new/removed pages):
scripts/crawlora.sh -X POST /monitors '{
  "url": "https://example.com/sitemap.xml",
  "target_type": "sitemap",
  "cadence_minutes": 1440
}' | jq '.'

# List monitors:
scripts/crawlora.sh /monitors | jq '.'

# Check one monitor's recent runs:
scripts/crawlora.sh /monitors/mon_abc123/checks | jq '.'

# Pause a monitor:
scripts/crawlora.sh -X PATCH /monitors/mon_abc123 '{"enabled": false}' | jq '.'

Use scripts/crawlora.sh for all requests; it keeps the API key out of command-line arguments.

Endpoint reference

See `reference/endpoints.md` for every Monitors endpoint this skill uses (method, path, params, description).

Examples

• Competitor pricing watch: POST /monitors on a competitor's pricing

page with notification.events: ["change.detected"], then poll /monitors/{id}/checks (or just wait for the webhook) to see when it actually changes.

• New-page discovery: target_type: "sitemap" on a site's sitemap URL

to get notified when a new product/blog page goes live, without crawling the whole site yourself.

• "Is my monitor actually working?" — GET /monitors/{id}/checks shows

whether recent runs completed, found a change, and whether the webhook delivered — the fastest way to debug a monitor that "should have fired."

Notes & limits

• Credits / pay-on-success: management calls (create, list, get, update,

delete) are free; only checks are billed. Free tier 2,000 credits/mo. Key at https://crawlora.net.

• Security: key lives in CRAWLORA_API_KEY only — never hardcode,

query-param, or commit it. Always verify the webhook signature before trusting a delivery's payload.

• Public data only — a monitor watches a publicly-reachable URL; it

can't authenticate into a page behind a login.

• `cadence_minutes` range is 5 to 10080 (5 minutes to 7 days) — pick the

cadence that matches how often the target actually changes; a tighter cadence than needed just burns checks faster.

• Every request scopes to the caller's own API key — there's no

cross-account listing or lookup.