firecrawl/cli

firecrawl-crawl

Bulk-extract many pages from one site or section.

View source

How to use

  1. Copy the installation command and review the source package before installing it in your own agent environment.
  2. Check the source document for required tools, credentials and supported hosts.
  3. Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl

Before you start

  • Installation does not include a model subscription, API allowance or permission to access your files.
  • Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

firecrawl crawl

Bulk extract content from a website. Crawls pages following links up to a depth/limit.

Prerequisite: crawl requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

Quick start

bash
# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json

# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json

# Check status of a running crawl
firecrawl crawl <job-id>

Run firecrawl crawl --help for the full option list.

Done when: the crawl reaches a terminal status and the saved output under .firecrawl/ contains the expected pages.

Tips

  • Use --wait when you need the results immediately. It has no default timeout; use --timeout <seconds> to bound polling. Without --wait, crawl returns a job ID for async polling.
  • Scope crawls with `--include-paths` whenever the request names a section — crawl only the pages you need.
  • Crawl consumes credits per page. Check firecrawl credit-usage before large crawls (credit-usage requires authentication).

See also

Related by task

from this repository

More skills

All skills
firecrawl
Official

firecrawl

Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For US legal or regulatory questions use gov; for papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index.

Total installs
104.9K
GitHub stars
694
Updated
Nov 19
firecrawl
Official

firecrawl-search

Find web sources with query-relevant page excerpts and optional full-page content, and discover workflows, data APIs, and indexes. Use for web research or finding structured records, listings, transcripts, and datasets. Supports semantic tool discovery, domain matching, and progressive catalogue browsing.

Total installs
85.5K
GitHub stars
694
Updated
Nov 19
firecrawl
Official

firecrawl-scrape

Read a known webpage or execute a discovered workflow or data-provider capability. Use for page content or structured results once the URL or tool is selected.

Total installs
84.5K
GitHub stars
694
Updated
Nov 19
firecrawl
Official

firecrawl-agent

Autonomously navigate websites and extract structured data across pages. Use when the task requires navigation or no suitable ready-made workflow or data provider covers it.

Total installs
82.2K
GitHub stars
694
Updated
Nov 19