Firecrawl Docs
Firecrawl Docs: get a webhook or email the moment a monitored webpage changes.
Last verified:
What is Firecrawl Docs?
The Firecrawl Docs monitoring feature detects when content on a website changes and sends notifications via webhook or email. It runs recurring scrapes or crawls and compares each result against the last retained snapshot to identify changes. Users can track product pages, documentation, blogs, changelogs, competitor sites, or any page where changes matter.
Each check records page-level results as same, new, changed, removed, or error. The tool supports two notification types: webhooks (sent as each monitored page finishes or when a full check completes) and email summaries (sent when changes or errors happen). It offers scrape monitors for explicit URLs and crawl monitors that diff every page discovered by a crawl on each check.
Key features include customizable schedules (cron or natural language like 'every day at noon'), snapshot retention, and diff configuration. Firecrawl Docs covers setup, API reference, and examples for both monitor types.
Firecrawl Docs pricing
Pricing model: Freemium
Monitors don't introduce a separate per-monitor fee. Each check pays for the underlying scrape or crawl it performs. Scrape monitors cost 1 credit per URL per check. Crawl monitors cost 1 credit per discovered page per check. When meaningful-change judging is enabled, an additional 1 credit is charged per changed page that the judge validates. Format add-ons (JSON, PDF, question, enhanced mode, etc.) use the same pricing as standalone scrape. The estimatedCreditsPerMonth is returned on monitor creation as an upper-bound estimate when judging is enabled.
Firecrawl Docs pros
- Detects website content changes automatically with webhook or email notifications
- Supports both scrape monitors for specific URLs and crawl monitors for entire sites
- Flexible scheduling with cron expressions or natural language like 'every 30 minutes'
- AI-powered meaningful-change judging filters out irrelevant noise
- JSON-mode change tracking for structured fields like price, headline, in-stock flag
- Provides unified text diff with parseDiff-style AST for markdown changes
- Per-field JSON diff keyed by field path for structured data tracking
- Mixed mode delivers both structured per-field diff and raw markdown unified diff
- Email summaries include diffs when includeDiffs is enabled
- Retry logic and automatic rate limiting built into the infrastructure
- Retention period configurable from 30 days default up to 365 days
- Supports 1-50 targets per monitor for monitoring multiple pages or sites
- Returns estimatedCreditsPerMonth and nextRunAt on monitor creation
- CLI support for creating monitors via firecrawl monitor create command
- SDKs available for Python, Node.js, and cURL examples provided
- Page-level webhook events include isMeaningful and judgment data
- Check results API supports filtering by status: queued, running, completed, failed, partial, skipped_overlap
- Snapshot JSON included in response so consumers don't need to re-fetch underlying scrape
Firecrawl Docs cons
- Minimum scheduling interval is 15 minutes, no sub-15-minute monitoring
- Judge credits charged separately for each changed page that is validated
- Each check always charges for underlying scrapes or crawls regardless of changes
- JSON-mode monitors report same even if surrounding markdown changed but tracked fields did not
- New recipients require email confirmation before receiving notifications
- Maximum 25 explicit email recipients per monitor
- Credit estimation for judges is upper-bound since credits only charged for actually judged changed pages
- Crawl monitors may over-crawl if includePaths and limit not carefully configured
Frequently asked questions about Firecrawl Docs
What is Firecrawl Monitoring and what does it do?
Firecrawl Monitoring detects when content on a website changes and sends notifications via webhook or email. It runs recurring scrapes or crawls and compares each result against the last retained snapshot. You can use it to track product pages, docs, blogs, changelogs, competitor sites, or any page where changes matter. Each check records page-level results as same, new, changed, removed, or error.
How do I create a monitor?
You create a monitor using the create_monitor function in the SDK (Python or Node.js), the POST /v2/monitor API endpoint, or the CLI with firecrawl monitor create. You specify a name, schedule (cron or natural language), goal for AI judging, targets (scrape type with URLs or crawl type with URL and crawlOptions), and notification settings (webhook and/or email). The create call returns the monitor with its ID, normalized cron, nextRunAt, and estimatedCreditsPerMonth.
What scheduling options are available?
Schedules can be provided as cron expressions or natural language text. Supported natural language examples include: every 30 minutes, every 15 minutes starting at :07, hourly, every 2 hours, daily, daily at 9:00, daily at 9am, daily at 5:30 PM, and weekly. The minimum interval is 15 minutes. API responses always return the normalized cron expression. Timezone controls when phrases like daily at 9am run.
What is the difference between scrape and crawl monitor targets?
Scrape targets run one scrape per URL in the urls array for explicit URLs you specify. Crawl targets run a full crawl for the specified URL on each check, then diff all discovered pages. Scrape monitors are for monitoring specific URLs, while crawl monitors are for monitoring every page discovered by crawling a site section. Crawl targets use crawlOptions for behavior (limit, includePaths, maxDiscoveryDepth) and scrapeOptions for each page scrape.
How does meaningful-change judging work?
When you add a plain-language goal and enable judging (automatically enabled if goal is present and judgeEnabled is omitted), Firecrawl runs the judge on changed pages. The judge returns a judgment with meaningful (boolean), confidence (high/medium/low), reason (explanation), and meaningfulChanges (array of specific changes). Each check charges for the underlying scrape/crawl, and if judging is enabled, adds 1 credit for each changed page it validates. Checks with no changed pages do not use judge credits.
What webhook events are available for monitors?
Firecrawl sends two monitor webhook events: monitor.page and monitor.check.completed. monitor.page is sent as each monitored scrape finishes in the scrape worker and includes isMeaningful and judgment when meaningful-change judging ran. monitor.check.completed is sent after the full check is reconciled and includes check status and summary counts (totalPages, same, changed, new, removed, error). You can configure webhook URL, headers, metadata, and which events to receive.
How do email notifications work?
Email summaries are sent only when a check has changed, new, removed, or errored pages. You enable email with notification.email.enabled: true, specify recipients (up to 25), and optionally set includeDiffs: true to include diffs. When a monitor has a goal and judging enabled, email summaries prioritize meaningful changed pages. If every changed page is judged as noise and there are no new, removed, or errored pages, the email is suppressed. New recipients require email confirmation via a confirmation link unless they are already team members.
How do I retrieve check results?
Use GET /v2/monitor/{monitorId}/checks to list checks and GET /v2/monitor/{monitorId}/checks/{checkId} to inspect a specific check. The SDKs auto-paginate by default. You can filter checks by status: queued, running, completed, failed, partial, or skipped_overlap. You can filter pages by status: same, new, changed, removed, or error. The check detail includes estimatedCredits, actualCredits, summary counts, and a paginated pages array. Each changed page includes inline diff data; JSON-mode pages also include snapshot with current extraction.
What are the credit costs for monitoring?
Scrape monitors cost 1 credit per URL per check. Crawl monitors cost 1 credit per discovered page per check. When meaningful-change judging is enabled, an additional 1 credit is charged per changed page that the judge validates. Format add-ons like JSON, PDF, question, and enhanced mode use the same pricing as standalone scrape. Each check always charges for the underlying scrape or crawl. estimatedCreditsPerMonth returned on creation is an upper-bound estimate when judging is enabled since judge credits are only charged for changed pages actually judged.