Firecrawl Build Scrape

From the author

Integrate Firecrawl `/scrape` into product code for single-page extraction. Use when an app already has a URL and needs markdown, HTML, links, screenshots, metadata, or structured page output. Prefer this skill over broader crawl patterns when the feature is page-level.

— the description by firecrawl

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The firecrawl-build-scrape skill provides a standardized approach for integrating the Firecrawl /scrape endpoint into product code. It is designed for scenarios where an application already possesses a specific URL and requires structured content, such as markdown, HTML, links, or metadata, for downstream processing. The skill emphasizes efficient data retrieval by leveraging cached content while providing mechanisms to control freshness via maxAge settings. It serves as the primary primitive for page-level extraction tasks, offering guidance on when to escalate to more complex interaction patterns or search-based discovery.

What it does

Extracts content from a single, known URL into formats like markdown or HTML.
Supports optional retrieval of links, screenshots, and metadata.
Manages content freshness through configurable maxAge parameters.
Provides metadata regarding cache state and source status for verification.
Integrates with LLM and indexing pipelines for downstream data processing.
Offers specific implementation guidance for multiple programming languages.

When to use it

When your application already has a specific URL to process.
When you need to ingest content for summarization, enrichment, or monitoring.
When you require a default extraction primitive before considering interactive browser sessions.
When you need to balance performance with data freshness requirements.

Advantages

Reduces latency by utilizing cached content for repeat requests.
Provides structured output suitable for direct LLM consumption.
Offers granular control over cache reuse to meet specific staleness requirements.
Simplifies integration with consistent patterns across multiple programming languages.

Usage examples

“How do I scrape a documentation page for my RAG pipeline?”: The agent will guide you to use the /scrape endpoint with the 'onlyMainContent' option to filter out navigation noise, returning clean markdown for your RAG ingestion.
“I need to check if a product page is still live and get its current price.”: The agent will recommend using /scrape with 'maxAge: 0' to ensure a fresh fetch, then advise your code to interpret the returned content for liveness signals.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/firecrawl/firecrawl-build-scrape/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip firecrawl-build-scrape.zip -d .agents/skills

Documentation by firecrawl

Use this when the application already has the URL and needs content from one page.

Use This When

the feature starts from a known URL
you need page content for retrieval, summarization, enrichment, or monitoring
you want the default extraction primitive before considering /interact

Default Recommendations

Return markdown unless the feature truly needs another format.
Use onlyMainContent for article-like pages where nav and chrome add noise.
Add waits or other rendering options only when the page needs them.

Freshness and Liveness

Firecrawl reuses recently indexed content, which is what makes repeat reads of the same URL fast. Set maxAge (milliseconds) to bound how old a reused copy may be, or maxAge: 0 to skip index reuse for a freshness-critical read.
Read metadata.cacheState and metadata.cachedAt to see what you actually got.
A successful scrape reports what the page returned. Whether the thing the page describes is still active is a source-specific judgment your code makes.
See references/freshness-and-liveness.md for the tradeoff, the metadata, and the decision rule.

Common Product Patterns

knowledge ingestion from known URLs
enrichment from a company, product, or docs page
pricing, changelog, and documentation extraction
page-level quality checks or monitoring

Escalation Rules

If you do not have the URL yet, start with firecrawl-build-search.
If content requires clicks, typing, or multi-step navigation, escalate to firecrawl-build-interact.

Implementation Notes

Keep the integration narrow: one feature, one URL, one extraction contract.
Treat /scrape as the default primitive for downstream LLM or indexing pipelines.
Request richer formats only when the consumer needs them, such as links, screenshots, or branding data.

Docs (Source of Truth)

Read the source-of-truth page for your project language before writing integration code:

Node / TypeScript: docs.firecrawl.dev/agent-source-of-truth/node
Python: docs.firecrawl.dev/agent-source-of-truth/python
Rust: docs.firecrawl.dev/agent-source-of-truth/rust
Java: docs.firecrawl.dev/agent-source-of-truth/java
Elixir: docs.firecrawl.dev/agent-source-of-truth/elixir
cURL / REST: docs.firecrawl.dev/agent-source-of-truth/curl

Files

ATTRIBUTION.mdAdded by AutoGineer923 B
LICENSELicence file added by AutoGineer735 B
SKILL.md3.7 KiB
references/freshness-and-liveness.md2.3 KiB

SKILL.md