AutoGineer
Generated banner for Firecrawl Knowledge Ingest
Generated logo for Firecrawl Knowledge Ingest

Firecrawl Knowledge Ingest

FirecrawlOfficialCreated by firecrawl

  • Skill
  • Workflow
  • Automation and integrations
  • ISC
  • No issues found
Download ZIP

From the author

Export documentation portals, help centers and API reference sites into local markdown or JSON with Firecrawl browser, including pages that render with JavaScript, hide content behind collapsible navigation or paginate. Use when the user wants a docs site, help center or support knowledge base saved as files, even without an explicit request to scrape.
— the description by firecrawl

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Firecrawl Knowledge Ingest provides a systematic workflow for capturing content from complex documentation sites, help centers, and API references. It is designed to handle modern web challenges such as JavaScript-rendered content, paginated lists, and collapsible navigation menus. By leveraging browser-based navigation, the skill ensures that content is accurately extracted, cleaned of site-specific UI elements, and organized into a structured format suitable for knowledge management or local archiving.

What it does

  • Navigates documentation portals to identify sections, categories, and article links.
  • Handles JavaScript-heavy pages and dynamic content loading.
  • Extracts article content into clean markdown or JSON formats.
  • Captures metadata including titles, sections, and timestamps.
  • Manages pagination and complex navigation structures.
  • Produces a summary report detailing extracted content and any access limitations.

When to use it

  • When you need to archive a documentation site or help center for offline use.
  • When content is hidden behind JavaScript rendering or complex navigation.
  • When you need to convert a web-based knowledge base into structured data files.
  • When you require a systematic way to track and report on the progress of a web scraping task.

Advantages

  • Bypasses common scraping hurdles like dynamic JS rendering and pagination.
  • Produces clean, structured output ready for documentation or LLM ingestion.
  • Includes metadata extraction for better content organization.
  • Provides a clear audit trail of extracted pages and potential failures.

Usage examples

“Please ingest the documentation from this API reference site into markdown files.”
The agent will navigate the site, extract all API documentation pages, and provide a summary report with the resulting markdown files.
“I need to save this help center as a JSON file for my local search tool.”
The agent will crawl the help center, extract articles, and output a structured JSON file containing the content and metadata.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/firecrawl/firecrawl-knowledge-ingest/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip firecrawl-knowledge-ingest.zip -d .agents/skills

Documentation by firecrawl

Firecrawl Knowledge Ingest

Use this when a docs portal needs browser navigation, auth, pagination, or JS rendering.

Onboarding Interview

Infer the portal URL, output format, auth needs, and page limit from context. If the portal is clear, proceed immediately.

Ask at most 1-3 concise questions only if blocked, such as the portal URL, whether authentication is required, or the desired output format.

Firecrawl Collection Plan

Use Firecrawl browser to:

  • open the portal and inspect navigation

  • identify sections, categories, sidebar links, and article URLs

  • follow sidebar navigation, next links, pagination, load-more controls, or search

  • scrape article content as markdown

  • extract metadata such as title, section, last updated date, author, and tags

Try Firecrawl map as a supplement for public URLs, but use browser navigation for auth-gated or JS-heavy content.

Final Deliverable

# Knowledge Ingest: [Portal]

## Summary
[Pages extracted, sections covered, limitations]

## Output
[JSON/markdown/merged file path or content]

## Sections
[Section names and article counts]

## Failed Or Restricted Pages
[Any access/loading issues]

## Sources
[URLs extracted]

## Rerun Inputs
workflow: firecrawl-knowledge-ingest
url: [portal url]
format: [json/markdown/merged]
max_pages: [number]

JSON Shape

Use source, url, extractedAt, totalArticles, and sections[] with article title, url, section, content, and metadata.

Quality Bar

  • Preserve code examples, tables, and formatting.

  • Strip nav chrome, headers, and footers.

  • Track extraction progress and page failures.

  • Respect authentication boundaries.

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
firecrawl
Publisher
Firecrawl (Official)
Source
firecrawl/skills · skills/workflows/firecrawl-knowledge-ingest · commit 458d9d0
Licence
ISC License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by firecrawl.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download firecrawl-knowledge-ingest.zip