Use Case
Transform web pages into AI-friendly markdown format:
- You need to process webpage content for LLM analysis
- You want to extract both content and links from web pages
- You need clean, formatted text without HTML markup
- You want to respect API rate limits while crawling pages
What this Workflow Does
The workflow uses Firecrawl.dev API to process webpages:
- Converts HTML content to markdown format
- Extracts all links from each webpage
- Handles API rate limiting automatically
- Processes URLs in batches from your database
Setup
- Create a Firecrawl.dev account and get your API key
- Add your Firecrawl API key to the HTTP Request node's Authorization header
- Connect your URL database to the input node (column name must be "Page") or edit the array in
Example fields from data source - Configure your preferred output database connection
How to Adjust it to Your Needs
- Modify input source to pull URLs from different databases
- Adjust rate limiting parameters if needed
- Customize output format for your specific use case
More templates and n8n workflows >>> @simonscrapes