Automation library · Data collection

Web crawler: Convert websites to AI-ready markdown

Started by an event in a connected app, and works across Google Sheets. 16 steps, 1 branch: a working afternoon to understand properly.

Medium

16 stepsMedium: a working afternoon
1Branch: places the flow splits
0Loops or waits
1Service it connects to

The flow

Starts it 1Logic 10Service 5

The steps in the order they run, from what starts it.

  1. 1Manual TriggerManual triggerStarts it
  2. 2Set WebsiteSetLogic
  3. 3Scrape HomepageHttp RequestService
  4. 4Extract Links from HTMLHtmlLogic
  5. 5Split LinksSplit OutLogic
  6. 6Remove Duplicate LinksRemove DuplicatesLogic
  7. 7Filter Real HyperlinksFilterLogic
  8. 8Separate Images and LinksSwitchLogic
  9. 9Aggregate ImagesAggregateLogic
  10. 10Aggregate LinksAggregateLogic
  11. 11Scrape Content LinksHttp RequestService
  12. 12Add Images to SheetGoogle SheetsService
  13. 13Add Links to SheetGoogle SheetsService
  14. 14Convert to MarkdownMarkdownLogic
  15. 15Aggregate Scraped ContentAggregateLogic
  16. 16Add Scraped Content to SheetGoogle SheetsService

How it's built

Starts on
An event in a connected app
Steps
16, with 1 branch
Size
Medium: a working afternoon
Connects to
Google Sheets
Model
None. This is plumbing, not judgement

Questions

What does it need to run?

An n8n account, plus your own Google Sheets account. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

No. Every step is fixed logic or a call to a service, so it does the same thing every time.

How long does it take to set up?

Plan an afternoon. 16 steps is enough that you should read each one before running it.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 16 steps before you point it at anything that matters.