Automation library · Data collection

Scrape and store data from multiple website pages with MongoDB

Started by an event in a connected app, and works across MongoDB. 23 steps, 3 branches, 1 loop or waits: large, and worth reading before you trust it with anything live.

Large

Download free
23 stepsLarge: read it before trusting it with anything live
3Branches: places the flow splits
1Loop or waits
1Service it connects to

The flow

Starts it 1Logic 9Service 13

The steps in the order they run, from what starts it.

  1. 1On clicking 'execute'Manual triggerStarts it
  2. 2MongoDBMongo DbService
  3. 3Create DirectoryExecute CommandService
  4. 4HTTP RequestHttp RequestService
  5. 5HTML ExtractHtml ExtractService
  6. 6Prepare countriesFunctionLogic
  7. 7SplitInBatchesSplit In BatchesLogic
  8. 8uProcUprocService
  9. 9Set Page to ScrapeFunction ItemService
  10. 10SetSetLogic
  11. 11Generate filenameFunction ItemService
  12. 12Read Binary FileRead Binary FileService
  13. 13File exists?IfLogic
  14. 14Read Binary File1Read Binary FileService
  15. 15WaitFunctionLogic
  16. 16HTML Extract1Html ExtractService
  17. 17HTTP Request1Http RequestService
  18. 18Prepare DocumentsFunctionLogic
  19. 19Write Binary FileWrite Binary FileService
  20. 20MongoDB1Mongo DbService
  21. 21Set More PagesFunctionLogic
  22. 22More PagesIfLogic
  23. 23More CountriesIfLogic

How it's built

Starts on
An event in a connected app
Steps
23, with 3 branches, 1 loop or waits
Size
Large: read it before trusting it with anything live
Connects to
MongoDB
Model
None. This is plumbing, not judgement

Questions

What does it need to run?

An n8n account, plus your own MongoDB account. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

No. Every step is fixed logic or a call to a service, so it does the same thing every time.

How long does it take to set up?

Take your time. At 23 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 23 steps before you point it at anything that matters.