# Scrape and store data from multiple website pages with MongoDB: n8n workflow

> Started by an event in a connected app, and works across MongoDB. 23 steps, 3 branches, 1 loop or waits: large, and worth reading before you trust it with...

Source: https://millionrounds.com/workflow/scrape-and-store-data-from-multiple-website-pages-with-mongo (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Data collection](https://millionrounds.com/workflows?category=Data+collection#library)

# Scrape and store data from multiple website pages with MongoDB

Started by an event in a connected app, and works across MongoDB. 23 steps, 3 branches, 1 loop or waits: large, and worth reading before you trust it with anything live.

Large

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fscrape-and-store-data-from-multiple-website-pages-with-mongo)

**23 steps** Large: read it before trusting it with anything live

**3** Branches: places the flow splits

**1** Loop or waits

**1** Service it connects to

## The flow

Starts it 1 Logic 9 Service 13

The steps in the order they run, from what starts it.

- 1 **On clicking 'execute'** Manual trigger *Starts it*
- 2 **MongoDB** Mongo Db *Service*
- 3 **Create Directory** Execute Command *Service*
- 4 **HTTP Request** Http Request *Service*
- 5 **HTML Extract** Html Extract *Service*
- 6 **Prepare countries** Function *Logic*
- 7 **SplitInBatches** Split In Batches *Logic*
- 8 **uProc** Uproc *Service*
- 9 **Set Page to Scrape** Function Item *Service*
- 10 **Set** Set *Logic*
- 11 **Generate filename** Function Item *Service*
- 12 **Read Binary File** Read Binary File *Service*
- 13 **File exists?** If *Logic*
- 14 **Read Binary File1** Read Binary File *Service*
- 15 **Wait** Function *Logic*
- 16 **HTML Extract1** Html Extract *Service*
- 17 **HTTP Request1** Http Request *Service*
- 18 **Prepare Documents** Function *Logic*
- 19 **Write Binary File** Write Binary File *Service*
- 20 **MongoDB1** Mongo Db *Service*
- 21 **Set More Pages** Function *Logic*
- 22 **More Pages** If *Logic*
- 23 **More Countries** If *Logic*

## How it's built

- **Starts on:** An event in a connected app
- **Steps:** 23, with 3 branches, 1 loop or waits
- **Size:** Large: read it before trusting it with anything live
- **Connects to:** MongoDB
- **Model:** None. This is plumbing, not judgement

## Questions

What does it need to run?

An n8n account, plus your own MongoDB account. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

No. Every step is fixed logic or a call to a service, so it does the same thing every time.

How long does it take to set up?

Take your time. At 23 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 23 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fscrape-and-store-data-from-multiple-website-pages-with-mongo)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Connects to

Each opens the other workflows that use it.

[MongoDB](https://millionrounds.com/workflows?service=MongoDB#library)

## Workflows like it

Also data collection, with the same services where possible, closest in size first.

- [**Migrate large Hugging Face datasets to MongoDB** 15 steps · MongoDB](https://millionrounds.com/workflow/migrate-large-hugging-face-datasets-to-mongodb)
- [**TikTok Shop data integration with MongoDB and Google Sheets** 11 steps · Google Sheets, MongoDB](https://millionrounds.com/workflow/tiktok-shop-data-integration-with-mongodb-and-google-sheets)
- [**Append MongoDB data to Google Sheets** 3 steps · Google Sheets, MongoDB](https://millionrounds.com/workflow/append-mongodb-data-to-google-sheets)
- [**Mongo2sheet with Google Sheets and MongoDB** 3 steps · Google Sheets, MongoDB](https://millionrounds.com/workflow/mongo2sheet-with-google-sheets-and-mongodb)
- [**Transfer users from MongoDB to Google Sheets** 3 steps · Google Sheets, MongoDB](https://millionrounds.com/workflow/transfer-users-from-mongodb-to-google-sheets)