# Industry sources & documentation scraper | 16 pre-configured: n8n workflow

> Started by a clock. It calls services over plain HTTP rather than through connectors. 16 steps, 3 branches, 1 loop or waits: a working afternoon to...

Source: https://millionrounds.com/workflow/industry-sources-and-documentation-scraper-16-pre-configured (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Research and reporting](https://millionrounds.com/workflows?category=Research+and+reporting#library)

# Industry sources & documentation scraper | 16 pre-configured

Started by a clock. It calls services over plain HTTP rather than through connectors. 16 steps, 3 branches, 1 loop or waits: a working afternoon to understand properly.

Medium

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Findustry-sources-and-documentation-scraper-16-pre-configured)

**16 steps** Medium: a working afternoon

**3** Branches: places the flow splits

**1** Loop or waits

**0** Services by connector: it calls them over plain HTTP

## The flow

Starts it 1 Logic 8 Service 7

The steps in the order they run, from what starts it.

- 1 **Schedule Daily at 3 AM** Schedule trigger *Starts it*
- 2 **Get Industry Sources** Function *Logic*
- 3 **Fetch Source Page** Http Request *Service*
- 4 **Extract Page Links** Function *Logic*
- 5 **Fetch Article Content** Http Request *Service*
- 6 **Parse Article Content** Function *Logic*
- 7 **Rate Limit Delay** Wait *Logic*
- 8 **Is Relevant?** If *Logic*
- 9 **Check if Technical** Function *Logic*
- 10 **Store in Knowledge Base** Http Request *Service*
- 11 **Analyze Content Opportunities** Http Request *Service*
- 12 **Is Technical Content?** If *Logic*
- 13 **Has Marketing Opportunity?** If *Logic*
- 14 **Extract Technical Codes** Http Request *Service*
- 15 **Create Response Campaign** Http Request *Service*
- 16 **Store Technical Codes** Http Request *Service*

## How it's built

- **Starts on:** A clock
- **Steps:** 16, with 3 branches, 1 loop or waits
- **Size:** Medium: a working afternoon
- **Connects to:** Nothing by connector; it calls services over plain HTTP
- **Model:** None. This is plumbing, not judgement

## Questions

What does it need to run?

An n8n account. You connect your own accounts after importing it; none are included.

What starts it?

It starts on a clock.

Does it use AI?

No. Every step is fixed logic or a call to a service, so it does the same thing every time.

How long does it take to set up?

Plan an afternoon. 16 steps is enough that you should read each one before running it.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 16 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Findustry-sources-and-documentation-scraper-16-pre-configured)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Workflows like it

Also research and reporting, with the same services where possible, closest in size first.

- [**AI-Powered CSAT Survey Analysis with Sentiment Scoring** 16 steps · Supabase](https://millionrounds.com/workflow/ai-powered-csat-survey-analysis-with-sentiment-scoring)
- [**Audit Android feature flags from GitHub with Firebase** 16 steps · Slack](https://millionrounds.com/workflow/audit-android-feature-flags-from-github-with-firebase)
- [**Backup Clockify to GitHub based on monthly reports** 16 steps · GitHub](https://millionrounds.com/workflow/backup-clockify-to-github-based-on-monthly-reports)
- [**Build a citation-ready web research pack with SearchCans** 16 steps](https://millionrounds.com/workflow/build-a-citation-ready-web-research-pack-with-searchcans)
- [**Clone & customize competitor Facebook ads with Gemini** 16 steps · Apify, Google Drive](https://millionrounds.com/workflow/clone-and-customize-competitor-facebook-ads-with-gemini)
- [**Create a balanced RSS topic digest with HTML** 16 steps · RSS](https://millionrounds.com/workflow/create-a-balanced-rss-topic-digest-with-html)