# Crawl documentation sites and build an AI knowledge base: n8n workflow

> Started by an event in a connected app, and works across Google Docs and Google Drive. It hands the judgement calls to Gemini.

Source: https://millionrounds.com/workflow/crawl-documentation-sites-and-build-an-ai-knowledge-base (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Documents and files](https://millionrounds.com/workflows?category=Documents+and+files#library)

# Crawl documentation sites and build an AI knowledge base

Started by an event in a connected app, and works across Google Docs and Google Drive. It hands the judgement calls to Gemini. 26 steps, 2 branches, 2 loops or waits: large, and worth reading before you trust it with anything live.

Large Calls a model: Gemini

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fcrawl-documentation-sites-and-build-an-ai-knowledge-base)

**26 steps** Large: read it before trusting it with anything live

**2** Branches: places the flow splits

**2** Loops or waits

**2** Services it connects to

## The flow

Starts it 1 Model 4 Logic 14 Service 7

The steps in the order they run, from what starts it.

- 1 **When clicking ‘Execute workflow’** Manual trigger *Starts it*
- 2 **Google Gemini Chat Model2** Lm Chat Google Gemini *Model*
- 3 **Google Gemini Chat Model** Lm Chat Google Gemini *Model*
- 4 **Create a map** Olostep Scrape *Service*
- 5 **Information Extractor** Information Extractor *Model*
- 6 **AI Agent** Agent *Model*
- 7 **Service Name** Set *Logic*
- 8 **Split Out1** Split Out *Logic*
- 9 **Merge1** Merge *Logic*
- 10 **Create a document** Google Docs *Service*
- 11 **Create Parent Folder** Google Drive *Service*
- 12 **Merge** Merge *Logic*
- 13 **Wait** Wait *Logic*
- 14 **Update a document** Google Docs *Service*
- 15 **Edit Fields** Set *Logic*
- 16 **Loop Over Items** Split In Batches *Logic*
- 17 **No Operation, do nothing** No Op *Logic*
- 18 **counter** Set *Logic*
- 19 **If1** If *Logic*
- 20 **Search files and folders** Google Drive *Service*
- 21 **set data** Set *Logic*
- 22 **If** If *Logic*
- 23 **Scrape a URL** Olostep Scrape *Service*
- 24 **counter++1** Set *Logic*
- 25 **Create folder** Google Drive *Service*
- 26 **counter++** Set *Logic*

## How it's built

- **Starts on:** An event in a connected app
- **Steps:** 26, with 2 branches, 2 loops or waits
- **Size:** Large: read it before trusting it with anything live
- **Connects to:** Google Docs, Google Drive
- **Model:** Gemini

## Questions

What does it need to run?

An n8n account, plus your own Google Docs and Google Drive accounts, and a key for Gemini. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

Yes. It calls Gemini. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Take your time. At 26 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 26 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fcrawl-documentation-sites-and-build-an-ai-knowledge-base)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Connects to

Each opens the other workflows that use it.

[Google Docs](https://millionrounds.com/workflows?service=Google+Docs#library)[Google Drive](https://millionrounds.com/workflows?service=Google+Drive#library)

## Workflows like it

Also documents and files, with the same services where possible, closest in size first.

- [**Chat with Ollama and Supabase Vector DB** 26 steps · calls a model · Google Drive, Supabase](https://millionrounds.com/workflow/chat-with-ollama-and-supabase-vector-db)
- [**Index Google Drive files into a Supabase vector store** 26 steps · calls a model · Google Drive, Postgres](https://millionrounds.com/workflow/index-google-drive-files-into-a-supabase-vector-store)
- [**Archive HR and legal documents with UploadToURL** 26 steps · Airtable, Gmail](https://millionrounds.com/workflow/archive-hr-and-legal-documents-with-uploadtourl)
- [**AI-Powered RAG Document Processing & Chatbot with Google Drive** 27 steps · calls a model · Google Drive, Supabase vectors](https://millionrounds.com/workflow/ai-powered-rag-document-processing-and-chatbot-with-google-d)
- [**Document RAG & chat agent: Google Drive to Qdrant with Mistral** 25 steps · calls a model · Google Drive, Qdrant](https://millionrounds.com/workflow/document-rag-and-chat-agent-google-drive-to-qdrant-with-mist)
- [**Review contract risk and rewrite clauses with Groq** 25 steps · calls a model · Google Drive, Notion](https://millionrounds.com/workflow/review-contract-risk-and-rewrite-clauses-with-groq)