# Detect semantic duplicate website pages with Google Drive: n8n workflow

> Started by an event in a connected app, and works across Google Drive and Postgres. 29 steps, 2 loops or waits: large, and worth reading before you trust it...

Source: https://millionrounds.com/workflow/detect-semantic-duplicate-website-pages-with-google-drive (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Content](https://millionrounds.com/workflows?category=Content#library)

# Detect semantic duplicate website pages with Google Drive

Started by an event in a connected app, and works across Google Drive and Postgres. 29 steps, 2 loops or waits: large, and worth reading before you trust it with anything live.

Large Retrieval

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fdetect-semantic-duplicate-website-pages-with-google-drive)

**29 steps** Large: read it before trusting it with anything live

**0** Branches: places the flow splits

**2** Loops or waits

**2** Services it connects to

## The flow

Starts it 1 Model 4 Logic 7 Service 17

The steps in the order they run, from what starts it.

- 1 **Generate Local Embeddings** Embeddings Ollama *Model*
- 2 **Chunk Text Recursively** Text Splitter Recursive Character Text Splitter *Model*
- 3 **Start Duplicate Check** Manual trigger *Starts it*
- 4 **Save Document Embeddings** Vector Store PGVector *Model*
- 5 **Context Injector** Document Default Data Loader *Model*
- 6 **Clear Vector Table** Postgres *Service*
- 7 **Dedup Processed Items** Remove Duplicates *Logic*
- 8 **Clear Scraped Pages Table** Postgres *Service*
- 9 **Batches for Embedding** Split In Batches *Logic*
- 10 **Scan Source Directory** Google Drive *Service*
- 11 **Create HNSW Index** Postgres *Service*
- 12 **Get Unprocessed Scraped Text** Postgres *Service*
- 13 **Isolate Target Documents** Filter *Logic*
- 14 **Compute Chunk Similarities** Postgres *Service*
- 15 **Loop Source File** Split In Batches *Logic*
- 16 **Generate Pairwise Report** Postgres *Service*
- 17 **GDrive Download Document** Google Drive *Service*
- 18 **Fetch Chunk Similarity Data** Postgres *Service*
- 19 **Extract Raw Text Content** Html *Logic*
- 20 **Export Chunk Similarity Report** Convert To File *Service*
- 21 **Format Database Payload** Set *Logic*
- 22 **Calculate Page Centroids** Postgres *Service*
- 23 **Save Scraped Page Text** Postgres *Service*
- 24 **Compute Centroid Distances** Postgres *Service*
- 25 **Code in JavaScript** Code *Logic*
- 26 **Aggregate Page-Level Metrics** Postgres *Service*
- 27 **Finalize Duplicate Flags** Postgres *Service*
- 28 **Fetch Page-Level Report Data** Postgres *Service*
- 29 **Export Page-Level Report** Convert To File *Service*

## How it's built

- **Starts on:** An event in a connected app
- **Steps:** 29, 2 loops or waits
- **Size:** Large: read it before trusting it with anything live
- **Connects to:** Google Drive, Postgres
- **Model:** None. This is plumbing, not judgement
- **Retrieval:** Answers are drawn from documents you supply, not from the model alone

## Questions

What does it need to run?

An n8n account, plus your own Google Drive and Postgres accounts. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

No. Every step is fixed logic or a call to a service, so it does the same thing every time.

How long does it take to set up?

Take your time. At 29 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 29 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fdetect-semantic-duplicate-website-pages-with-google-drive)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Connects to

Each opens the other workflows that use it.

[Google Drive](https://millionrounds.com/workflows?service=Google+Drive#library)[Postgres](https://millionrounds.com/workflows?service=Postgres#library)

## Workflows like it

Also content, with the same services where possible, closest in size first.

- [**Create a RAG Telegram bot with Google Drive and PostgreSQL** 29 steps · Google Drive, Postgres](https://millionrounds.com/workflow/create-a-rag-telegram-bot-with-google-drive-and-postgresql)
- [**Post to Bluesky social with Gmail and Google Drive** 29 steps · Gmail, Google Drive](https://millionrounds.com/workflow/postar-no-bluesky-social-with-gmail-and-google-drive)
- [**Split shared rental utility bills with Postgres and Gmail** 29 steps · Discord, Gmail](https://millionrounds.com/workflow/split-shared-rental-utility-bills-with-postgres-and-gmail)
- [**Automate LinkedIn content creation with OpenAI and Notion** 29 steps · calls a model · Gmail, Google Drive](https://millionrounds.com/workflow/automate-linkedin-content-creation-with-openai-and-notion)
- [**Create faceless videos with Gemini and ElevenLabs** 29 steps · calls a model · Google Drive](https://millionrounds.com/workflow/create-faceless-videos-with-gemini-and-elevenlabs)
- [**GPT-4o, RunwayML, ElevenLabs for Social Media with Gemini** 29 steps · calls a model · Google Drive, Google Sheets](https://millionrounds.com/workflow/gpt-4o-runwayml-elevenlabs-for-social-media-with-gemini)