# Filter URLs with Gemini and Groq: n8n workflow template

> Started by a clock, and works across Postgres. It hands the judgement calls to Gemini, Groq and one other.

Source: https://millionrounds.com/workflow/filter-urls-with-gemini-and-groq (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Data collection](https://millionrounds.com/workflows?category=Data+collection#library)

# Filter URLs with Gemini and Groq

Started by a clock, and works across Postgres. It hands the judgement calls to Gemini, Groq and one other. 37 steps, 4 branches: large, and worth reading before you trust it with anything live.

Large Calls a model: Gemini, Groq

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Ffilter-urls-with-gemini-and-groq)

**37 steps** Large: read it before trusting it with anything live

**4** Branches: places the flow splits

**0** Loops or waits

**1** Service it connects to

## The flow

Starts it 2 Model 5 Logic 16 Service 14

The steps in the order they run, from what starts it.

- 1 **Schedule Trigger** Schedule trigger *Starts it*
- 2 **Mistral Cloud Chat Model** Lm Chat Mistral Cloud *Model*
- 3 **Groq Chat Model** Lm Chat Groq *Model*
- 4 **Google Gemini Chat Model** Lm Chat Google Gemini *Model*
- 5 **Delete robots.txt Table** Postgres *Service*
- 6 **Start** Execute Workflow trigger *Starts it*
- 7 **Check all robots.txt Table** Postgres *Service*
- 8 **Check all forbidden urls Table** Postgres *Service*
- 9 **Delete forbidden urls Table** Postgres *Service*
- 10 **Manual Upsert robots.txt Table** Postgres *Service*
- 11 **Manual Upsert forbidden urls Table** Postgres *Service*
- 12 **Manual Create forbidden urls table** Postgres *Service*
- 13 **Manually Create robots.txt Table** Postgres *Service*
- 14 **Start (Tests)** Set *Logic*
- 15 **Model Selector** Model Selector *Model*
- 16 **Get Base URL** Code *Logic*
- 17 **Information Extractor** Information Extractor *Model*
- 18 **Create forbidden urls table** Postgres *Service*
- 19 **If Link Allowed 2** If *Logic*
- 20 **Assume False Output** Set *Logic*
- 21 **Check forbidden urls table** Postgres *Service*
- 22 **Prepare Output for Link Allowed** Set *Logic*
- 23 **Prepare Output for Link Disallowed** Set *Logic*
- 24 **If forbidden url Detected** If *Logic*
- 25 **Prepare Output** Set *Logic*
- 26 **Return Disallow** Set *Logic*
- 27 **Create robots.txt Table** Postgres *Service*
- 28 **Upsert robots.txt Table** Postgres *Service*
- 29 **Check robots.txt Table** Postgres *Service*
- 30 **Output** Set *Logic*
- 31 **If robots.txt Found and Updated** If *Logic*
- 32 **Prepare Robots.txt Check 2** Set *Logic*
- 33 **Prepare Robots.txt Check** Set *Logic*
- 34 **Check Robots.txt** Code *Logic*
- 35 **Get Robots.txt** Http Request *Service*
- 36 **If Link Allowed** If *Logic*
- 37 **Assume Disallow** Set *Logic*

## How it's built

- **Starts on:** A clock
- **Steps:** 37, with 4 branches
- **Size:** Large: read it before trusting it with anything live
- **Connects to:** Postgres
- **Model:** Gemini, Groq, Mistral

## Questions

What does it need to run?

An n8n account, plus your own Postgres account, and a key for Gemini and Groq. You connect your own accounts after importing it; none are included.

What starts it?

It starts on a clock.

Does it use AI?

Yes. It calls Gemini, Groq and Mistral. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Take your time. At 37 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 37 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Ffilter-urls-with-gemini-and-groq)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Connects to

Each opens the other workflows that use it.

[Postgres](https://millionrounds.com/workflows?service=Postgres#library)

## Workflows like it

Also data collection, with the same services where possible, closest in size first.

- [**Central AI agent of an IT company with Gemini and OpenRouter** 39 steps · calls a model · Google Drive, Pinecone](https://millionrounds.com/workflow/central-ai-agent-of-an-it-company-with-gemini-and-openrouter)
- [**From sitemap crawling to vector storage: Creating an efficient** 35 steps · calls a model · Postgres, Supabase](https://millionrounds.com/workflow/from-sitemap-crawling-to-vector-storage-creating-an-efficien)
- [**Find and score event and speaker opportunities with Gumloop** 35 steps · Gmail, Postgres](https://millionrounds.com/workflow/find-and-score-event-and-speaker-opportunities-with-gumloop)
- [**Create a self-updating RAG chatbot with Google Drive** 34 steps · calls a model · Google Drive, Postgres](https://millionrounds.com/workflow/create-a-self-updating-rag-chatbot-with-google-drive)
- [**AI chatbot call center: Taxi service with Grok and Postgres** 33 steps · calls a model · Postgres, Redis](https://millionrounds.com/workflow/ai-chatbot-call-center-taxi-service-with-grok-and-postgres)
- [**Build an omnichannel OTP verification flow with Postgres** 33 steps · Email, Postgres](https://millionrounds.com/workflow/build-an-omnichannel-otp-verification-flow-with-postgres)