Automation library · Data collection

Autonomous AI crawler with OpenAI and Supabase

Started by an event in a connected app, and works across Supabase. It hands the judgement calls to OpenAI. 28 steps: large, and worth reading before you trust it with anything live.

LargeCalls a model: OpenAI

Download free
28 stepsLarge: read it before trusting it with anything live
0Branches: places the flow splits
0Loops or waits
1Service it connects to

The flow

Starts it 1Model 5Logic 18Service 4

The steps in the order they run, from what starts it.

  1. 1TextTool WorkflowModel
  2. 2URLsTool WorkflowModel
  3. 3OpenAI Chat ModelLm Chat Open AiModel
  4. 4JSON ParserOutput Parser StructuredModel
  5. 5Execute workflowManual triggerStarts it
  6. 6Set domain (text)SetLogic
  7. 7Set domain (URL)SetLogic
  8. 8Crawl websiteAgentModel
  9. 9Get companiesSupabaseService
  10. 10Add protocool to domain (text)SetLogic
  11. 11Add protocool to domain (URL)SetLogic
  12. 12Set social media arraySetLogic
  13. 13Select company name and websiteSetLogic
  14. 14Get website (text)Http RequestService
  15. 15Get website (URL)Http RequestService
  16. 16Merge all dataMergeLogic
  17. 17Map company name and websiteSetLogic
  18. 18Convert HTML to MarkdownMarkdownLogic
  19. 19Retrieve URLsHtmlLogic
  20. 20Insert new rowSupabaseService
  21. 21Set response (text)SetLogic
  22. 22Split out URLsSplit OutLogic
  23. 23Filter out empty hrefsFilterLogic
  24. 24Remove duplicatedRemove DuplicatesLogic
  25. 25Set domain to pathSetLogic
  26. 26Filter out invalid URLsFilterLogic
  27. 27Aggregate URLsAggregateLogic
  28. 28Set response (URL)SetLogic

How it's built

Starts on
An event in a connected app
Steps
28
Size
Large: read it before trusting it with anything live
Connects to
Supabase
Model
OpenAI

Questions

What does it need to run?

An n8n account, plus your own Supabase account, and a key for OpenAI. You connect your own accounts after importing it; none are included.

What starts it?

It starts on an event in a connected app.

Does it use AI?

Yes. It calls OpenAI. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Take your time. At 28 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 28 steps before you point it at anything that matters.