# Evaluate AI prompts with OpenRouter LLM-as-judge and Notion: n8n workflow

> Started by someone submitting a form, and works across Notion. It hands the judgement calls to OpenRouter.

Source: https://millionrounds.com/workflow/evaluate-ai-prompts-with-openrouter-llm-as-judge-and-notion (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [Research and reporting](https://millionrounds.com/workflows?category=Research+and+reporting#library)

# Evaluate AI prompts with OpenRouter LLM-as-judge and Notion

Started by someone submitting a form, and works across Notion. It hands the judgement calls to OpenRouter. 17 steps, 1 loop or waits: a working afternoon to understand properly.

Medium Calls a model: OpenRouter

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fevaluate-ai-prompts-with-openrouter-llm-as-judge-and-notion)

**17 steps** Medium: a working afternoon

**0** Branches: places the flow splits

**1** Loop or waits

**1** Service it connects to

## The flow

Starts it 1 Model 8 Logic 6 Service 2

The steps in the order they run, from what starts it.

- 1 **When Form Submitted** Form trigger *Starts it*
- 2 **OpenAI Dataset Generator** Lm Chat Open Router *Model*
- 3 **OpenAI Answer Agent** Lm Chat Open Router *Model*
- 4 **OpenAI Grader Agent** Lm Chat Open Router *Model*
- 5 **OpenAI Synthesis Agent** Lm Chat Open Router *Model*
- 6 **Set Notion IDs** Set *Logic*
- 7 **Dataset Generator** Agent *Model*
- 8 **Answer Agent** Agent *Model*
- 9 **Grader Agent** Agent *Model*
- 10 **Synthesis Agent** Agent *Model*
- 11 **Flatten Dataset Code** Code *Logic*
- 12 **Extract Grader Results** Code *Logic*
- 13 **Upload Report to Notion** Notion *Service*
- 14 **Batch Process Items** Split In Batches *Logic*
- 15 **Collate Evaluation Results** Code *Logic*
- 16 **Upload Item to Notion** Notion *Service*
- 17 **Prepare Evaluation Item** Code *Logic*

## How it's built

- **Starts on:** Someone submitting a form
- **Steps:** 17, 1 loop or waits
- **Size:** Medium: a working afternoon
- **Connects to:** Notion
- **Model:** OpenRouter

## Questions

What does it need to run?

An n8n account, plus your own Notion account, and a key for OpenRouter. You connect your own accounts after importing it; none are included.

What starts it?

It starts on someone submitting a form.

Does it use AI?

Yes. It calls OpenRouter. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Plan an afternoon. 17 steps is enough that you should read each one before running it.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 17 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fevaluate-ai-prompts-with-openrouter-llm-as-judge-and-notion)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Connects to

Each opens the other workflows that use it.

[Notion](https://millionrounds.com/workflows?service=Notion#library)

## Workflows like it

Also research and reporting, with the same services where possible, closest in size first.

- [**AI-powered customer feedback analysis & routing** 18 steps · calls a model · Gmail, Notion](https://millionrounds.com/workflow/ai-powered-customer-feedback-analysis-and-routing)
- [**Analyze competitor strategy from YouTube & RSS with Gemini** 18 steps · calls a model · Apify, Notion](https://millionrounds.com/workflow/analyze-competitor-strategy-from-youtube-and-rss-with-gemini)
- [**Autonomous Expert Intelligence Layer: Strategic Insight Engine** 16 steps · calls a model · Notion, Slack](https://millionrounds.com/workflow/autonomous-expert-intelligence-layer-strategic-insight-engin)
- [**X & Reddit tech trend analysis with Gemini and Telegram** 16 steps · calls a model · Notion, Telegram](https://millionrounds.com/workflow/x-and-reddit-tech-trend-analysis-with-gemini-and-telegram)
- [**Analyze & Tag User Feedback in Notion with OpenAI** 19 steps · calls a model · Notion](https://millionrounds.com/workflow/analyze-and-tag-user-feedback-in-notion-with-openai)
- [**CallForge - 02 - prep Gong calls with Google Sheets and Notion** 20 steps · Google Sheets, Notion](https://millionrounds.com/workflow/callforge-02-prep-gong-calls-with-google-sheets-and-notion)