# Evaluate AI agent response correctness with OpenAI and RAGAS: n8n workflow

> Started by someone opening a chat with it. It calls services over plain HTTP rather than through connectors. It hands the judgement calls to OpenAI.

Source: https://millionrounds.com/workflow/evaluate-ai-agent-response-correctness-with-openai-and-ragas (Millionrounds; every figure on the page links to its filing or article)

[Automation library](https://millionrounds.com/workflows) · [AI assistants](https://millionrounds.com/workflows?category=AI+assistants#library)

# Evaluate AI agent response correctness with OpenAI and RAGAS

Started by someone opening a chat with it. It calls services over plain HTTP rather than through connectors. It hands the judgement calls to OpenAI. 24 steps: large, and worth reading before you trust it with anything live.

Large Calls a model: OpenAI

[Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fevaluate-ai-agent-response-correctness-with-openai-and-ragas)

**24 steps** Large: read it before trusting it with anything live

**0** Branches: places the flow splits

**0** Loops or waits

**0** Services by connector: it calls them over plain HTTP

## The flow

Starts it 2 Model 5 Logic 12 Service 5

The steps in the order they run, from what starts it.

- 1 **Examples1** Output Parser Structured *Model*
- 2 **OpenAI Chat Model** Lm Chat Open Ai *Model*
- 3 **OpenAI Chat Model1** Lm Chat Open Ai *Model*
- 4 **When fetching a dataset row** Evaluation trigger *Starts it*
- 5 **When chat message received** Chat trigger *Starts it*
- 6 **Correctness Classifier** Chain Llm *Model*
- 7 **AI Agent** Agent *Model*
- 8 **Remap Input** Set *Logic*
- 9 **Calculate F1 Score** Code *Logic*
- 10 **Evaluation** Evaluation *Service*
- 11 **Merge** Merge *Logic*
- 12 **Set Input Fields** Set *Logic*
- 13 **No Operation, do nothing** No Op *Logic*
- 14 **Correctness Score** Code *Logic*
- 15 **Get Embeddings1** Http Request *Service*
- 16 **GroundTruth to Items** Split Out *Logic*
- 17 **Update Outputs** Evaluation *Service*
- 18 **Remap Embeddings** Set *Logic*
- 19 **Get Embeddings** Http Request *Service*
- 20 **Update Metrics** Evaluation *Service*
- 21 **Create Embeddings Result** Merge *Logic*
- 22 **Remap Embeddings1** Set *Logic*
- 23 **Calculate Similarity Score** Code *Logic*
- 24 **Aggregate** Aggregate *Logic*

## How it's built

- **Starts on:** Someone opening a chat with it
- **Steps:** 24
- **Size:** Large: read it before trusting it with anything live
- **Connects to:** Nothing by connector; it calls services over plain HTTP
- **Model:** OpenAI

## Questions

What does it need to run?

An n8n account, and a key for OpenAI. You connect your own accounts after importing it; none are included.

What starts it?

It starts on someone opening a chat with it.

Does it use AI?

Yes. It calls OpenAI. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Take your time. At 24 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

**Before you run it**

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 24 steps before you point it at anything that matters.

## Run it in n8n

- **Create your n8n account** Free to start. n8n is where the workflow runs. [Create an n8n account ↗](https://millionrounds.com/n8n)
- **Download the workflow** Free with a Millionrounds account. [Download free](https://millionrounds.com/login?next=%2Fapp%2Fworkflow%2Fevaluate-ai-agent-response-correctness-with-openai-and-ragas)
- **Paste it into n8n** Open a new workflow in n8n and paste. Then connect your own accounts.

Stored with the author's traces removed: canvas notes, instance identifiers, saved credential names and any live document they had left behind.

## Workflows like it

Also ai assistants, with the same services where possible, closest in size first.

- [**AI-powered multi-channel customer journey with SMS and OpenAI** 24 steps · calls a model · Gmail, Twilio](https://millionrounds.com/workflow/ai-powered-multi-channel-customer-journey-with-sms-and-opena)
- [**AI-powered system for Creating & Executing your own Claude Skills** 24 steps · calls a model · GitHub](https://millionrounds.com/workflow/ai-powered-system-for-creating-and-executing-your-own-claude)
- [**Analyze and modernize university curricula with OpenAI** 24 steps · calls a model · in-memory vectors, n8n data table](https://millionrounds.com/workflow/analyze-and-modernize-university-curricula-with-openai)
- [**Automate multilingual Slack communication (JA EN) with Gemini** 24 steps · calls a model](https://millionrounds.com/workflow/automate-multilingual-slack-communication-ja-en-with-gemini)
- [**Build a RAG system with Qdrant and Gemini** 24 steps · calls a model · Google Drive, Qdrant](https://millionrounds.com/workflow/build-a-rag-system-with-qdrant-and-gemini)
- [**Build and update RAG system with Google Drive and Qdrant** 24 steps · calls a model · Google Drive, Qdrant](https://millionrounds.com/workflow/build-and-update-rag-system-with-google-drive-and-qdrant)