Automation library · Documents and files

Evaluate RAG response accuracy with OpenAI:

Started by someone opening a chat with it, and works across in-memory vectors. It puts OpenAI behind a retrieval step, so answers are drawn from documents you supply rather than from the model alone. 21 steps: large, and worth reading before you trust it with anything live.

LargeCalls a model: OpenAIRetrieval

Download free
21 stepsLarge: read it before trusting it with anything live
0Branches: places the flow splits
0Loops or waits
1Service it connects to

The flow

Starts it 3Model 11Logic 3Service 4

The steps in the order they run, from what starts it.

  1. 1When clicking ‘Execute workflow’Manual triggerStarts it
  2. 2Embeddings OpenAIEmbeddings Open AiModel
  3. 3Recursive Character Text SplitterText Splitter Recursive Character Text SplitterModel
  4. 4OpenAI Chat ModelLm Chat Open AiModel
  5. 5Embeddings OpenAI1Embeddings Open AiModel
  6. 6When chat message receivedChat triggerStarts it
  7. 7When fetching a dataset rowEvaluation triggerStarts it
  8. 8OpenAI Chat Model1Lm Chat Open AiModel
  9. 9Structured Output ParserOutput Parser StructuredModel
  10. 10Get DatasheetHttp RequestService
  11. 11Simple Vector StoreVector Store In MemoryModel
  12. 12Default Data LoaderDocument Default Data LoaderModel
  13. 13AI AgentAgentModel
  14. 14Simple Vector Store1Vector Store In MemoryModel
  15. 15Remap InputSetLogic
  16. 16Document GroundingChain LlmModel
  17. 17EvaluationEvaluationService
  18. 18Set OutputsEvaluationService
  19. 19Get DocumentsSetLogic
  20. 20No Operation, do nothingNo OpLogic
  21. 21Set MetricsEvaluationService

How it's built

Starts on
Someone opening a chat with it
Steps
21
Size
Large: read it before trusting it with anything live
Connects to
in-memory vectors
Model
OpenAI
Retrieval
Answers are drawn from documents you supply, not from the model alone

Questions

What does it need to run?

An n8n account, plus your own in-memory vectors account, and a key for OpenAI. You connect your own accounts after importing it; none are included.

What starts it?

It starts on someone opening a chat with it.

Does it use AI?

Yes. It calls OpenAI. Its answers draw on documents you give it, not on the model alone. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Take your time. At 21 steps it is large: read it before trusting it with anything live.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 21 steps before you point it at anything that matters.