Automation library · Alerts and monitoring

Monitor AI quality drift with OpenAI and Slack

Started by a clock, and works across Slack. It hands the judgement calls to OpenAI. 14 steps, 1 branch: a working afternoon to understand properly.

MediumCalls a model: OpenAI

14 stepsMedium: a working afternoon
1Branch: places the flow splits
0Loops or waits
1Service it connects to

The flow

Starts it 2Model 3Logic 5Service 4

The steps in the order they run, from what starts it.

  1. 1Daily ScheduleSchedule triggerStarts it
  2. 2When fetching a dataset rowEvaluation triggerStarts it
  3. 3OpenAI Chat ModelLm Chat Open AiModel
  4. 4AI AgentAgentModel
  5. 5Format eval inputCodeLogic
  6. 6Evaluating?EvaluationService
  7. 7Score ResponseOpen AiModel
  8. 8Production logicNo OpLogic
  9. 9Evaluation - Set OutputsEvaluationService
  10. 10Set MetricsEvaluationService
  11. 11Check ThresholdCodeLogic
  12. 12Below Threshold?IfLogic
  13. 13Slack AlertSlackService
  14. 14All ClearNo OpLogic

How it's built

Starts on
A clock
Steps
14, with 1 branch
Size
Medium: a working afternoon
Connects to
Slack
Model
OpenAI

Questions

What does it need to run?

An n8n account, plus your own Slack account, and a key for OpenAI. You connect your own accounts after importing it; none are included.

What starts it?

It starts on a clock.

Does it use AI?

Yes. It calls OpenAI. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Plan an afternoon. 14 steps is enough that you should read each one before running it.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 14 steps before you point it at anything that matters.