Automation library · Data collection

Find and track flaky CI jobs in GitHub Actions with OpenAI

Started by a clock, and works across GitHub, Slack and n8n data table. It hands the judgement calls to OpenAI. 19 steps, 1 branch: a working afternoon to understand properly.

MediumCalls a model: OpenAI

19 stepsMedium: a working afternoon
1Branch: places the flow splits
0Loops or waits
3Services it connects to

The flow

Starts it 2Model 2Logic 9Service 6

The steps in the order they run, from what starts it.

  1. 1Look For Flakes Each MorningSchedule triggerStarts it
  2. 2Or Run It NowManual triggerStarts it
  3. 3Flake ReaderLm Chat Open AiModel
  4. 4SettingsSetLogic
  5. 5Group The FailuresChain LlmModel
  6. 6Ask For The Recent RunsHttp RequestService
  7. 7Build The ReportCodeLogic
  8. 8Pick The Runs Worth ReadingCodeLogic
  9. 9Post The ReportSlackService
  10. 10Read The Jobs In Each RunHttp RequestService
  11. 11Score Every JobCodeLogic
  12. 12Read What You ReportedData TableService
  13. 13Only The New Or WorseCodeLogic
  14. 14Anything Random?IfLogic
  15. 15One Row Per JobCodeLogic
  16. 16One Issue Per Confirmed FlakeCodeLogic
  17. 17Nothing Random TodayNo OpLogic
  18. 18Remember What You ReportedData TableService
  19. 19Open An IssueGithubService

How it's built

Starts on
A clock
Steps
19, with 1 branch
Size
Medium: a working afternoon
Connects to
GitHub, Slack, n8n data table
Model
OpenAI

Questions

What does it need to run?

An n8n account, plus your own GitHub and Slack accounts, and a key for OpenAI. You connect your own accounts after importing it; none are included.

What starts it?

It starts on a clock.

Does it use AI?

Yes. It calls OpenAI. Model calls are billed by the provider, not by n8n.

How long does it take to set up?

Plan an afternoon. 19 steps is enough that you should read each one before running it.

How do I import it?

Copy the workflow, open a new workflow in n8n and paste. The steps appear on the canvas, ready for your own accounts.

Before you run it

Nothing here has been tested by us. It is somebody's working automation, read and described from its own steps. Treat the connections it wants as a list of accounts it will touch, and read the 19 steps before you point it at anything that matters.