# How we keep our production error backlog tidy Every hour a Machine agent sorts new and spiking Sentry errors, checks GitHub for existing issues, drafts issues for the worst ones and posts to Slack. It never resolves, ignores or assigns. Canonical page: https://themachine.myapps.ai/use-cases/error-triage Sentry never misses an error. The hard part is turning that flood into tasks people will actually take on. Fresh error types land beside ones that have been climbing for days. Some already have a GitHub issue, some do not, and you cannot tell which without digging. By the time an engineer finds a free hour to sort through it, the pile is so long that sorting it is a project in itself. Most weeks that hour never comes, and the pile keeps growing. So we gave the job to an agent on Machine that tidies the backlog every hour. It reads Sentry as it stands right now, groups the new and spiking errors, sees which already have a GitHub issue, and drafts an issue with the stack trace and impact for the ones that do not. Then it posts a summary in Slack. This is housekeeping, not paging: it never wakes anyone up, and it never declares anything fixed. - **Team:** Machine - **Runs on:** A cron job, every hour - **Connected systems:** Sentry · GitHub · Slack - **Mode:** Drafts issues and posts alerts · never resolves or assigns ## The problem Errors pile up in Sentry faster than anyone gets round to them. A few are brand new. Others have ticked along unresolved for weeks at a slow, even pace. Others took off after this morning's deploy. Deciding which ones deserve an engineer, and whether someone already opened a GitHub issue for them, means reading the entire list by hand. That is precisely the chore that slides every time something more pressing turns up. The usual fixes shift the pile around instead of tidying it. Opening a GitHub issue for every Sentry error buries your tracker in duplicates and one-off blips nobody will touch. Treating Sentry itself as the record means the backlog is only as fresh as the last time someone looked at the dashboard. On-call tools solve the other half of the problem, the single loud alert happening now, not the quiet heap of errors that never paged anyone but still deserve a decision. ## What we built Once an hour, a cron job on Machine wakes the agent. It starts a new **session**, a fresh cloud computer (a sandbox) for that run alone, which can read Sentry but not change it. The agent collects errors that are new or that are firing well above their recent baseline, and groups them by fingerprint. Before doing anything with an error, it searches our GitHub repo for an issue that already covers it. For the biggest offenders with no issue yet, it drafts one with the stack trace and the impact: how many events, how many users, and when it first appeared. Then it posts a Slack summary of what is new, what is spiking and what it drafted. It leaves the state of every Sentry issue alone, and it never assigns anything to anyone. ## How it works ### Wake up every hour A **cron trigger** starts the agent once an hour, and each run gets a brand new **session** on its own sandbox. Every hour is its own run on a throwaway machine, so the backlog is read fresh from Sentry each time and nothing is carried over from the last run. ### Teach it how you sort errors Our sorting rules are kept as **skills** and **memory**, a playbook the agent carries and keeps improving. It spells out what counts as a real spike and what is just background noise, how to rank errors by impact, the issue template the stack trace and impact go into, and the labels and habits of our tracker. Tweak a threshold or the template in that file, and the next hourly run follows the new version. ### Connect Sentry to read and GitHub to draft A scoped **connector** lets the agent read Sentry. The key stays on our servers, so the model never sees a raw token. A second GitHub token, with very narrow permissions, lets it draft issues through the `gh` CLI. Together they let it: - **Read what is in Sentry**: new issues, plus issues firing well above their recent baseline, with the full stack trace, first-seen time, and event and user counts. - **Look for an existing GitHub issue**: before drafting, search for an issue that already tracks this error, using its Sentry issue ID. - **Draft a new GitHub issue**: only for the worst untracked offenders, with the stack trace and impact included. - **Post in Slack**: a summary of the run covering new errors, spikes, fresh drafts and errors that were already tracked. ### Draw the lines it cannot cross Resolving, ignoring or muting an error in Sentry is off limits, and so is assigning a GitHub issue to anyone. Those calls belong to your team. The agent writes exactly two things: the issue draft and the Slack message. Credentials are stored encrypted in the **Secrets Manager** and only handed to the agents you grant them to. ### Let the backlog tidy itself From then on, every hour the raw Sentry feed becomes a short, ranked list of errors that actually need a decision. You see the new ones and the genuine spikes, each checked against the tracker so nothing is filed twice, and the most serious already drafted as issues with the trace and impact attached. Your team reads the Slack summary and picks priorities. Nothing gets resolved, ignored or assigned unless they do it. > **The pattern** > An hourly **cron** opens a session with a read-only **connector** to Sentry and > a tightly scoped GitHub token for drafting. The sorting rules live in **skills** > and **memory**. The agent skips anything already tracked, drafts issues for > the worst errors and posts to Slack, and it never resolves, ignores or assigns > an error. ## Guardrails This agent reads live production errors and writes issues other people will act on, so its access is narrow and only flows one way: - **Isolation.** Each run happens in a private sandbox. The session may read Sentry and search GitHub, and the only things it sends out are the drafted issue and the Slack message. - **Scoped secrets.** The Sentry key is handled by the connector on our servers. The GitHub token is stored encrypted in the **Secrets Manager** and passed into the sandbox when the run starts. Both are only handed to the agents you grant them to. - **Never resolves, ignores, or mutes.** There is simply no way for the agent to change an error's status in Sentry. It reports on the backlog; it does not run it. - **Never assigns.** Drafted GitHub issues go out with no assignee. Your team decides priority and ownership. - **Everything is code.** The sorting rules, thresholds and per-system permissions are files in the repo, kept in version control. Changing them means a reviewed **change request**, not a dashboard setting. ## The outcome - **Every hour:** The backlog read fresh from Sentry as it is right now - **Deduped:** Each error checked against open GitHub issues first - **Draft-only:** Drafts and a Slack post, no resolving, ignoring or assigning Sorting errors used to need a spare hour nobody had. Now the backlog shows up already tidied, every hour: new errors and real spikes ranked by impact, checked against what is tracked, and the worst offenders drafted as issues with their trace attached. Your team spends its time choosing what to fix instead of working out what is broken.