# How we catch cloud cost spikes long before the bill arrives Our cost agent reads AWS Cost Explorer each day, learns normal spend for every service and account, flags anything that breaks pattern with its likely cause, and alerts Slack. It never touches a resource or budget. Canonical page: https://themachine.myapps.ai/use-cases/cloud-cost-anomaly Cloud bills shock people because nobody watches the meter while it runs. Cost Explorer will tell you the truth sooner or later, but out of the box a spike shows up at month end, on the invoice, weeks after whatever caused it started running. Someone tries a new instance type and leaves it on overnight. A service quietly scales past its usual ceiling. Something lands in a region nobody meant to use. Each is a tiny choice that later becomes a line item nobody can explain. So we gave the job to an agent on Machine. It checks AWS spend every day in one long-running session that remembers what "normal" looks like for each service and each account. It only reads billing data. Its single output is a Slack alert showing how far spend moved and its best guess at why. That's how we catch a cost problem when it's a day old instead of an invoice old. - **Team:** Machine - **Runs on:** A daily timer that reuses one ongoing session - **Connected systems:** AWS Cost Explorer · Slack - **Mode:** Reads only · alerts only, never changes a resource or a budget ## The problem Spend is easy to look up and hard to keep an eye on. Ask Cost Explorer what you spent over any period and it will answer. What it won't do is speak up on its own to say one service cost 40% more yesterday than it usually does. Nobody opens the console each morning to scan forty line items across a dozen accounts, so a spike keeps running up the bill until someone spots the invoice. The standard fixes leave a gap. A monthly budget alert only fires once the total for the month crosses a line, and by then you've overspent many times over. A single fixed threshold per service treats one that usually costs $50/day the same as one that usually costs $5,000/day, so it ends up either shouting all the time or missing things. And none of these tell you *why*. A dashboard shows that the number changed, not what changed it. ## What we built A daily timer on Machine wakes the same long-lived agent **session** each time, with read-only access to AWS Cost Explorer. It pulls yesterday's spend, split by service and by linked account, and updates its running baseline for each pair. Then it flags anything that leaves its usual range. That isn't one fixed threshold. It's a departure from what this exact service in this exact account normally costs. For every anomaly it works out the likely cause (a resource that just launched, a surge in traffic, spend in a region that had none before) and posts a single alert to Slack with the change and the suspected cause. AWS gets nothing written back: no resource is touched and no budget is changed. ## How it works ### Reopen the same session every day A **cron trigger** (a timer) wakes the agent daily. Instead of starting a new sandbox each time, it's **one session re-prompted daily** (`session_mode: reuse`). The agent picks up its own memory of what every service and account usually spends, rather than starting cold each morning. The longer it runs, the sharper that baseline gets. ### Teach it what an anomaly looks like The rules sit in **skills** and **memory**, the playbook the agent keeps and improves. They define what counts as breaking from the baseline, how much history sets a service's "normal", and how to read the shape of a spike: sudden or gradual, one account or many. When a spike turns out to be a planned load test or a known seasonal bump, we write that down and the agent stops raising it. ### Give it read-only access to Cost Explorer The agent uses a **connector**: a scoped connection to AWS, with the key held on the server so the model never sees a raw credential. Through it the agent reads: - **Daily cost and usage per service and linked account**: the raw figures that the baseline and the anomaly check are built on. - **Resource-level detail for the flagged window**: what actually launched, scaled up or moved region, to back up the guess at the cause. - **A Slack post**: the change, the suspected cause and nothing more. ### Explain the spike, not just its size Knowing a number went up doesn't tell you what to fix. For each anomaly, the agent looks at what shifted underneath: a resource that came online in that window, usage figures that fit a traffic surge, or spend showing up in a region that used to have none. It names its best guess at the cause next to the change, so whoever reads the alert can act on it straight away. ### Keep it watching, not touching The agent is **read-only** across AWS billing and Cost Explorer. It has no permission to change or delete a resource, or to alter a budget or any spending control. Watching and alerting is all it can do. Credentials are stored encrypted in the **Secrets Manager** and only handed to the agents you grant them to. ### Get the alert with the change and its likely cause Each day brings no more than one Slack alert per anomaly: which service and account, how far spend strayed from its baseline, and the suspected cause. If nothing is unusual, nothing is posted. Your engineering team reads the alert and decides whether to act. Nothing changes in AWS by itself. > **The pattern** > A daily **cron** reopens one long-lived **session** with a read-only > **connector** into AWS Cost Explorer. The baseline is kept in **skills** and > **memory** and carries over from one run to the next. The agent reads > everything, and the Slack alert is the only thing it writes. ## Guardrails This agent can see billing for every linked AWS account, so its access is narrow and only ever flows one way: - **Isolation.** Each run happens in its own sandbox. The session can reach only AWS Cost Explorer, and the Slack alert is the only thing that leaves it. - **Scoped secrets.** The AWS credential is stored encrypted in the Secrets Manager, delivered into the sandbox when the run starts, and only handed to the agents you grant them to. - **Read-only, no exceptions.** The AWS connector can only read. The agent can't launch, change or delete a resource, and it can't alter a budget or any spending control. Alerting is its whole job. - **Alert only.** A Slack post is everything it produces. There's no automatic fix, no change to auto-scaling and no resource shut down. A person decides what, if anything, to do about a spike. - **Everything is code.** The baseline logic, the thresholds and the permissions for each system are files in your repo. They're versioned, and you change them through a reviewed **change request**, not a dashboard toggle. ## The outcome - **Every day:** Each service and account measured against its own track record - **Read-only:** No resource in AWS changed, launched or deleted, ever - **1 alert:** The change and its likely cause, sent only when spend leaves its normal range A cost problem that once turned up as a baffling line item at month end now turns up the next morning, already labelled with the service, the account, the size of the jump and a likely cause. The agent reads and alerts. Your engineering team decides what to do about each one.