Most business automations break quietly, and nobody notices for weeks

When a web app breaks, someone sees it immediately: a page 500s and a user complains within minutes. When an automation breaks, nothing happens. No one is looking at it. The invoices simply stop syncing, and three weeks later someone asks why the numbers are wrong.
That asymmetry should change how you build them, because an automation's whole risk profile sits in silence.
Build for the failure you will actually get
The failures I see in practice are rarely logic bugs:
- An API changed a field name in a minor release
- A token expired and the refresh was never wired up
- A rate limit was hit, and the retry made it worse
- Someone renamed a column in the spreadsheet everything depends on
- The workflow ran fine but a downstream step silently dropped the payload
Notice how few of these are your code being wrong. Most are other systems changing under the assumptions your integration made.
Three things that make one survivable
Fail loudly. Every workflow needs a path that notifies a human when it breaks, and that alert has to reach somewhere people actually look. An error log nobody opens is indistinguishable from no error handling. Route it to whatever channel already interrupts them.
Make it idempotent. You will re-run things. A retry that creates a second invoice is worse than no retry at all. Key every operation on something stable from the source system so running it twice is a no-op.
Add a heartbeat. This is the one people skip. A broken workflow and a workflow with nothing to do look identical from the outside, because both do nothing. Emit a signal on every successful run and alert when the signal stops, not only when an error appears. Otherwise you are relying on someone noticing an absence.
The first two are one database constraint and one upsert:
-- Make the duplicate impossible in the database, not in your workflow logic.
create unique index invoices_source_key on invoices (source_system, source_id);async function syncInvoice(record: SourceInvoice) {
await db
.insertInto('invoices')
.values({
source_system: 'billing',
source_id: record.id, // stable in the source system, not generated here
amount_cents: record.amountCents,
synced_at: new Date()
})
// Re-running the workflow updates the row instead of creating a second one.
.onConflict((c) =>
c.columns(['source_system', 'source_id']).doUpdateSet({ synced_at: new Date() })
)
.execute()
// The heartbeat: prove the run finished, so silence becomes detectable.
await heartbeat('invoice-sync')
}Note where the idempotency lives. Enforcing it in workflow code means every future branch has to remember the rule; enforcing it as a unique index means the database refuses the duplicate no matter which path reached it.
Do not automate a process you have not written down
The most expensive automation work I have done was untangling a workflow that faithfully encoded a process nobody had ever agreed on. Every exception was somebody's undocumented habit, and each one had been translated into a branch.
Write the process down first, in plain language, including who handles the exceptions. If that document is contradictory, the automation will be too, and much harder to read.
Where the value actually is
The wins are boring, repetitive, low-judgement jobs: moving data between systems that should already talk, generating the same document weekly, chasing a status nobody wants to chase.
Anything requiring real judgement should stay with a person, with automation doing the gathering so the decision arrives pre-assembled. That division holds up far better than trying to encode the judgement itself.
I build automations that tell you when they break.
If someone on your team repeats the same job every day, or a sync only looks healthy because nobody has complained yet, I can look at it and tell you what is worth automating first.