The request sounded straightforward: automate a manual approval that delayed every release.
We could have built a workflow engine, notifications, escalation timers, audit pages, and an override role. Instead, we asked why this approval existed.
The answer was a production incident from much earlier. One unsafe configuration change had escaped review, so an approval step was added for every deployment. The underlying configuration path had since changed, but the process remained.
Automation would have preserved the wrong thing #
The manual step was slow and inconsistent. Automating it would make the same policy faster and more consistent. It would not make the policy useful.
We separated the original risk from the accumulated ritual:
- What unsafe action had caused the incident?
- Could the system reject it automatically?
- Which changes were actually high risk?
- What evidence did an approver inspect?
- Was approval preventing failure or distributing accountability?
Most releases did not touch the risky configuration. The approver usually checked that tests passed and the deployment was staged—facts the pipeline already knew.
The resolution #
We encoded the invariant at the boundary that could enforce it, added a policy test for dangerous configuration combinations, and required explicit review only when a change crossed that boundary. Ordinary releases no longer entered the approval workflow.
The final system had less code than the proposed automation:
ordinary change -> automated evidence -> staged rollout
risky change -> automated evidence -> named review -> staged rolloutThe audit trail improved because the exceptional decision now carried a reason. Previously, hundreds of routine approvals hid the few that mattered.
What I kept #
Before automating a process, ask whether the process is a product requirement, a control, or scar tissue. Automation multiplies whichever one it receives.
Deleting a workflow can look less impressive than building a platform. It is often the higher-leverage engineering decision because it removes code, waiting, operational ownership, and an opportunity for the organization to confuse activity with safety.
All Field Notes · The cost of complexity · Verify what you deploy