Automating processes with software is hard
Automating real‑world processes with software often trades one set of problems for another, as rigid rules and validation struggle with messy edge cases, inconsistent inputs, and evolving business needs. Commenters argue that effective automation focuses on limiting state and complexity, accepting that 100% coverage is unrealistic and designing hybrid workflows where humans handle exceptions through explicit interfaces, overrides, and audit trails. Several practical strategies emerge: start by systematizing and documenting the manual process (e.g., checklists or “do nothing” scripts), automate only stable, high‑value steps, and be willing to say no to rarely used or high‑cost special cases.
Why automating processes is hard
- Automation makes processes rigid and legible but strips nuance and informal resilience; over‑automation can damage what originally worked well.
- Software forces determinism and consistent state; many real-world processes are fuzzy, inconsistent, or context-dependent, so they resist clean formalization.
State explosion and process design
- A recurring theme: good automation is “state limitation.”
- Each exception or edge case adds new states and intermediate states, leading to spaghetti logic.
- Recommended patterns:
- Funnel-shaped flows that normalize inputs and collapse branches early.
- Route out divergent cases and, if needed, build separate flows for them rather than one giant system.
- Reunify divergent paths into a single representation (e.g., “estimated income” regardless of data source).
Human-in-the-loop and overrides
- Many argue for explicit “I know what I’m doing” / override mechanisms for edge cases and outages.
- These must be carefully designed:
- Risk: downstream bugs and broken assumptions.
- Need: logging, approvals, audits, and treating overrides more like code review than rubber stamps.
- Toyota-style “jidoka”: automation must be stoppable and inspectable by humans.
When (not) to automate
- Don’t aim for 100% automation; accept that some portion will stay manual.
- Question whether to automate at all:
- Essential in domains like logistics or manufacturing; more dubious for rare, low-impact internal processes.
- Some say best practice is “automate everything”; others counter that cost/benefit and opportunity cost are often ignored.
- Sometimes the right move is to change or simplify the business process (e.g., saying “no” to rare special cases) rather than encode every exception.
Gradual and hybrid approaches
- “Do-nothing scripts” and checklist automation: first script the steps as prompts, then automate individual steps over time.
- Hybrid flows (part software, part notes/manual work) help discover patterns safely before full automation.
- Several deployment and operations stories emphasize incremental trust-building: start manual-with-tooling, then progressively remove human steps.
Organizational and human factors
- Automation projects often fail when domain experts are excluded; software alone can’t fix process problems.
- Overzealous product/management pushes (“automate everything”, LLM-defined requirements) are seen as a major risk.
- Simplicity, explicit state, and post-launch cleanup are repeatedly cited as key but chronically under-valued.