Failure Atlas

Navigate failure domains, deterministic reproductions, containment guardrails, and postmortems as one connected documentation system.

FP = Failure Pattern FM = Failure Mode / Reproduction GR = Guardrail PM = Postmortem

FP_001 · Duplicate Execution After Retry Timeout

Type
FP
Failure domain
State Consistency
Mechanism
Retry Idempotence Window
Status
validated
View source on GitHub

Related artifacts

FM_001 · GR_001 · PM_001 · FP_002 · FP_003

Failure Pattern

At-least-once retry after lease timeout can apply one logical effect more than once unless a durable idempotency boundary exists.

Hidden Assumption

"Timeout implies no committed side effect was produced."

Trigger Condition

Worker A lease expires before completion; Worker B retries the same job while A may still complete.

Failure Mechanism

Two executions race toward effect application. Without first-commit gating, both cross the effect boundary.

Observable Symptoms

  • count_effects(job_id) > 1
  • multiple successful executions for one logical job
  • correctness drift requiring reconciliation

Detection

count_effects(job_id) > 1

Lab Reproduction

  • lab/failure_modes/FM_001_duplicate_retry/

Relevant Guardrails

  • guardrails/GR_001_idempotent_commit_boundary.md

Postmortem

  • lab/postmortems/PM_001_duplicate_execution.md
  • FP_002 Extension Authority Persistence
  • FP_003 Read-only Enforcement Gap