Fastly: Single customer config change triggers global Fastly CDN outage
A valid customer configuration change exposed a latent software bug in Fastly's edge servers, causing 85% of the network to return errors. Took down major sites including Amazon, Reddit, Twitch, NYT, UK gov.uk, and Stack Overflow simultaneously.
2 of the 4 practices that failed in this incident are part of the Sentinel-10 — the engineering protocols Concordance flags as most degraded under AI-accelerated development.
This incident pre-dates today's AI-velocity surge. The thesis is that the same practices that failed here will fail faster under AI velocity if not actively governed. Read the Velocity Governance thesis →
Impact
Approximately 85% of Fastly's global network returned errors. Affected services serving billions of requests. Highlighted internet's dependence on a small number of CDN providers.
Root cause (from published RCA)
A valid customer configuration triggered a bug in software that had been deployed on May 12 — the bug had been latent for nearly a month before customer config exposed it. When the configuration was applied, ~85% of the network began returning errors. The bug was in code that processed VCL configuration changes; affected POPs were all in the same software version.
Concordance protocols that map to this root cause
Click any protocol to see every other indexed incident where it failed.
Primary sources
Related incidents
Other incidents that failed at least one of the same protocols.