Fort Grid
The Storage Cascade
Power and cooling look fine. Applications slow, then error.
More dependents breach if retries keep amplifying a shared storage fault.
DEMONSTRATION SCENARIO
This is a demonstration scenario. It is not a live demonstration and does not report an executed result.
The situation
Applications depend on one storage path. Facility power and cooling are up.
Applications slow, then error. Power and cooling do not explain the room.
Why this is hard
The applications can be failing because the shared store is impaired and retries make it worse, because each application broke alone, or because the hall lost power. A green storage light is not application success.
Repairing each application, or stopping at a restored storage bit, should not count while latency and errors remain.
What the operator can see
- NormalPower and cooling up
- VisibleApps slow, then error
- At riskRetries spread it
What V8Y must determine
V8Y must distinguish a shared store and its retries from applications that failed alone and from a hall that lost power, and judge recovery by application latency.
What may happen next
More dependents breach if retries continue.
What counts as recovery
Not recovery
Success is not a storage light.
Recovery
Success is that the applications meet latency and success.
Observed results
No executed results are published for this thread.
What this evaluation would test
- Storage separated from power and cooling.
- Retries treated as part of the fault.
- Many applications, one path.
- Application latency is the edge.
These are the conditions the evaluation would test. They are not reported results.
Operational basis
View operational basis and sources
Derived from Azure’s 18–19 July 2024 Central US storage incident: a shared storage fault spread into dependent services, which took longer to recover than the first storage repair. The facility setting is synthetic.
Discuss evaluating this situation
Does your environment face a similar problem?
The request keeps this mission and thread. It is not a claim that the demonstration has been validated in your environment.
The printable flyer is being revised and is not available for download.