Alerting Patterns and Anti-Patterns

Alerting Patterns and Anti-Patterns is where observability becomes interruption. Dashboards, metrics, and traces help people investigate when they choose to look. Alerts choose for them. That is why alert design is unusually sensitive: a good alert accelerates response to real customer impact, while a weak alert wastes attention, erodes trust, and trains teams to ignore the system.

Alerting therefore needs a higher standard than general telemetry. An alert should represent a condition that is both important and actionable. It should reach the right people, at the right severity, with enough context to start the right response. When that discipline is absent, the organization gets duplicate pages, ambiguous ownership, and fatigue that eventually makes the observability stack less credible.

This chapter covers symptom-first alerting, threshold and anomaly strategies, routing and escalation, and the anti-patterns that turn an on-call program into a noise distribution system rather than an incident-response mechanism.

In this section

• Alerting on Symptoms vs Causes
  Why pager-worthy alerts should usually represent user-visible symptoms, while cause-level signals are better used for diagnosis and enrichment.
• Threshold, Anomaly, and Multi-Signal Alerts
  When simple thresholds work, when anomaly models help, and how multi-signal alerts reduce noise by combining stronger evidence.
• Routing, Escalation, and Human Response Design
  How alert ownership, escalation rules, and human-friendly context determine whether the right responder can act quickly and correctly.
• Alert Fatigue and Anti-Patterns
  The common alerting failures that create flapping, duplication, low trust, and chronic interruption without better incident response.