2022-05-26 23:06:25 +08:00
---
2024-03-14 23:58:18 +08:00
aliases:
- ../fundamentals/data-source-alerting/ # /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/data-source-alerting/
- ../fundamentals/alert-rules/alert-instances/ # /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/alert-instances/
2024-05-22 04:43:36 +08:00
- ../fundamentals/alert-rules/organising-alerts/ # /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/organising-alerts/
2024-03-14 23:58:18 +08:00
- ../fundamentals/alert-rules/alert-rule-types/ # /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/alert-rule-types/
2023-07-18 18:29:00 +08:00
canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/alert-rules/
2023-12-12 18:26:36 +08:00
description: Learn about alert rules
2022-05-26 23:06:25 +08:00
keywords:
- grafana
- alerting
- rules
Explicitly set all front matter labels in the source files (#71548)
* Set every page to have defaults of 'Enterprise' and 'Open source' labels
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration pages to have of 'Cloud', 'Enterprise', and 'Open source' labels
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration/enterprise-licensing pages to have 'Enterprise' labels
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration/organization-management pages to have 'Enterprise' and 'Open source' labels
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration/provisioning pages to have 'Enterprise' and 'Open source' labels
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration/recorded-queries pages to have labels cloud,enterprise
* Set administration/roles-and-permissions/access-control pages to have labels cloud,enterprise
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set administration/stats-and-license pages to have labels cloud,enterprise
* Set alerting pages to have labels cloud,enterprise,oss
* Set breaking-changes pages to have labels cloud,enterprise,oss
* Set dashboards pages to have labels cloud,enterprise,oss
* Set datasources pages to have labels cloud,enterprise,oss
* Set explore pages to have labels cloud,enterprise,oss
* Set fundamentals pages to have labels cloud,enterprise,oss
* Set introduction/grafana-cloud pages to have labels cloud
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix introduction pages products
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set panels-visualizations pages to have labels cloud,enterprise,oss
* Set release-notes pages to have labels cloud,enterprise,oss
* Set search pages to have labels cloud,enterprise,oss
* Set setup-grafana/configure-security/audit-grafana pages to have labels cloud,enterprise
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set setup-grafana/configure-security/configure-authentication pages to have labels cloud,enterprise,oss
* Set setup-grafana/configure-security/configure-authentication/enhanced-ldap pages to have labels cloud,enterprise
* Set setup-grafana/configure-security/configure-authentication/saml pages to have labels cloud,enterprise
* Set setup-grafana/configure-security/configure-database-encryption/encrypt-secrets-using-hashicorp-key-vault pages to have labels cloud,enterprise
* Set setup-grafana/configure-security/configure-request-security pages to have labels cloud,enterprise,oss
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set setup-grafana/configure-security/configure-team-sync pages to have labels cloud,enterprise
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set setup-grafana/configure-security/export-logs pages to have labels cloud,enterprise
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Set troubleshooting pages to have labels cloud,enterprise,oss
* Set whatsnew pages to have labels cloud,enterprise,oss
* Apply updated labels from review
Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
---------
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
2023-07-18 16:10:12 +08:00
labels:
products:
- cloud
- enterprise
- oss
2022-05-26 23:06:25 +08:00
title: Alert rules
2024-02-28 16:10:19 +08:00
weight: 100
2024-05-21 20:43:21 +08:00
refs:
2025-01-14 20:34:02 +08:00
shared-configure-prometheus-data-source-alerting:
- pattern: /docs/grafana/
destination: /docs/grafana/< GRAFANA_VERSION > /datasources/prometheus/configure-prometheus-data-source/#alerting
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/prometheus/configure-prometheus-data-source/#alerting
2024-05-21 20:43:21 +08:00
queries-and-conditions:
- pattern: /docs/grafana/
2025-02-12 17:42:53 +08:00
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/queries-conditions/#data-source-queries
2024-05-21 20:43:21 +08:00
- pattern: /docs/grafana-cloud/
2025-02-12 17:42:53 +08:00
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/queries-conditions/#data-source-queries
alert-condition:
- pattern: /docs/grafana/
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/queries-conditions/#alert-condition
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/queries-conditions/#alert-condition
recorded-queries:
- pattern: /docs/
destination: /docs/grafana/< GRAFANA_VERSION > /administration/recorded-queries/
2024-05-21 20:43:21 +08:00
notification-images:
- pattern: /docs/grafana/
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/configure-notifications/template-notifications/images-in-notifications/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/template-notifications/images-in-notifications/
2024-06-03 17:00:52 +08:00
notifications:
2024-05-21 20:43:21 +08:00
- pattern: /docs/grafana/
2024-06-03 17:00:52 +08:00
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/notifications/
2024-05-21 20:43:21 +08:00
- pattern: /docs/grafana-cloud/
2024-06-03 17:00:52 +08:00
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/
2024-05-21 20:43:21 +08:00
create-recording-rules:
- pattern: /docs/grafana/
2024-10-29 16:41:59 +08:00
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/alerting-rules/create-recording-rules/
2024-05-21 20:43:21 +08:00
- pattern: /docs/grafana-cloud/
2024-10-29 16:41:59 +08:00
destination: /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-recording-rules/
2024-05-21 20:43:21 +08:00
expression-queries:
- pattern: /docs/grafana/
2025-02-12 17:42:53 +08:00
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/queries-conditions/#advanced-options-expressions
2024-05-21 20:43:21 +08:00
- pattern: /docs/grafana-cloud/
2025-02-12 17:42:53 +08:00
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/queries-conditions/#advanced-options-expressions
2024-05-21 20:43:21 +08:00
alert-rule-evaluation:
- pattern: /docs/grafana/
destination: /docs/grafana/< GRAFANA_VERSION > /alerting/fundamentals/alert-rules/rule-evaluation/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/rule-evaluation/
2022-05-26 23:06:25 +08:00
---
2022-05-25 04:04:25 +08:00
2023-04-21 19:24:43 +08:00
# Alert rules
2022-05-25 04:04:25 +08:00
2024-03-21 18:06:49 +08:00
An alert rule is a set of evaluation criteria for when an alert rule should fire. An alert rule consists of:
2022-05-25 04:04:25 +08:00
2025-02-12 17:42:53 +08:00
- [Queries ](ref:queries-and-conditions ) that select the dataset to evaluate.
- An [alert condition ](ref:alert-condition ) (the threshold) that the query must meet or exceed to trigger the alert instance.
2024-05-21 20:43:21 +08:00
- An interval that specifies the frequency of [alert rule evaluation ](ref:alert-rule-evaluation ) and a duration indicating how long the condition must be met to trigger the alert instance.
2024-03-21 18:06:49 +08:00
- Other customizable options, for example, setting what should happen in the absence of data, notification messages, and more.
2022-05-25 04:04:25 +08:00
2024-03-21 18:06:49 +08:00
Grafana supports two different alert rule types: Grafana-managed alert rules and data source-managed alert rules.
2024-03-14 23:58:18 +08:00
## Grafana-managed alert rules
2024-10-29 16:41:59 +08:00
Grafana-managed alert rules are the most flexible alert rule type. They allow you to create alert rules that can act on data from any of the [supported data sources ](#supported-data-sources ), and use multiple data sources in a single alert rule.
2024-03-14 23:58:18 +08:00
2024-03-21 18:06:49 +08:00
{{< figure src = "/media/docs/alerting/grafana-managed-alerting-architecture.png" max-width = "750px" caption = "How Grafana-managed alerting works by default" > }}
2024-03-14 23:58:18 +08:00
2025-01-14 20:34:02 +08:00
1. Alert rules are created and stored within Grafana.
1. Alert rules can query one or more supported data sources.
1. Alert rules are evaluated by the Alert Rule Evaluation Engine within Grafana.
2024-06-03 17:00:52 +08:00
1. Firing and resolved alert instances are forwarded to [handle their notifications ](ref:notifications ).
2024-03-14 23:58:18 +08:00
### Supported data sources
2024-10-29 16:41:59 +08:00
Grafana-managed alert rules can query backend data sources if Grafana Alerting is enabled by specifying `{"backend": true, "alerting": true}` in the `plugin.json` file.
2024-03-14 23:58:18 +08:00
2024-04-15 16:56:01 +08:00
Find the public data sources supporting Alerting in the [Grafana Plugins directory ](/grafana/plugins/data-source-plugins/?features=alerting ).
2024-03-14 23:58:18 +08:00
## Data source-managed alert rules
2025-01-14 20:34:02 +08:00
Data source-managed alert rules can only be created using Grafana Mimir or Grafana Loki data sources. Both data source backends can provide high availability and fault tolerance, enabling you to scale your alerting setup.
2024-03-14 23:58:18 +08:00
2024-03-21 18:06:49 +08:00
{{< figure src = "/media/docs/alerting/mimir-managed-alerting-architecture-v2.png" max-width = "750px" caption = "Mimir-managed alerting architecture" > }}
2024-03-14 23:58:18 +08:00
2025-01-14 20:34:02 +08:00
1. Alert rules are stored within the Mimir or Loki data source.
1. Alert rules can query only their specific data source.
1. Alert rules are evaluated by the Alert Rule Evaluation Engine within the data source.
2024-06-03 17:00:52 +08:00
1. Firing and resolved alert instances are forwarded to [handle their notifications ](ref:notifications ).
2024-03-14 23:58:18 +08:00
2025-01-14 20:34:02 +08:00
{{< docs / shared lookup = "alerts/note-prometheus-ds-rules.md" source = "grafana" version = "<GRAFANA_VERSION>" > }}
2024-12-09 16:43:19 +08:00
## Comparison between alert rule types
2024-03-14 23:58:18 +08:00
2024-12-09 16:43:19 +08:00
We recommend using Grafana-managed alert rules whenever possible, and opting for data source-managed alert rules when you need to scale your alerting setup.
2024-03-14 23:58:18 +08:00
2024-12-09 16:43:19 +08:00
The table below compares Grafana-managed and data source-managed alert rules.
2024-03-14 23:58:18 +08:00
2024-12-09 16:43:19 +08:00
| < div style = "width:200px" > Feature< / div > | < div style = "width:200px" > Grafana-managed alert rule< / div > | < div style = "width:200px" > Data source-managed alert rule |
| ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
2025-01-14 20:34:02 +08:00
| Create alert rules< wbr /> that query [data sources supporting Alerting ](#supported-data-sources ) | Yes | Only supports creating rules for Mimir and Loki. |
2024-12-09 16:43:19 +08:00
| Mix and match data sources | Yes | No |
| Add [expressions ](ref:expression-queries ) to transform< wbr /> your data and set [alert conditions ](ref:alert-condition ) | Yes | No |
| Use [images in alert notifications ](ref:notification-images ) | Yes | No |
| Support for [recording rules ](#recording-rules ) | Yes | Yes |
| Organization | Organize and manage access with folders | Use namespaces |
| Alert rule evaluation and delivery | Alert evaluation is done in Grafana, while delivery can be handled by Grafana or an external Alertmanager. | Alert rule evaluation and alert delivery are distributed. |
| Scaling | Alert rules are stored in the Grafana database, which may experience transient errors. It only scales vertically. | Alert rules are stored within the data source and allow for horizontal scaling. |
2024-03-14 23:58:18 +08:00
2024-12-09 16:43:19 +08:00
## Recording rules
Similar to alert rules, recording rules are evaluated periodically. A recording rule pre-computes frequently used or computationally expensive queries, and saves the results as a new time series metric.
The new recording metric can then be used in alert rules and dashboards to optimize their queries.
2024-03-14 23:58:18 +08:00
2024-12-09 16:43:19 +08:00
For more details, refer to [Create recording rules ](ref:create-recording-rules ).