mirror of https://github.com/grafana/grafana.git
[release-11.6.1] docs(alerting): Configure Grafana IRM (#103578)
docs(alerting): Configure Grafana IRM (#103549)
* docs(alerting): Configure Grafana IRM
* update `PagerDuty`
(cherry picked from commit 21fe9480a1)
This commit is contained in:
Pepe Cano
GitHub
parent
3549a95aa6
commit
646b11a964
|
|
@ -66,11 +66,11 @@ refs:
|
||||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/pager-duty/
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/pager-duty/
|
||||||
- pattern: /docs/grafana-cloud/
|
- pattern: /docs/grafana-cloud/
|
||||||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/pager-duty/
|
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/pager-duty/
|
||||||
oncall:
|
irm:
|
||||||
- pattern: /docs/grafana/
|
- pattern: /docs/grafana/
|
||||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-oncall/
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-irm/
|
||||||
- pattern: /docs/grafana-cloud/
|
- pattern: /docs/grafana-cloud/
|
||||||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/configure-oncall/
|
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/configure-irm/
|
||||||
slack:
|
slack:
|
||||||
- pattern: /docs/grafana/
|
- pattern: /docs/grafana/
|
||||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-slack/
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-slack/
|
||||||
|
|
@ -115,7 +115,7 @@ refs:
|
||||||
|
|
||||||
# Configure contact points
|
# Configure contact points
|
||||||
|
|
||||||
Use contact points to specify where to receive alert notifications. Contact points contain the configuration for sending alert notifications, including destinations like email, Slack, OnCall, webhooks, and their notification messages.
|
Use contact points to specify where to receive alert notifications. Contact points contain the configuration for sending alert notifications, including destinations like email, Slack, IRM, webhooks, and their notification messages.
|
||||||
|
|
||||||
A contact point can have one or multiple destinations, known as [contact point integrations](#supported-contact-point-integrations). Alert notifications are sent to each integration within the chosen contact point.
|
A contact point can have one or multiple destinations, known as [contact point integrations](#supported-contact-point-integrations). Alert notifications are sent to each integration within the chosen contact point.
|
||||||
|
|
||||||
|
|
@ -145,7 +145,7 @@ Each contact point integration has its own configuration options and setup proce
|
||||||
- [Discord](ref:discord)
|
- [Discord](ref:discord)
|
||||||
- [Email](ref:email)
|
- [Email](ref:email)
|
||||||
- [Google Chat](ref:gchat)
|
- [Google Chat](ref:gchat)
|
||||||
- [Grafana OnCall](ref:oncall)
|
- [Grafana IRM](ref:irm)
|
||||||
- Kafka REST Proxy
|
- Kafka REST Proxy
|
||||||
- [Jira](ref:jira)
|
- [Jira](ref:jira)
|
||||||
- Line
|
- Line
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,155 @@
|
||||||
|
---
|
||||||
|
aliases:
|
||||||
|
- ../../../configure-notifications/manage-contact-points/integrations/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-oncall/
|
||||||
|
- ../../../alerting-rules/manage-contact-points/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
||||||
|
- ../../../alerting-rules/manage-contact-points/integrations/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/integrations/configure-oncall/
|
||||||
|
- ../configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
||||||
|
canonical: https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/integrations/configure-irm/
|
||||||
|
description: Configure the Alerting - Grafana IRM integration to connect alerts generated by Grafana Alerting with Grafana IRM
|
||||||
|
keywords:
|
||||||
|
- grafana
|
||||||
|
- alerting
|
||||||
|
- oncall
|
||||||
|
- irm
|
||||||
|
- integration
|
||||||
|
labels:
|
||||||
|
products:
|
||||||
|
- cloud
|
||||||
|
menuTitle: Grafana IRM
|
||||||
|
title: Configure Grafana IRM for Alerting
|
||||||
|
weight: 120
|
||||||
|
refs:
|
||||||
|
notification-templates:
|
||||||
|
- pattern: /docs/grafana/
|
||||||
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/template-notifications/
|
||||||
|
- pattern: /docs/grafana-cloud/
|
||||||
|
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/template-notifications/
|
||||||
|
irm:
|
||||||
|
- pattern: /docs/
|
||||||
|
destination: /docs/grafana-cloud/alerting-and-irm/irm/
|
||||||
|
irm-alert-templates:
|
||||||
|
- pattern: /docs/
|
||||||
|
destination: /docs/grafana-cloud/alerting-and-irm/irm/configure/escalation-routing/alert-templates/
|
||||||
|
webhook-json-payload:
|
||||||
|
- pattern: /docs/grafana/
|
||||||
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier/#json-payload
|
||||||
|
- pattern: /docs/grafana-cloud/
|
||||||
|
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier/#json-payload
|
||||||
|
configure-contact-points:
|
||||||
|
- pattern: /docs/grafana/
|
||||||
|
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/
|
||||||
|
- pattern: /docs/grafana-cloud/
|
||||||
|
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/
|
||||||
|
---
|
||||||
|
|
||||||
|
# Configure Grafana IRM for Alerting
|
||||||
|
|
||||||
|
Use the Grafana IRM integration in contact points to send alerts to [Grafana IRM](ref:irm). You can then route them based on the escalation chains and schedules defined in Grafana IRM.
|
||||||
|
|
||||||
|
## Before you begin
|
||||||
|
|
||||||
|
The Grafana IRM contact point is only available when using the built-in Grafana Alertmanager.
|
||||||
|
|
||||||
|
Ensure that Grafana IRM is activated in your Grafana Cloud instance.
|
||||||
|
|
||||||
|
## Configure Grafana IRM for a contact point
|
||||||
|
|
||||||
|
To create a contact point with a Grafana IRM integration, complete the following steps:
|
||||||
|
|
||||||
|
1. Navigate to **Alerts & IRM** -> **Alerting** -> **Contact points**.
|
||||||
|
1. Click **+ Add contact point**.
|
||||||
|
1. Enter a name for the contact point.
|
||||||
|
1. From the **Integration** list, select **Grafana IRM**.
|
||||||
|
1. Choose whether to add a new IRM integration or add an existing one.
|
||||||
|
- If you add a new one, enter an integration name.
|
||||||
|
- If you add an existing one, choose from the list of available integrations.
|
||||||
|
1. (Optional) Configure [optional settings](#webhook-settings).
|
||||||
|
1. Click **Save contact point**.
|
||||||
|
|
||||||
|
For more details on contact points, including how to test them and enable notifications, refer to [Configure contact points](ref:configure-contact-points).
|
||||||
|
|
||||||
|
### Optional settings
|
||||||
|
|
||||||
|
The IRM contact point in Grafana Alerting integrates with Grafana IRM using webhook requests. The following settings allow you to customize webhook requests when needed.
|
||||||
|
|
||||||
|
| Key | Description |
|
||||||
|
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| HTTP Method | Specifies the HTTP method to use: `POST` or `PUT`. |
|
||||||
|
| Basic Authentication Username | Username for HTTP Basic Authentication. |
|
||||||
|
| Basic Authentication Password | Password for HTTP Basic Authentication. |
|
||||||
|
| Authentication Header Scheme | Scheme for the `Authorization` Request Header. Default is `Bearer`. |
|
||||||
|
| Authentication Header Credentials | Credentials for the `Authorization` Request header. |
|
||||||
|
| Max Alerts | Maximum number of alerts to include in a notification. Any alerts exceeding this limit are ignored. `0` means no limit. |
|
||||||
|
|
||||||
|
### Optional payload settings
|
||||||
|
|
||||||
|
The IRM contact point sends alert messages to Grafana IRM using the [JSON payload format used by the webhook integration](ref:webhook-json-payload):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"orgId": 1,
|
||||||
|
"state": "alerting",
|
||||||
|
"title": "[FIRING:1] TestAlert Grafana ",
|
||||||
|
"alerts": [
|
||||||
|
{
|
||||||
|
"endsAt": "0001-01-01T00:00:00Z",
|
||||||
|
"labels": {
|
||||||
|
"instance": "Grafana",
|
||||||
|
"alertname": "TestAlert"
|
||||||
|
},
|
||||||
|
"status": "firing",
|
||||||
|
"values": null,
|
||||||
|
"panelURL": "",
|
||||||
|
"startsAt": "2025-04-07T14:42:52.388041666Z",
|
||||||
|
"silenceURL": "https://alerttest.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DTestAlert&matcher=instance%3DGrafana",
|
||||||
|
"annotations": {
|
||||||
|
"summary": "Notification test"
|
||||||
|
},
|
||||||
|
"fingerprint": "57c6d9296de2ad39",
|
||||||
|
"valueString": "[ metric='foo' labels={instance=bar} value=10 ]",
|
||||||
|
"dashboardURL": "",
|
||||||
|
"generatorURL": ""
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "firing",
|
||||||
|
"message": "**Firing**\n\nValue: [no value]\nLabels:\n - alertname = TestAlert\n - instance = Grafana\nAnnotations:\n - summary = Notification test\nSilence: https://alerttest.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DTestAlert&matcher=instance%3DGrafana\n",
|
||||||
|
"version": "1",
|
||||||
|
"groupKey": "test-57c6d9296de2ad39-1744036972",
|
||||||
|
"receiver": "test",
|
||||||
|
"numFiring": 1,
|
||||||
|
"externalURL": "https://alerttest.grafana.net/",
|
||||||
|
"groupLabels": {
|
||||||
|
"instance": "Grafana",
|
||||||
|
"alertname": "TestAlert"
|
||||||
|
},
|
||||||
|
"numResolved": 0,
|
||||||
|
"commonLabels": {
|
||||||
|
"instance": "Grafana",
|
||||||
|
"alertname": "TestAlert"
|
||||||
|
},
|
||||||
|
"truncatedAlerts": 0,
|
||||||
|
"commonAnnotations": {
|
||||||
|
"summary": "Notification test"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also customize some fields in the JSON payload using [notification templates](ref:notification-templates).
|
||||||
|
|
||||||
|
| Option | Description |
|
||||||
|
| ------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| Title | Sends the value as a string in the `title` field of the JSON payload. Supports [notification templates](ref:notification-templates). |
|
||||||
|
| Message | Sends the value as a string in the `message` field of the JSON payload. Supports [notification templates](ref:notification-templates). |
|
||||||
|
|
||||||
|
{{< admonition type="note" >}}
|
||||||
|
|
||||||
|
A Grafana IRM contact point can either create or use a Grafana Alerting integration from your Grafana IRM account.
|
||||||
|
|
||||||
|
The Grafana Alerting integration in IRM includes preconfigured templates that automatically process JSON messages sent from Grafana Alerting. For additional details, refer to [Configure IRM alert templates](ref:irm-alert-templates).
|
||||||
|
{{< /admonition >}}
|
||||||
|
|
||||||
|
### Optional notification settings
|
||||||
|
|
||||||
|
| Key | Description |
|
||||||
|
| ------------------------ | ------------------------------------------------------------------- |
|
||||||
|
| Disable resolved message | Enable this option to prevent notifications when an alert resolves. |
|
||||||
|
|
@ -45,7 +45,7 @@ To create a contact point with a Jira integration, complete the following steps:
|
||||||
5. Set up the required [settings](#required-settings) for your Jira configuration.
|
5. Set up the required [settings](#required-settings) for your Jira configuration.
|
||||||
6. Click **Save contact point**.
|
6. Click **Save contact point**.
|
||||||
|
|
||||||
For more details on contact points, including how to test them and enable notifications, refer to [Configure contact points](/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/).
|
For more details on contact points, including how to test them and enable notifications, refer to [Configure contact points](ref:configure-contact-points).
|
||||||
|
|
||||||
### Required Settings
|
### Required Settings
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,73 +0,0 @@
|
||||||
---
|
|
||||||
aliases:
|
|
||||||
- ../../../alerting-rules/manage-contact-points/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
|
||||||
- ../../../alerting-rules/manage-contact-points/integrations/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/integrations/configure-oncall/
|
|
||||||
- ../configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
|
||||||
canonical: https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/integrations/configure-oncall/
|
|
||||||
description: Configure the Alerting - Grafana OnCall integration to connect alerts generated by Grafana Alerting with Grafana OnCall
|
|
||||||
keywords:
|
|
||||||
- grafana
|
|
||||||
- alerting
|
|
||||||
- oncall
|
|
||||||
- integration
|
|
||||||
labels:
|
|
||||||
products:
|
|
||||||
- cloud
|
|
||||||
- enterprise
|
|
||||||
- oss
|
|
||||||
menuTitle: Grafana OnCall
|
|
||||||
title: Configure Grafana OnCall for Alerting
|
|
||||||
weight: 120
|
|
||||||
---
|
|
||||||
|
|
||||||
# Configure Grafana OnCall for Alerting
|
|
||||||
|
|
||||||
Use the Grafana Alerting - Grafana OnCall integration to effortlessly connect alerts generated by Grafana Alerting with Grafana OnCall, where you can then route them according to defined escalation chains and schedules.
|
|
||||||
|
|
||||||
{{< admonition type="note" >}}
|
|
||||||
You can also configure the integration from Grafana OnCall. For more information, refer to [Grafana OnCall documentation](http://grafana.com/docs/oncall/latest/configure/integrations/references/grafana-alerting/).
|
|
||||||
{{< /admonition >}}
|
|
||||||
|
|
||||||
## Before you begin
|
|
||||||
|
|
||||||
Grafana OnCall is available in Grafana Cloud by default.
|
|
||||||
|
|
||||||
If you are using Grafana OSS, [install and enable the Grafana OnCall plugin](http://grafana.com/docs/oncall/latest/set-up/open-source/#install-grafana-oncall-oss). Also, ensure your version of Grafana is up-to-date and supports the new features.
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
|
|
||||||
To set up the Grafana OnCall integration using the Grafana Alerting application, complete the following steps.
|
|
||||||
|
|
||||||
1. Navigate to **Alerts & IRM** -> **Alerting** -> **Contact points**.
|
|
||||||
1. Click **+ Add contact point**.
|
|
||||||
1. Enter a contact point name.
|
|
||||||
1. From the **Integration** list, select **Grafana OnCall**.
|
|
||||||
|
|
||||||
{{< admonition type="note" >}}
|
|
||||||
The Grafana OnCall integration is only available for Grafana Alertmanager.
|
|
||||||
{{< /admonition >}}
|
|
||||||
|
|
||||||
1. Choose whether to add a new OnCall integration or add an existing one.
|
|
||||||
- If you add a new one, enter an Integration name.
|
|
||||||
- If you add an existing one, choose from the list of available integrations.
|
|
||||||
1. Click **Save contact point**.
|
|
||||||
1. On the contact points list view page, you should see a link to Grafana OnCall.
|
|
||||||
|
|
||||||
If the integration is not being used anywhere in the notification policies tree, it has **Unused** as its status in the **Health** column. It won’t receive any notifications, because there are no notifications using that integration.
|
|
||||||
|
|
||||||
1. To see the integration details and test the integration, click the link to Grafana OnCall on the contact points list view page.
|
|
||||||
1. Click **Send demo alert** > **Send alert**.
|
|
||||||
1. Go to Grafana OnCall **Alert Groups**. You should see the demo alert.
|
|
||||||
|
|
||||||
## Next steps
|
|
||||||
|
|
||||||
The OnCall contact point is ready to receive alert notifications.
|
|
||||||
|
|
||||||
To add the contact point to your alert rule, complete the following next steps:
|
|
||||||
|
|
||||||
1. In Grafana, navigate to **Alerting** > **Alert rules**.
|
|
||||||
1. Edit or create a new alert rule.
|
|
||||||
1. Scroll down to the **Configure labels and notifications** section.
|
|
||||||
1. Under **Notifications**, click **Select contact point**.
|
|
||||||
1. From the drop-down menu, select the contact point you created.
|
|
||||||
1. Click **Save rule and exit**.
|
|
||||||
|
|
@ -105,7 +105,7 @@ Grafana supports a wide range of contact points with varied support for images i
|
||||||
| MQTT | No | No |
|
| MQTT | No | No |
|
||||||
| Microsoft Teams | No | Yes |
|
| Microsoft Teams | No | Yes |
|
||||||
| Opsgenie | No | Yes |
|
| Opsgenie | No | Yes |
|
||||||
| Pagerduty | No | Yes |
|
| PagerDuty | No | Yes |
|
||||||
| Prometheus Alertmanager | No | No |
|
| Prometheus Alertmanager | No | No |
|
||||||
| Pushover | Yes (Maximum of 1 per notification) | No |
|
| Pushover | Yes (Maximum of 1 per notification) | No |
|
||||||
| Sensu Go | No | No |
|
| Sensu Go | No | No |
|
||||||
|
|
|
||||||
|
|
@ -108,7 +108,7 @@ A rule using the PromQL expression above creates as many alert instances as the
|
||||||
|
|
||||||
### Contact points
|
### Contact points
|
||||||
|
|
||||||
[Contact points](ref:contact-points) determine the notification message and where notifications are sent. For example, you might have a contact point that sends notifications to an email address, to Slack, to an incident management system (IRM) such as Grafana OnCall or Pagerduty, or to a webhook.
|
[Contact points](ref:contact-points) determine the notification message and where notifications are sent. For example, you might have a contact point that sends notifications to an email address, to Slack, to an incident management system (IRM) such as Grafana IRM or PagerDuty, or to a webhook.
|
||||||
|
|
||||||
### Notification messages
|
### Notification messages
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -82,7 +82,7 @@ Start defining your [contact points](ref:contact-points) to specify how to recei
|
||||||
|
|
||||||
### Contact points
|
### Contact points
|
||||||
|
|
||||||
[Contact points](ref:contact-points) contain the configuration for sending alert notifications, specifying destinations like email, Slack, OnCall, webhooks, and their notification messages.
|
[Contact points](ref:contact-points) contain the configuration for sending alert notifications, specifying destinations like email, Slack, IRM, webhooks, and their notification messages.
|
||||||
|
|
||||||
A contact point is a list of integrations, each sending a message to a specific destination.
|
A contact point is a list of integrations, each sending a message to a specific destination.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -43,14 +43,14 @@ A contact point includes one or more contact point integrations for sending aler
|
||||||
- Discord
|
- Discord
|
||||||
- Email
|
- Email
|
||||||
- Google Chat
|
- Google Chat
|
||||||
- Grafana Oncall
|
- Grafana IRM
|
||||||
- Jira
|
- Jira
|
||||||
- Kafka REST Proxy
|
- Kafka REST Proxy
|
||||||
- Line
|
- Line
|
||||||
- Microsoft Teams
|
- Microsoft Teams
|
||||||
- MQTT
|
- MQTT
|
||||||
- Opsgenie
|
- Opsgenie
|
||||||
- Pagerduty
|
- PagerDuty
|
||||||
- Pushover
|
- Pushover
|
||||||
- Sensu Go
|
- Sensu Go
|
||||||
- Slack
|
- Slack
|
||||||
|
|
@ -62,7 +62,7 @@ A contact point includes one or more contact point integrations for sending aler
|
||||||
|
|
||||||
{{< /column-list >}}
|
{{< /column-list >}}
|
||||||
|
|
||||||
For example, a contact point could contain a Pagerduty integration; an email and Slack integration; or a Pagerduty integration, a Slack integration, and two email integrations. You can also configure a contact point with no integrations; in which case no notifications are sent.
|
For example, a contact point could contain a PagerDuty integration; an email and Slack integration; or a PagerDuty integration, a Slack integration, and two email integrations. You can also configure a contact point with no integrations; in which case no notifications are sent.
|
||||||
|
|
||||||
Each contact point integration can also define the notification message to be sent, which can use the predefined message, a custom message, or notification templates.
|
Each contact point integration can also define the notification message to be sent, which can use the predefined message, a custom message, or notification templates.
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue