当前位置：首页 > news >正文

钉钉告警+prometheus+alertmanager【prometheus-webhook-dingtalk】

news 2026/6/16 11:35:49

背景

业务需要，项目部署了prometheus作为监控，想把告警接入企业钉钉

现有环境

prometheus+node_exporter+redis_exporter+blackbox_exporter+mysql_expoort+alertmanager

钉钉告警程序包(prometheus-webhook-dingtalk)

gitlab

https://github.com/timonwong/prometheus-webhook-dingtalk

下载地址

https://github.com/timonwong/prometheus-webhook-dingtalk/releases/download/v2.1.0/prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz

config.yml

#web:
#  listen_address: ":8060"
## Request timeout
# timeout: 5s## Uncomment following line in order to write template from scratch (be careful!)
#no_builtin_template: true## Customizable templates path
templates:- /apps/apps/prometheus-webhook-dingtalk/webhook.tmpl## You can also override default template using `default_message`
## The following example to use the 'legacy' template from v0.3.0
#default_message:
#  title: '{{ template "legacy.title" . }}'
#  text: '{{ template "legacy.content" . }}'## Targets, previously was known as "profiles"
targets:webhook1:url: https://oapi.dingtalk.com/robot/send?access_token=667xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx# secret for signaturesecret: SEC8e0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
#  webhook2:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#  webhook_legacy:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    # Customize template content
#    message:
#      # Use legacy template
#      title: '{{ template "legacy.title" . }}'
#      text: '{{ template "legacy.content" . }}'
#  webhook_mention_all:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    mention:
#      all: true
#  webhook_mention_users:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#    mention:
#      mobiles: ['156xxxx8827', '189xxxx8325']

prometheus-webhook-dingtalk.service

[Unit]
Description=Prometheus Webhook DingTalk
After=network.target[Service]
Type=simple
User=prometheus
Group=prometheus# 代理配置
Environment="HTTP_PROXY=http://10.0.0.100:3128"
Environment="HTTPS_PROXY=http://10.0.0.100:3128"# 启动命令 - 使用配置文件
ExecStart=/app/prometheus-webhook-dingtalk/prometheus-webhook-dingtalk \--config.file=/app/prometheus-webhook-dingtalk/prometheus-webhook-dingtalk/config.yml \--web.listen-address=:8060Restart=always
RestartSec=10[Install]
WantedBy=multi-user.target
##webhook.tmpl

webhook.tmpl

{{ define "__subject" }}[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}{{ end }}
{{ define "__alertmanagerURL" }}{{ $alertURL := "http://192.168.64.158:9093" }} {{- $alertURL -}}/#/alerts?receiver={{ .Receiver }}&tmp={{ .ExternalURL }}{{ end }}{{ define "__text_alert_list" }}{{ range . }}
**Labels**
{{ range .Labels.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Annotations**
{{ range .Annotations.SortedPairs }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}
**Source:** [{{ .GeneratorURL }}]({{ .GeneratorURL }})
{{ end }}{{ end }}{{/* Firing */}}{{ define "default.__text_alert_list" }}{{ range . }}**触发时间:** {{ dateInZone "2006.01.02 15:04:05" (.StartsAt) "Asia/Shanghai" }}**摘要:** {{ .Annotations.summary }}**描述:** {{ .Annotations.description }}**监控:** [grafana](http://localhost:3000)**详情:**
{{ range .Labels.SortedPairs }}{{ if and (ne (.Name) "severity") (ne (.Name) "summary") }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}{{ end }}
{{ end }}{{ end }}{{/* Resolved */}}{{ define "default.__text_resolved_list" }}{{ range . }}**触发时间:** {{ dateInZone "2006.01.02 15:04:05" (.StartsAt) "Asia/Shanghai" }}**解除时间:** {{ dateInZone "2006.01.02 15:04:05" (.EndsAt) "Asia/Shanghai" }}**摘要:** {{ .Annotations.summary }}**监控:** [grafana](http://localhost:3000)**详情:**
{{ range .Labels.SortedPairs }}{{ if and (ne (.Name) "severity") (ne (.Name) "summary") }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}{{ end }}
{{ end }}{{ end }}{{/* Default */}}
{{ define "default.title" }}{{ template "__subject" . }}{{ end }}
{{ define "default.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ if gt (len .Alerts.Firing) 0 -}}![Firing-img](https://pic.rmb.bdstatic.com/bjh/news/10977507ae0edad0c17c820ff5d80f5e.png)**告警通知**
{{ template "default.__text_alert_list" .Alerts.Firing }}
{{- end }}
{{ if gt (len .Alerts.Resolved) 0 -}}![Resolved-img](https://pic.616pic.com/ys_img/00/06/20/64dXxVfv6k.jpg)**告警解除**
{{ template "default.__text_resolved_list" .Alerts.Resolved }}
{{- end }}
{{- end }}{{/* Legacy */}}
{{ define "legacy.title" }}{{ template "__subject" . }}{{ end }}
{{ define "legacy.content" }}#### \[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}\] **[{{ index .GroupLabels "alertname" }}]({{ template "__alertmanagerURL" . }})**
{{ template "__text_alert_list" .Alerts.Firing }}
{{- end }}{{/* Following names for compatibility */}}
{{ define "ding.link.title" }}{{ template "default.title" . }}{{ end }}
{{ define "ding.link.content" }}{{ template "default.content" . }}{{ end }}

创建钉钉建群并创建机器人

获取Webhook地址和加签的一串秘钥

配置config.yml和prometheus-webhook-dingtalk.service，并启动

配置alertmanager并启动

global:resolve_timeout: 5m
templates:- templates/wechat.tmpl
route:group_by: ['alertname']group_wait: 10sgroup_interval: 10srepeat_interval: 10mreceiver: 'webhook'
receivers:
- name: 'webhook'webhook_configs:- url: http://192.168.60.185:8060/dingtalk/webhook1/sendsend_resolved: true
inhibit_rules:
- equal: ['alertname', 'cluster', 'service']source_match:severity: 'high'target_match:severity: 'warning'