Skip to content

Prometheus Alertmanager

Prometheus Alertmanager can send alerts directly to OpsFusion using its native webhook format — no code changes needed on your end.

See API Overview for authentication and finding your Account ID/Team ID. See Send Alerts if you’re integrating a non-Prometheus tool instead.

Add the following to your alertmanager.yml:

receivers:
- name: 'opsfusion'
webhook_configs:
- url: 'https://api.opsfusion.cloud/v1/account/YOUR_ACCOUNT_ID/push/alertmanager'
send_resolved: true
max_alerts: 25
http_config:
authorization:
type: Bearer
credentials_file: /etc/alertmanager/opsfusion-apikey.txt
# Optional: Set the default team and severity
http_headers:
ofteamid:
values: ["YOUR_TEAM_ID"]
ofseverity:
values: ["2"]
route:
receiver: 'opsfusion'
group_wait: 10s
group_interval: 10s
repeat_interval: 1h

OpsFusion supports flexible alert routing using a priority chain system. You can set team ID and severity at multiple levels, and OpsFusion will use the most specific value available.

For Team ID:

  1. Per-alert annotation (highest priority) - ofteamid in alert annotations
  2. Webhook header - ofteamid header in Alertmanager config
  3. Default Team ID (fallback) - Falls back to your account’s default team ID if no team is specified

For Severity:

  1. Per-alert annotation (highest priority) - ofseverity in alert annotations
  2. Webhook header - ofseverity header in Alertmanager config
  3. Default (fallback) - Defaults to 3 (info) if not specified

Severity can be set using either numeric or text values:

NumericTextLevelWhen to Use
"1""critical"CriticalService down, data loss, security breach
"2""warning"HighDegraded performance, approaching limits
"3""info"Medium/LowWarning conditions, informational

Option 1: Per-Alert Control

Set severity and team in each Prometheus alert rule for maximum flexibility:

groups:
- name: production-alerts
rules:
- alert: HighCPU
expr: cpu_usage > 90
for: 5m
annotations:
ofteamid: "1234567890123" # Backend team
ofseverity: "critical" # or use "1"
summary: "High CPU usage on {{ $labels.instance }}"
description: "CPU usage is {{ $value }}% for 5 minutes"
- alert: HighMemory
expr: memory_usage > 85
for: 10m
annotations:
ofteamid: "9876543210987" # Platform team
ofseverity: "warning" # or use "2"
summary: "High memory usage on {{ $labels.instance }}"
description: "Memory usage is {{ $value }}%"

Option 2: Webhook-Level Defaults

Set team and severity at the webhook level when all alerts should go to the same team:

receivers:
- name: 'opsfusion-backend'
webhook_configs:
- url: 'https://api.opsfusion.cloud/v1/account/YOUR_ACCOUNT_ID/push/alertmanager'
http_config:
headers:
ofteamid: "1234567890123"
ofseverity: "2"

Then use minimal alert annotations:

groups:
- name: example
rules:
- alert: HighCPU
expr: cpu_usage > 90
for: 5m
annotations:
summary: "High CPU usage on {{ $labels.instance }}"
description: "CPU usage is {{ $value }}% for 5 minutes"
# teamId and severity inherited from webhook headers

Option 3: Mixed Approach

Set defaults at webhook level, override for specific alerts:

# Alertmanager webhook with defaults
receivers:
- name: 'opsfusion'
webhook_configs:
- url: 'https://api.opsfusion.cloud/v1/account/YOUR_ACCOUNT_ID/push/alertmanager'
http_config:
headers:
ofteamid: "1234567890123" # Default team
ofseverity: "2" # Default to warning
# Alert rules - override when needed
groups:
- name: alerts
rules:
- alert: ServiceDown
expr: up == 0
annotations:
ofseverity: "critical" # Override default severity
summary: "Service {{ $labels.job }} is down"
description: "Instance {{ $labels.instance }} is unreachable"
- alert: HighLatency
annotations:
summary: "High latency detected"
description: "P95 latency is {{ $value }}ms"
# Uses webhook defaults (team + warning severity)

Option 4: Dynamic Severity from the Alert Value

ofseverity is templated the same way summary/description already are, so it doesn’t have to be a static string per rule — it can be computed from $value at evaluation time, using the same Go template conditionals Prometheus supports everywhere else in annotations:

groups:
- name: production-alerts
rules:
- alert: HighCPU
expr: cpu_usage > 80
for: 5m
annotations:
ofseverity: '{{ if gt $value 95.0 }}critical{{ else if gt $value 85.0 }}warning{{ else }}info{{ end }}'
summary: "High CPU usage on {{ $labels.instance }}"
description: "CPU usage is {{ $value }}% for 5 minutes"

This sends critical above 95%, warning above 85%, and info otherwise — all from one alert rule, without needing separate rules per severity threshold.

Send alerts in Prometheus Alertmanager webhook format.

https://api.opsfusion.cloud/v1/account/{accountId}/push/alertmanager

Replace {accountId} with your OpsFusion account ID.

All requests must include authentication in the Authorization header.

Authorization: YOUR_API_KEY

See Create an API Key for instructions on generating API keys.

ParameterRequiredDescription
accountIdYesYour OpsFusion account ID
HeaderRequiredValueDescription
AuthorizationYesYOUR_API_KEYYour OpsFusion API key
Content-TypeYesapplication/jsonRequest content type
ofteamidNo13-digit team IDDefault team for all alerts (see Priority Chain)
ofseverityNo"1", "2", "3", "critical", "warning", or "info"Default severity for all alerts (see Priority Chain)

The request body follows the Prometheus Alertmanager webhook format.

FieldTypeRequiredDescription
alertsarrayYesArray of alert objects (1-25 alerts)
FieldTypeRequiredDescription
statusstringYes"firing" or "resolved"
labelsobjectYesAlert labels (max 50 labels, key 1-64 characters, value 1-128 characters)
annotationsobjectYesAlert annotations containing required fields
FieldTypeRequiredDescriptionConstraints
ofteamidstringNoTeam ID for this alert (overrides header)Exactly 13 digits
ofseveritystringNoSeverity level (overrides header)"1", "2", "3", "critical", "warning", or "info"
summarystringYesBrief alert summary1-200 characters
descriptionstringYesDetailed description1-2048 characters
  • Alerts array: 1-25 alerts per request
  • Summary: Automatically truncated to 200 characters
  • Description: Automatically truncated to 2048 characters
  • Status conversion: "firing" → status 1, "resolved" → status 3
  • Labels - Max 50 key-value pairs, each key 1-64 characters, and each value 1-128 characters

Example 1: With per-alert team and severity

Terminal window
curl -X POST 'https://api.opsfusion.cloud/v1/account/YOUR_ACCOUNT_ID/push/alertmanager' \
-H 'Authorization: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "HighCPU",
"instance": "web-server-01",
"job": "node-exporter"
},
"annotations": {
"ofteamid": "1234567890123",
"ofseverity": "critical",
"summary": "High CPU usage detected",
"description": "CPU usage is above 90% for 5 minutes"
}
}
]
}'

Example 2: With webhook headers (no team/severity in annotations)

Terminal window
curl -X POST 'https://api.opsfusion.cloud/v1/account/YOUR_ACCOUNT_ID/push/alertmanager' \
-H 'Authorization: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-H 'ofteamid: 1234567890123' \
-H 'ofseverity: 2' \
-d '{
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "HighCPU",
"instance": "web-server-01"
},
"annotations": {
"summary": "High CPU usage detected",
"description": "CPU usage is above 90% for 5 minutes"
}
}
]
}'

Response codes and error handling are the same as the Standard endpoint — see Send Alerts.

400 Bad Request

  • Verify JSON is properly formatted
  • Check that all required fields are present
  • Ensure field values meet constraints (e.g., ofteamid is 13 digits)
  • Verify ofseverity uses an allowed value

403 Forbidden

  • Check that your API key is valid and not expired
  • Verify you’re using the correct Authorization header format

404 Not Found

  • Verify the account ID in the URL path is correct

For integration support: