Monitoring & Alerts API

Create and manage monitors and notification channels through the API. All endpoints accept YAML payloads (Content-Type: application/yaml).

Permissions

Access to monitors and channels is controlled by resource patterns in your role:

Version: 2025-01-01
Notifications:
  Monitor:
    - Actions: [read, write, delete, execute]
      Resources: ["team-monitor-*", "shared-*"]
  Channel:
    - Actions: [read, write, delete]
      Resources: ["team-channel-*", "shared-*"]

• Use * for all resources
• Use prefixes like team-* for scoped access
• Specify exact IDs for individual resources

Notification Channels

Create Channel

POST /alert

config:
  name: oncall-webhook
  description: Production webhook
  channel_type: webhook
  is_enabled: true
  webhook:
    url: https://hooks.example.com/test-webhook

Optional: specify channel_id at top level for pattern-based permissions.

Response:

{
  "channel_id": "a1b2c3d4-...",
  "config": {
    "name": "oncall-webhook",
    "description": "Production webhook",
    "channel_type": "webhook",
    "is_enabled": true,
    "webhook": {
      "url": "https://hooks.example.com/test-webhook"
    }
  }
}

Other Channel Endpoints

• GET /alert/{channel_id} - Get channel
• GET /alerts - List channels (filtered by user permissions)
• PATCH /alert/{channel_id} - Update channel
• DELETE /alert/{channel_id} - Delete channel

Monitors

Create Monitor

POST /monitor

_id: team-monitor-errors  # Optional, auto-generates UUID if omitted
name: Error Monitor
type: monitor
monitor_type: native
enabled: true
schedule:
  period:
    interval: 1
    unit: MINUTES
query:
  type: querydsl
  datasets:
    - logs-prod
  body:
    size: 1
    query:
      match:
        level: "error"
triggers:
  - name: email-trigger
    severity: 1
    condition:
      type: field_contains
      path: "/hits/hits/0/_source/message"
      value: "error"
    actions:
      - id: notify-action
        name: send-email
        destination_id: team-channel-oncall
        subject_template:
          source: "Error Alert from {{monitor.name}}"
          lang: mustache
        message_template:
          source: "Monitor {{monitor.name}} fired at {{timestamp}}"
          lang: mustache

Response:

{
  "_id": "team-monitor-errors",
  "_version": 1,
  "monitor": {
    "_id": "team-monitor-errors",
    "monitor_id": "team-monitor-errors",
    "name": "Error Monitor",
    "type": "monitor",
    "monitor_type": "native",
    "enabled": true,
    "schedule": { ... },
    "query": { ... },
    "triggers": [ ... ]
  }
}

Other Monitor Endpoints

• GET /monitor/{monitor_id} - Get monitor
• GET /monitors - List monitors (filtered by user permissions)
• PATCH /monitor/{monitor_id} - Update monitor
• DELETE /monitor/{monitor_id} - Delete monitor
• POST /monitor/{monitor_id}/execute - Run monitor now

Execute Monitor

POST /monitor/{monitor_id}/execute

trigger_ids:
  - email-trigger

Body is optional - omit trigger_ids to run all triggers.

Query Types

QueryDSL

query:
  type: querydsl
  datasets:
    - logs-prod
  body:
    size: 1
    query:
      match:
        level: "error"

SQL

query:
  type: sql
  statement: |
    SELECT COUNT(*) AS total_count FROM "metrics"
    WHERE cpu > 80

PromQL

query:
  type: promql
  index: prometheus-metrics
  query: 'cpu_usage{host="server1"}'
  start: 1640000000000
  end: 1640001000000
  step: 60

Fino AI

query:
  type: fino
  datasets:
    - logs-prod
  body:
    query: "Show me a summary of log levels in the data"

Trigger Conditions

Script-based

condition:
  script:
    source: "ctx.results[0].hits.total.value > 10"

Structured Conditions

Count comparison:

condition:
  type: count_greater_than
  threshold: 0

Field contains (uses JSON pointer to check query result fields):

condition:
  type: field_contains
  path: "/hits/hits/0/_source/message"
  value: "error"

Other types: count_less_than, count_equals, field_equals, field_not_equals, field_starts_with, field_ends_with, field_matches, field_greater_than, field_less_than, always

Scheduling

Period-based

schedule:
  period:
    interval: 5
    unit: MINUTES  # MINUTES, HOURS, DAYS

Cron-based

schedule:
  cron: "0 0 * * * *"  # Every hour

Common cron expressions:

• @hourly, @daily, @weekly - Shortcuts
• */30 * * * * * - Every 30 seconds
• 0 0 9 * * MON-FRI - Weekdays at 9am

Bulk Configuration

POST /alerts/config

Upload multiple channels and monitors at once:

channels:
  - type: notification_channel
    name: ops-email
    channel_type: email
    email:
      smtp_account_id: 3defe4bc-3799-4f6e-aa16-467d1c0cd97f
      recipients:
        - ops@example.com

monitors:
  - name: hourly-errors
    monitor_type: native
    enabled: true
    schedule:
      cron: "0 0 * * * *"
    query:
      type: querydsl
      datasets: ["logs-prod-*"]
      body:
        query:
          match: { "log.level": "ERROR" }
    triggers:
      - name: send-email
        condition:
          type: count_greater_than
          threshold: 0
        actions:
          - name: email-action
            destination_id: ops-email

Existing IDs are updated; new ones are created.