Skip to content

Blackbox Exporter: check the site and the TLS certificate expiry

Monitoring · HTTP · TLS
Not studied

Added 2026-10-11. Requires Prometheus, Blackbox Exporter and working alert delivery.

Problem

The container is running, but the domain doesn't open because of DNS, Nginx or an expired certificate. An internal container check doesn't cover the whole path to the public site.

Solution

The blackbox exporter "allows blackbox probing of endpoints over HTTP, HTTPS, DNS, TCP, ICMP and gRPC". Probe the HTTPS address with it and watch three things at once: the probe result, the availability of the exporter itself, and the certificate expiry. An external vantage point is needed separately: a probe from the same VPS doesn't prove the site is reachable from other networks.

Example

blackbox.yml — HTTP 200, valid TLS and a characteristic piece of site text:

modules:
  devops_kb_https:
    prober: http
    timeout: 5s
    http:
      valid_status_codes: [200]
      fail_if_not_ssl: true
      fail_if_body_not_matches_regexp: ['devops-kb']
      tls_config:
        insecure_skip_verify: false

Run it on the host for testing:

blackbox_exporter --config.file=blackbox.yml \
  --web.listen-address=127.0.0.1:9115

Add to the existing Prometheus scrape_configs. Here Prometheus and the exporter run on the same host. If Prometheus runs in a container, use an exporter address reachable from it: 127.0.0.1 inside a container means the container itself.

scrape_configs:
  - job_name: blackbox-devops-kb
    scrape_interval: 30s
    scrape_timeout: 10s
    metrics_path: /probe
    params:
      module: [devops_kb_https]
    static_configs:
      - targets: ['https://devops-kb.is-a.dev']
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 127.0.0.1:9115

A rules file loaded via Prometheus rule_files:

groups:
  - name: devops-kb-external
    rules:
      - alert: DevopsKbProbeFailed
        expr: probe_success{job="blackbox-devops-kb"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: 'The devops-kb probe is failing'
      - alert: DevopsKbProbeUnavailable
        expr: up{job="blackbox-devops-kb"} == 0 or absent(up{job="blackbox-devops-kb"})
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: 'Prometheus is not receiving probe results'
      - alert: DevopsKbCertificateExpiring
        expr: (probe_ssl_earliest_cert_expiry{job="blackbox-devops-kb"} - time()) / 86400 < 14
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: 'The devops-kb certificate expires in less than 14 days'

These rules won't detect Prometheus itself stopping. Check its availability and the notification channel with an independent monitor. For a domain with both A and AAAA records, check both address families separately: one successful probe doesn't prove that every IP works.

When to use

After moving the site behind Nginx and getting a certificate. Probe the URL visitors actually use, not just the application's internal port.

Source

How to apply it yourself

  1. Validate the configuration and rules with promtool check config and promtool check rules, passing the file paths.
  2. Make a test target return 500; then make a test exporter unreachable. Get both alerts.
  3. Test the certificate expiry rule with a temporary threshold above the remaining lifetime in a test rule set.
  4. Confirm alerts are delivered and resolved, then mark it "Studied".