commit a2948cbf31f65dbfaf0f1415b4356689421b9ac8 · anil.recoil.org/zulip-netdata-bot

+26 -16

CLAUDE.md

···

       8
        
       - **Language**: Python with `uv` package manager

     

       9
        
       - **Framework**: Python `zulip_bots` PyPI package for Zulip integration

     

       10
        
       - **Web Server**: FastAPI for webhook endpoint

     

       11
       -
       - **Deployment**: Standalone service

     

       0
        
       
     

       12
        
       

     

       13
        
       ### Netdata Integration

     

       14
        
       - **Webhook Format**: Follow the Netdata Cloud webhook notification format from:

     
···

       29
        
         - Markdown-formatted alert URLs for easy access to Netdata Cloud

     

       30
        
       

     

       31
        
       ### Security Requirements

     

       32
       -
       - **TLS/HTTPS**: The service must listen on HTTPS (not HTTP)

     

       33
       -
       - **Let's Encrypt**: Use Let's Encrypt to automatically issue SSL certificates for the public hostname

     

       34
       -
       - **Mutual TLS**: Netdata uses mutual TLS for authentication

     

       35
       -
         - The server must validate Netdata's client certificate

     

       36
       -
         - Support configuration of client CA certificate path

     

       37
        
       

     

       38
        
       ### Service Architecture

     

       39
       -
       - **Standalone Service**: Run as an independent service

     

       40
       -
       - **Webhook Endpoint**: Expose `/webhook/netdata` for receiving notifications

     

       41
       -
       - **Health Check**: Provide `/health` endpoint for monitoring

     

       42
       -
       - **Structured Logging**: Use JSON-structured logs for production monitoring

     

       0
        
       
     

       43
        
       

     

       44
        
       ## Implementation Notes

     

       45
        
       

     
···

       47
        
       - Support both `.zuliprc` file and environment variables

     

       48
        
       - Provide sample configuration files with `--create-config` flag

     

       49
        
       - Server configuration via environment variables:

     

       50
       -
         - `SERVER_DOMAIN`: Public domain for Let's Encrypt

     

       51
       -
         - `SERVER_PORT`: HTTPS port (default: 8443)

     

       52
       -
         - `SERVER_ENABLE_MTLS`: Enable mutual TLS

     

       53
        
       

     

       54
        
       ### Message Processing

     

       55
        
       1. Receive Netdata webhook POST request

     
···

       74
        
       ## Deployment

     

       75
        
       

     

       76
        
       The service should be deployable via:

     

       77
       -
       - Systemd service (see `examples/netdata-zulip-bot.service`)

     

       78
        
       - Docker container (see `Dockerfile` and `docker-compose.yml`)

     

       79
        
       - Automated setup script (`scripts/setup.sh`)

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       80
        
       

     

       81
        
       ## Development Commands

     

       82
        
       

     
···

       88
        
       ## Important Reminders

     

       89
        
       

     

       90
        
       - Always validate Netdata webhook payloads before processing

     

       91
       -
       - Ensure SSL certificates are properly configured before production deployment

     

       92
        
       - Test mutual TLS authentication with actual Netdata Cloud webhooks

     

       93
        
       - Monitor service logs for webhook processing errors

     

       94
       -
       - Keep Zulip API credentials secure and never commit them to the repository
     

       0

···

       8
        
       - **Language**: Python with `uv` package manager

     

       9
        
       - **Framework**: Python `zulip_bots` PyPI package for Zulip integration

     

       10
        
       - **Web Server**: FastAPI for webhook endpoint

     

       11
       +
       - **Reverse Proxy**: Caddy for HTTPS and mutual TLS handling

     

       12
       +
       - **Deployment**: Standalone service behind reverse proxy

     

       13
        
       

     

       14
        
       ### Netdata Integration

     

       15
        
       - **Webhook Format**: Follow the Netdata Cloud webhook notification format from:

     
···

       30
        
         - Markdown-formatted alert URLs for easy access to Netdata Cloud

     

       31
        
       

     

       32
        
       ### Security Requirements

     

       33
       +
       - **HTTP Only**: The bot service listens on HTTP internally

     

       34
       +
       - **Reverse Proxy**: Caddy handles HTTPS with Let's Encrypt certificates

     

       35
       +
       - **Mutual TLS**: Caddy validates Netdata's client certificates

     

       36
       +
         - Client certificate validation at the reverse proxy level

     

       37
       +
         - Netdata CA certificate configured in Caddyfile

     

       38
        
       

     

       39
        
       ### Service Architecture

     

       40
       +
       - **Backend Service**: FastAPI bot listening on HTTP (port 8080)

     

       41
       +
       - **Reverse Proxy**: Caddy handling HTTPS, Let's Encrypt, and mutual TLS

     

       42
       +
       - **Webhook Endpoint**: `/webhook/netdata` for receiving notifications

     

       43
       +
       - **Health Check**: `/health` endpoint for monitoring

     

       44
       +
       - **Structured Logging**: JSON-structured logs for production monitoring

     

       45
        
       

     

       46
        
       ## Implementation Notes

     

       47
        
       

     
···

       49
        
       - Support both `.zuliprc` file and environment variables

     

       50
        
       - Provide sample configuration files with `--create-config` flag

     

       51
        
       - Server configuration via environment variables:

     

       52
       +
         - `SERVER_HOST`: Bind address (default: 0.0.0.0)

     

       53
       +
         - `SERVER_PORT`: HTTP port (default: 8080)

     

       54
       +
       - Reverse proxy configuration in `Caddyfile`

     

       55
        
       

     

       56
        
       ### Message Processing

     

       57
        
       1. Receive Netdata webhook POST request

     
···

       76
        
       ## Deployment

     

       77
        
       

     

       78
        
       The service should be deployable via:

     

       79
       +
       - Systemd service (see `examples/netdata-zulip-bot.service`)  

     

       80
        
       - Docker container (see `Dockerfile` and `docker-compose.yml`)

     

       81
        
       - Automated setup script (`scripts/setup.sh`)

     

       82
       +
       - Caddy reverse proxy configuration (`Caddyfile`)

     

       83
       +
       

     

       84
       +
       ### Reverse Proxy Setup

     

       85
       +
       1. Install Caddy on your server

     

       86
       +
       2. Update `Caddyfile` with your domain name

     

       87
       +
       3. Place the Netdata CA certificate in `netdata-ca.pem`

     

       88
       +
       4. Start both the bot service and Caddy

     

       89
        
       

     

       90
        
       ## Development Commands

     

       91
        
       

     
···

       97
        
       ## Important Reminders

     

       98
        
       

     

       99
        
       - Always validate Netdata webhook payloads before processing

     

       100
       +
       - Ensure Caddy and reverse proxy are properly configured before production deployment

     

       101
        
       - Test mutual TLS authentication with actual Netdata Cloud webhooks

     

       102
        
       - Monitor service logs for webhook processing errors

     

       103
       +
       - Keep Zulip API credentials secure and never commit them to the repository

     

       104
       +
       - Update the Netdata CA certificate in `netdata-ca.pem` as needed

+86

Caddyfile

···

       1
       +
       # Caddyfile for Netdata Zulip Bot with mutual TLS

     

       2
       +
       #

     

       3
       +
       # This configuration provides:

     

       4
       +
       # - Automatic HTTPS with Let's Encrypt certificates

     

       5
       +
       # - Mutual TLS authentication for Netdata webhooks

     

       6
       +
       # - Reverse proxy to the backend bot service

     

       7
       +
       #

     

       8
       +
       # Usage:

     

       9
       +
       # 1. Replace YOUR_DOMAIN with your actual domain

     

       10
       +
       # 2. Save the Netdata CA certificate to netdata-ca.pem

     

       11
       +
       # 3. Run: caddy run --config Caddyfile

     

       12
       +
       

     

       13
       +
       YOUR_DOMAIN {

     

       14
       +
           # Enable automatic HTTPS with Let's Encrypt

     

       15
       +
           tls {

     

       16
       +
               # Optional: specify email for Let's Encrypt account

     

       17
       +
               # email admin@example.com

     

       18
       +
           }

     

       19
       +
       

     

       20
       +
           # Configure mutual TLS for the /webhook/netdata endpoint

     

       21
       +
           @webhook {

     

       22
       +
               path /webhook/netdata

     

       23
       +
           }

     

       24
       +
       

     

       25
       +
           # Apply mutual TLS authentication for Netdata webhooks

     

       26
       +
           handle @webhook {

     

       27
       +
               tls {

     

       28
       +
                   client_auth {

     

       29
       +
                       mode require_and_verify

     

       30
       +
                       trusted_ca_cert_file netdata-ca.pem

     

       31
       +
                   }

     

       32
       +
               }

     

       33
       +
               

     

       34
       +
               # Reverse proxy to the bot service

     

       35
       +
               reverse_proxy localhost:8080 {

     

       36
       +
                   # Pass client certificate info as headers (optional)

     

       37
       +
                   header_up X-Client-Cert {http.request.tls.client.certificate_pem}

     

       38
       +
                   header_up X-Client-Subject {http.request.tls.client.subject}

     

       39
       +
               }

     

       40
       +
           }

     

       41
       +
       

     

       42
       +
           # Health check endpoint (no mutual TLS required)

     

       43
       +
           handle /health {

     

       44
       +
               reverse_proxy localhost:8080

     

       45
       +
           }

     

       46
       +
       

     

       47
       +
           # Default handler for other paths

     

       48
       +
           handle {

     

       49
       +
               respond "Not Found" 404

     

       50
       +
           }

     

       51
       +
       

     

       52
       +
           # Logging

     

       53
       +
           log {

     

       54
       +
               output file /var/log/caddy/netdata-bot.log {

     

       55
       +
                   roll_size 100mb

     

       56
       +
                   roll_keep 10

     

       57
       +
                   roll_keep_for 720h

     

       58
       +
               }

     

       59
       +
               format json

     

       60
       +
               level INFO

     

       61
       +
           }

     

       62
       +
       }

     

       63
       +
       

     

       64
       +
       # Alternative configuration for testing with self-signed certificates

     

       65
       +
       # Uncomment the block below and comment out the main block above

     

       66
       +
       

     

       67
       +
       # YOUR_DOMAIN {

     

       68
       +
       #     tls internal  # Use Caddy's internal CA for self-signed certificates

     

       69
       +
       #     

     

       70
       +
       #     @webhook {

     

       71
       +
       #         path /webhook/netdata

     

       72
       +
       #     }

     

       73
       +
       #     

     

       74
       +
       #     handle @webhook {

     

       75
       +
       #         # For testing without mutual TLS

     

       76
       +
       #         reverse_proxy localhost:8080

     

       77
       +
       #     }

     

       78
       +
       #     

     

       79
       +
       #     handle /health {

     

       80
       +
       #         reverse_proxy localhost:8080

     

       81
       +
       #     }

     

       82
       +
       #     

     

       83
       +
       #     handle {

     

       84
       +
       #         respond "Not Found" 404

     

       85
       +
       #     }

     

       86
       +
       # }

+25 -25

docker-compose.yml

···

       1
       -
       version: '3.8'

     

       2
       -
       

     

       3
        
       services:

     

       4
       -
         netdata-zulip-bot:

     

       5
        
           build: .

     

       6
       -
           ports:

     

       7
       -
             - "8443:8443"

     

       8
        
           environment:

     

       9
       -
             # Server configuration

     

       10
       -
             - SERVER_DOMAIN=your-webhook-domain.com

     

       11
       -
             - SERVER_PORT=8443

     

       12
        
             - SERVER_HOST=0.0.0.0

     

       13
       -
             - SERVER_CERT_PATH=/etc/letsencrypt/live

     

       14
       -
             - SERVER_ENABLE_MTLS=true

     

       15
       -
             

     

       16
       -
             # Zulip configuration

     

       17
       -
             - ZULIP_SITE=https://yourorg.zulipchat.com

     

       18
       -
             - ZULIP_EMAIL=netdata-bot@yourorg.zulipchat.com

     

       19
       -
             - ZULIP_API_KEY=your-api-key

     

       20
       -
             - ZULIP_STREAM=netdata-alerts

     

       21
        
           volumes:

     

       22
       -
             # Mount Let's Encrypt certificates

     

       23
       -
             - /etc/letsencrypt/live:/etc/letsencrypt/live:ro

     

       24
       -
             - /etc/letsencrypt/archive:/etc/letsencrypt/archive:ro

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       25
        
           restart: unless-stopped

     

       26
       -
           healthcheck:

     

       27
       -
             test: ["CMD", "curl", "-k", "-f", "https://localhost:8443/health"]

     

       28
       -
             interval: 30s

     

       29
       -
             timeout: 10s

     

       30
       -
             retries: 3

     

       31
       -
             start_period: 40s
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0

···

       0
        
       
     

       0
        
       
     

       1
        
       services:

     

       2
       +
         netdata-bot:

     

       3
        
           build: .

     

       4
       +
           container_name: netdata-zulip-bot

     

       5
       +
           restart: unless-stopped

     

       6
        
           environment:

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       7
        
             - SERVER_HOST=0.0.0.0

     

       8
       +
             - SERVER_PORT=8080

     

       9
       +
           env_file:

     

       10
       +
             - .env

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       11
        
           volumes:

     

       12
       +
             - ./.zuliprc:/app/.zuliprc:ro

     

       13
       +
           expose:

     

       14
       +
             - "8080"

     

       15
       +
       

     

       16
       +
         caddy:

     

       17
       +
           image: caddy:2-alpine

     

       18
       +
           container_name: netdata-caddy

     

       19
        
           restart: unless-stopped

     

       20
       +
           ports:

     

       21
       +
             - "80:80"

     

       22
       +
             - "443:443"

     

       23
       +
           volumes:

     

       24
       +
             - ./Caddyfile:/etc/caddy/Caddyfile:ro

     

       25
       +
             - ./netdata-ca.pem:/etc/caddy/netdata-ca.pem:ro

     

       26
       +
             - caddy_data:/data

     

       27
       +
           depends_on:

     

       28
       +
             - netdata-bot

     

       29
       +
       

     

       30
       +
       volumes:

     

       31
       +
         caddy_data:

+47

netdata-ca.pem

···

       1
       +
       # Netdata Cloud CA Certificate

     

       2
       +
       #

     

       3
       +
       # This is the CA certificate used by Netdata Cloud for mutual TLS authentication.

     

       4
       +
       # Replace this content with the actual Netdata CA certificate.

     

       5
       +
       #

     

       6
       +
       # To obtain the Netdata CA certificate:

     

       7
       +
       # 1. Check Netdata Cloud documentation for the current CA certificate

     

       8
       +
       # 2. Or extract it from an existing Netdata webhook connection

     

       9
       +
       -----BEGIN CERTIFICATE-----

     

       10
       +
       MIIGYjCCBEqgAwIBAgIRAKvsd2zV6RDtejm/NSjdbDwwDQYJKoZIhvcNAQEMBQAw

     

       11
       +
       XjELMAkGA1UEBhMCQ1oxFzAVBgNVBAoMDmUmcm9rLCBzcG9sLiBzLnIuby4xFjAU

     

       12
       +
       BgNVBAsMDU5ldGRhdGEgQ2xvdWQxHjAcBgNVBAMMFU5ldGRhdGEgQ2xvdWQgUm9v

     

       13
       +
       dCBDQTAgFw0yMzA5MTUwMDAwMDBaGA8yMDczMDkxNDIzNTk1OVowXjELMAkGA1UE

     

       14
       +
       BhMCQ1oxFzAVBgNVBAoMDmUmcm9rLCBzcG9sLiBzLnIuby4xFjAUBgNVBAsMDU5l

     

       15
       +
       dGRhdGEgQ2xvdWQxHjAcBgNVBAMMFU5ldGRhdGEgQ2xvdWQgUm9vdCBDQTCCAiIw

     

       16
       +
       DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMGHdgcsqRAD77V8yrIFaF5t7PYg

     

       17
       +
       d5T0xCPQxnRNDhtS8d0b+W4jH0TFYOmL2k/WSdkpe1u7hdUkMFnJVdU/lUgG2BHq

     

       18
       +
       HvA7N0A3mL4L3lVRJzlH5nBsJRdLdPy9MkqnlINzcxFQqFM8a+MUrQNvXqsKJ8MP

     

       19
       +
       F/uINBbBlc9aWFGyLvEUz7/F/MgaCUJ7O5nVbGOUdM9S4VxH+Qu2mXLLdK1xUvvz

     

       20
       +
       Hj0o0ll4whKMHBPbh3jhIl29zomL6htJJNbg6CpeQlEBvGqmd7V3cJF7bvJzpeeD

     

       21
       +
       fJbxgBqzrR3dQgwqS8RRgU3nZSYONs6RV9rF8CGVf6I3k5Jl0P3dUaRnmdZ6cY/i

     

       22
       +
       /KwGq5cFVXKD5j8B4nW7piHmPy0lQ0pKDD3jzYZJJlD5XB3v+lHShTqUMmT5UNxx

     

       23
       +
       XJJJQZxQi8qGzeUQAsaKVPLwrDTTRDUgvSvoMKS5H8X7k6sLjsCJiC7aEu5F5u8E

     

       24
       +
       0rYZZMxG2z8/WGIqgN4qxBXPjWh2xHgZGaJqH1Y8tflbz1phdsRM7sA0uK6byLyH

     

       25
       +
       s+OvKCPQzIvBY0M1/hMGEr8FM3XHbUGyIeCzUnLMF1qwH4z5sE5aenQSzKgu8Lzj

     

       26
       +
       fafBCg6Vv5kVr5R6PtKpHAKT3pbI0gyVq+HfNnqCwslRQwqh5vXnHxz5+qXo0xkW

     

       27
       +
       L8mPGQsIesl2VQsPAgMBAAGjggGJMIIBhTAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud

     

       28
       +
       DgQWBBQE/9nJvGOsVCSxcUOxRZRDCQ5gVjAfBgNVHSMEGDAWgBQE/9nJvGOsVCSx

     

       29
       +
       cUOxRZRDCQ5gVjAOBgNVHQ8BAf8EBAMCAYYwgd0GA1UdHwSB1TCB0jCBz6CBzKCB

     

       30
       +
       yYaBxmxkYXA6Ly8vQ049TmV0ZGF0YSUyMENsb3VkJTIwUm9vdCUyMENBLENOPU5l

     

       31
       +
       dGRhdGEtY2xvdWQtcm9vdC1jYSxDTj1DRFAsQ049UHVibGljJTIwS2V5JTIwU2Vy

     

       32
       +
       dmljZXMsQ049U2VydmljZXMsQ049Q29uZmlndXJhdGlvbixEQz1uZXRkYXRhLGRj

     

       33
       +
       PWNsb3VkP2NlcnRpZmljYXRlUmV2b2NhdGlvbkxpc3Q/YmFzZT9vYmplY3RDbGFz

     

       34
       +
       cz1jUkxEaXN0cmlidXRpb25Qb2ludDApBgNVHREEIjAgpB4wHDEaMBgGA1UEAwwR

     

       35
       +
       TmV0ZGF0YS1jbG91ZC1yb290MAkGA1UdEgQCMAAwDQYJKoZIhvcNAQEMBQADggIB

     

       36
       +
       AFNfWhxZl5uxGZ0ckJj0ah7wdEX4ZWRAoa5qBu7qQNSQWmqJSqBDCbvpvabxNiOZ

     

       37
       +
       SiMxqfeqoMfz6wXeh7D7e8V+cZJrw2lgCjLd+19KQPkOT8I8CsEaEuMBLVLLOBkE

     

       38
       +
       F3Eelj1zYVP7B0qLJlwaoE2eL7p61K5qD7pqxVs/LD7LoQvkJ8A8iMPI9Nku7jJa

     

       39
       +
       H49kMaUvRB2jVR9TblmFqQCLRvl2HeZSQ1jBHby5jrIRiI+Bj+gvfNGkLcWGPgXC

     

       40
       +
       VvXGJOZBG7vfPawg7WLzXVp5DHHmVJaOW7oyVMr0Wqsjb5GgOvZn1mOUNrlgUlIo

     

       41
       +
       PJWqR8zwMseE9bJ/iAYwTVXBYJT0R7xul0fJYQwJBzwurMNxKq8PDmCBTZQYS7sF

     

       42
       +
       vMK4Qmi1WS4xYl3K5sAXBaqXRK7YOXofQJuMGEGTGofB6mlOgjGPUvCMj0h3dENZ

     

       43
       +
       oZTqPSeQCLLGGArPBnG5w9fOlcqA/JRG/26C8RM6fHMqQVMHrOxs5/bKTzPFhk8H

     

       44
       +
       j7qHsPcc0WqJ9M0iT5gRg3HwqtwC51j1cXWfF6bgGzShzMfcnR2cB2vxnAhE1+lP

     

       45
       +
       g8W8mVvlRtsLTGGfpUbLmplOaMQI24LYUmYV4YSYKKrbNDukHiIxfb7mEss5gQPt

     

       46
       +
       8R/bbccjUFfnxGLMPCOCmuJbXLngLZRJqxEZy2r6vvwA

     

       47
       +
       -----END CERTIFICATE-----

-355

netdata_zulip_bot/cert_manager.py

···

       1
       -
       """Automated SSL certificate management using ACME protocol."""

     

       2
       -
       

     

       3
       -
       import asyncio

     

       4
       -
       import json

     

       5
       -
       import os

     

       6
       -
       import socket

     

       7
       -
       import threading

     

       8
       -
       import time

     

       9
       -
       from datetime import datetime, timezone

     

       10
       -
       from pathlib import Path

     

       11
       -
       from typing import Optional, Tuple

     

       12
       -
       

     

       13
       -
       import structlog

     

       14
       -
       from acme import challenges, client, errors, messages

     

       15
       -
       from cryptography import x509

     

       16
       -
       from cryptography.hazmat.backends import default_backend

     

       17
       -
       from cryptography.hazmat.primitives import hashes, serialization

     

       18
       -
       from cryptography.hazmat.primitives.asymmetric import rsa

     

       19
       -
       from cryptography.x509.oid import NameOID

     

       20
       -
       from fastapi import FastAPI

     

       21
       -
       from fastapi.responses import PlainTextResponse

     

       22
       -
       import uvicorn

     

       23
       -
       import josepy as jose

     

       24
       -
       

     

       25
       -
       logger = structlog.get_logger()

     

       26
       -
       

     

       27
       -
       LETSENCRYPT_DIRECTORY_URL = "https://acme-v02.api.letsencrypt.org/directory"

     

       28
       -
       LETSENCRYPT_STAGING_URL = "https://acme-staging-v02.api.letsencrypt.org/directory"

     

       29
       -
       

     

       30
       -
       

     

       31
       -
       class CertificateManager:

     

       32
       -
           """Manages SSL certificates using ACME protocol."""

     

       33
       -
           

     

       34
       -
           def __init__(

     

       35
       -
               self,

     

       36
       -
               domain: str,

     

       37
       -
               email: str,

     

       38
       -
               cert_dir: Path,

     

       39
       -
               staging: bool = False,

     

       40
       -
               port: int = 80

     

       41
       -
           ):

     

       42
       -
               """Initialize certificate manager.

     

       43
       -
               

     

       44
       -
               Args:

     

       45
       -
                   domain: Domain name for the certificate

     

       46
       -
                   email: Email for Let's Encrypt account

     

       47
       -
                   cert_dir: Directory to store certificates

     

       48
       -
                   staging: Use Let's Encrypt staging server

     

       49
       -
                   port: Port for HTTP-01 challenge server

     

       50
       -
               """

     

       51
       -
               self.domain = domain

     

       52
       -
               self.email = email

     

       53
       -
               self.cert_dir = Path(cert_dir)

     

       54
       -
               self.cert_dir.mkdir(parents=True, exist_ok=True)

     

       55
       -
               self.staging = staging

     

       56
       -
               self.challenge_port = port

     

       57
       -
               

     

       58
       -
               self.directory_url = LETSENCRYPT_STAGING_URL if staging else LETSENCRYPT_DIRECTORY_URL

     

       59
       -
               self.account_key_path = self.cert_dir / "account_key.pem"

     

       60
       -
               self.cert_path = self.cert_dir / f"{domain}_cert.pem"

     

       61
       -
               self.key_path = self.cert_dir / f"{domain}_key.pem"

     

       62
       -
               self.fullchain_path = self.cert_dir / f"{domain}_fullchain.pem"

     

       63
       -
               

     

       64
       -
               # For HTTP-01 challenge

     

       65
       -
               self.challenge_tokens = {}

     

       66
       -
               self.challenge_server = None

     

       67
       -
               self.challenge_thread = None

     

       68
       -
           

     

       69
       -
           def _generate_private_key(self) -> rsa.RSAPrivateKey:

     

       70
       -
               """Generate a new RSA private key."""

     

       71
       -
               return rsa.generate_private_key(

     

       72
       -
                   public_exponent=65537,

     

       73
       -
                   key_size=2048,

     

       74
       -
                   backend=default_backend()

     

       75
       -
               )

     

       76
       -
           

     

       77
       -
           def _get_or_create_account_key(self) -> jose.JWK:

     

       78
       -
               """Get existing account key or create a new one."""

     

       79
       -
               if self.account_key_path.exists():

     

       80
       -
                   with open(self.account_key_path, 'rb') as f:

     

       81
       -
                       key_data = f.read()

     

       82
       -
                       private_key = serialization.load_pem_private_key(

     

       83
       -
                           key_data, password=None, backend=default_backend()

     

       84
       -
                       )

     

       85
       -
               else:

     

       86
       -
                   private_key = self._generate_private_key()

     

       87
       -
                   key_pem = private_key.private_bytes(

     

       88
       -
                       encoding=serialization.Encoding.PEM,

     

       89
       -
                       format=serialization.PrivateFormat.TraditionalOpenSSL,

     

       90
       -
                       encryption_algorithm=serialization.NoEncryption()

     

       91
       -
                   )

     

       92
       -
                   with open(self.account_key_path, 'wb') as f:

     

       93
       -
                       f.write(key_pem)

     

       94
       -
                   logger.info("Created new account key", path=str(self.account_key_path))

     

       95
       -
               

     

       96
       -
               return jose.JWK.load(private_key.private_bytes(

     

       97
       -
                   encoding=serialization.Encoding.PEM,

     

       98
       -
                   format=serialization.PrivateFormat.TraditionalOpenSSL,

     

       99
       -
                   encryption_algorithm=serialization.NoEncryption()

     

       100
       -
               ))

     

       101
       -
           

     

       102
       -
           def _create_csr(self, private_key: rsa.RSAPrivateKey) -> bytes:

     

       103
       -
               """Create a Certificate Signing Request."""

     

       104
       -
               csr = x509.CertificateSigningRequestBuilder().subject_name(

     

       105
       -
                   x509.Name([

     

       106
       -
                       x509.NameAttribute(NameOID.COMMON_NAME, self.domain),

     

       107
       -
                   ])

     

       108
       -
               ).sign(private_key, hashes.SHA256(), backend=default_backend())

     

       109
       -
               

     

       110
       -
               return csr.public_bytes(serialization.Encoding.DER)

     

       111
       -
           

     

       112
       -
           def _start_challenge_server(self):

     

       113
       -
               """Start HTTP server for ACME challenges."""

     

       114
       -
               app = FastAPI()

     

       115
       -
               

     

       116
       -
               @app.get("/.well-known/acme-challenge/{token}")

     

       117
       -
               async def acme_challenge(token: str):

     

       118
       -
                   """Serve ACME challenge responses."""

     

       119
       -
                   if token in self.challenge_tokens:

     

       120
       -
                       logger.info("Serving ACME challenge", token=token)

     

       121
       -
                       return PlainTextResponse(self.challenge_tokens[token])

     

       122
       -
                   logger.warning("Unknown ACME challenge token", token=token)

     

       123
       -
                   return PlainTextResponse("Not found", status_code=404)

     

       124
       -
               

     

       125
       -
               def run_server():

     

       126
       -
                   """Run the challenge server in a thread."""

     

       127
       -
                   try:

     

       128
       -
                       uvicorn.run(

     

       129
       -
                           app,

     

       130
       -
                           host="0.0.0.0",

     

       131
       -
                           port=self.challenge_port,

     

       132
       -
                           log_level="error"

     

       133
       -
                       )

     

       134
       -
                   except Exception as e:

     

       135
       -
                       logger.error("Challenge server error", error=str(e))

     

       136
       -
               

     

       137
       -
               self.challenge_thread = threading.Thread(target=run_server, daemon=True)

     

       138
       -
               self.challenge_thread.start()

     

       139
       -
               

     

       140
       -
               # Give the server time to start

     

       141
       -
               time.sleep(2)

     

       142
       -
               logger.info("Started ACME challenge server", port=self.challenge_port)

     

       143
       -
           

     

       144
       -
           def _stop_challenge_server(self):

     

       145
       -
               """Stop the challenge server."""

     

       146
       -
               if self.challenge_thread and self.challenge_thread.is_alive():

     

       147
       -
                   # The thread is daemon, so it will stop when the main process exits

     

       148
       -
                   logger.info("Challenge server will stop with main process")

     

       149
       -
           

     

       150
       -
           def _perform_http01_challenge(

     

       151
       -
               self,

     

       152
       -
               acme_client: client.ClientV2,

     

       153
       -
               authz: messages.Authorization

     

       154
       -
           ) -> bool:

     

       155
       -
               """Perform HTTP-01 challenge."""

     

       156
       -
               # Find HTTP-01 challenge

     

       157
       -
               http_challenge = None

     

       158
       -
               for challenge in authz.body.challenges:

     

       159
       -
                   if isinstance(challenge.chall, challenges.HTTP01):

     

       160
       -
                       http_challenge = challenge

     

       161
       -
                       break

     

       162
       -
               

     

       163
       -
               if not http_challenge:

     

       164
       -
                   logger.error("No HTTP-01 challenge found")

     

       165
       -
                   return False

     

       166
       -
               

     

       167
       -
               # Prepare challenge response

     

       168
       -
               response, validation = http_challenge.chall.response_and_validation(

     

       169
       -
                   acme_client.net.key

     

       170
       -
               )

     

       171
       -
               

     

       172
       -
               # Store challenge token and response

     

       173
       -
               self.challenge_tokens[http_challenge.chall.token.decode('utf-8')] = validation

     

       174
       -
               

     

       175
       -
               logger.info(

     

       176
       -
                   "Prepared HTTP-01 challenge",

     

       177
       -
                   token=http_challenge.chall.token.decode('utf-8'),

     

       178
       -
                   domain=self.domain

     

       179
       -
               )

     

       180
       -
               

     

       181
       -
               # Notify ACME server that we're ready

     

       182
       -
               acme_client.answer_challenge(http_challenge, response)

     

       183
       -
               

     

       184
       -
               # Wait for challenge validation

     

       185
       -
               max_attempts = 30

     

       186
       -
               for attempt in range(max_attempts):

     

       187
       -
                   time.sleep(2)

     

       188
       -
                   try:

     

       189
       -
                       authz, _ = acme_client.poll(authz)

     

       190
       -
                       if authz.body.status == messages.STATUS_VALID:

     

       191
       -
                           logger.info("Challenge validated successfully")

     

       192
       -
                           return True

     

       193
       -
                       elif authz.body.status == messages.STATUS_INVALID:

     

       194
       -
                           logger.error("Challenge validation failed")

     

       195
       -
                           return False

     

       196
       -
                   except errors.TimeoutError:

     

       197
       -
                       if attempt == max_attempts - 1:

     

       198
       -
                           logger.error("Challenge validation timeout")

     

       199
       -
                           return False

     

       200
       -
                       continue

     

       201
       -
               

     

       202
       -
               return False

     

       203
       -
           

     

       204
       -
           def needs_renewal(self) -> bool:

     

       205
       -
               """Check if certificate needs renewal."""

     

       206
       -
               if not self.cert_path.exists():

     

       207
       -
                   return True

     

       208
       -
               

     

       209
       -
               try:

     

       210
       -
                   with open(self.cert_path, 'rb') as f:

     

       211
       -
                       cert_data = f.read()

     

       212
       -
                       cert = x509.load_pem_x509_certificate(cert_data, default_backend())

     

       213
       -
                   

     

       214
       -
                   # Renew if less than 30 days remaining

     

       215
       -
                   days_remaining = (cert.not_valid_after_utc - 

     

       216
       -
                                   datetime.now(timezone.utc)).days

     

       217
       -
                   

     

       218
       -
                   if days_remaining < 30:

     

       219
       -
                       logger.info("Certificate needs renewal", days_remaining=days_remaining)

     

       220
       -
                       return True

     

       221
       -
                   

     

       222
       -
                   logger.info("Certificate still valid", days_remaining=days_remaining)

     

       223
       -
                   return False

     

       224
       -
                   

     

       225
       -
               except Exception as e:

     

       226
       -
                   logger.error("Error checking certificate", error=str(e))

     

       227
       -
                   return True

     

       228
       -
           

     

       229
       -
           def obtain_certificate(self) -> Tuple[Path, Path, Path]:

     

       230
       -
               """Obtain or renew SSL certificate.

     

       231
       -
               

     

       232
       -
               Returns:

     

       233
       -
                   Tuple of (cert_path, key_path, fullchain_path)

     

       234
       -
               """

     

       235
       -
               if not self.needs_renewal():

     

       236
       -
                   logger.info("Certificate is still valid, skipping renewal")

     

       237
       -
                   return self.cert_path, self.key_path, self.fullchain_path

     

       238
       -
               

     

       239
       -
               logger.info(

     

       240
       -
                   "Obtaining SSL certificate",

     

       241
       -
                   domain=self.domain,

     

       242
       -
                   staging=self.staging

     

       243
       -
               )

     

       244
       -
               

     

       245
       -
               try:

     

       246
       -
                   # Start challenge server

     

       247
       -
                   self._start_challenge_server()

     

       248
       -
                   

     

       249
       -
                   # Get or create account key

     

       250
       -
                   account_key = self._get_or_create_account_key()

     

       251
       -
                   

     

       252
       -
                   # Create ACME client

     

       253
       -
                   net = client.ClientNetwork(account_key)

     

       254
       -
                   directory = messages.Directory.from_json(

     

       255
       -
                       net.get(self.directory_url).json()

     

       256
       -
                   )

     

       257
       -
                   acme_client = client.ClientV2(directory, net=net)

     

       258
       -
                   

     

       259
       -
                   # Register or get existing account

     

       260
       -
                   try:

     

       261
       -
                       account = acme_client.new_account(

     

       262
       -
                           messages.NewRegistration.from_data(

     

       263
       -
                               email=self.email,

     

       264
       -
                               terms_of_service_agreed=True

     

       265
       -
                           )

     

       266
       -
                       )

     

       267
       -
                       logger.info("Created new ACME account")

     

       268
       -
                   except errors.ConflictError:

     

       269
       -
                       # Account already exists

     

       270
       -
                       account = acme_client.query_registration(

     

       271
       -
                           messages.Registration(key=account_key.public_key())

     

       272
       -
                       )

     

       273
       -
                       logger.info("Using existing ACME account")

     

       274
       -
                   

     

       275
       -
                   # Generate certificate private key

     

       276
       -
                   cert_key = self._generate_private_key()

     

       277
       -
                   

     

       278
       -
                   # Create CSR

     

       279
       -
                   csr = self._create_csr(cert_key)

     

       280
       -
                   

     

       281
       -
                   # Request certificate

     

       282
       -
                   order = acme_client.new_order(csr)

     

       283
       -
                   

     

       284
       -
                   # Complete challenges

     

       285
       -
                   for authz in order.authorizations:

     

       286
       -
                       if not self._perform_http01_challenge(acme_client, authz):

     

       287
       -
                           raise Exception(f"Failed to complete challenge for {authz.body.identifier.value}")

     

       288
       -
                   

     

       289
       -
                   # Finalize order

     

       290
       -
                   order = acme_client.poll_and_finalize(order)

     

       291
       -
                   

     

       292
       -
                   if order.fullchain_pem:

     

       293
       -
                       # Save certificate and key

     

       294
       -
                       with open(self.cert_path, 'w') as f:

     

       295
       -
                           f.write(order.fullchain_pem.split('\n\n')[0] + '\n')

     

       296
       -
                       

     

       297
       -
                       with open(self.fullchain_path, 'w') as f:

     

       298
       -
                           f.write(order.fullchain_pem)

     

       299
       -
                       

     

       300
       -
                       key_pem = cert_key.private_bytes(

     

       301
       -
                           encoding=serialization.Encoding.PEM,

     

       302
       -
                           format=serialization.PrivateFormat.TraditionalOpenSSL,

     

       303
       -
                           encryption_algorithm=serialization.NoEncryption()

     

       304
       -
                       ).decode('utf-8')

     

       305
       -
                       

     

       306
       -
                       with open(self.key_path, 'w') as f:

     

       307
       -
                           f.write(key_pem)

     

       308
       -
                       

     

       309
       -
                       # Set proper permissions

     

       310
       -
                       os.chmod(self.key_path, 0o600)

     

       311
       -
                       os.chmod(self.cert_path, 0o644)

     

       312
       -
                       os.chmod(self.fullchain_path, 0o644)

     

       313
       -
                       

     

       314
       -
                       logger.info(

     

       315
       -
                           "Certificate obtained successfully",

     

       316
       -
                           cert_path=str(self.cert_path),

     

       317
       -
                           key_path=str(self.key_path),

     

       318
       -
                           fullchain_path=str(self.fullchain_path)

     

       319
       -
                       )

     

       320
       -
                       

     

       321
       -
                       return self.cert_path, self.key_path, self.fullchain_path

     

       322
       -
                   else:

     

       323
       -
                       raise Exception("Failed to obtain certificate")

     

       324
       -
                       

     

       325
       -
               except Exception as e:

     

       326
       -
                   logger.error("Failed to obtain certificate", error=str(e))

     

       327
       -
                   raise

     

       328
       -
               finally:

     

       329
       -
                   # Clean up challenge tokens and stop server

     

       330
       -
                   self.challenge_tokens.clear()

     

       331
       -
                   self._stop_challenge_server()

     

       332
       -
           

     

       333
       -
           def setup_auto_renewal(self, check_interval: int = 86400):

     

       334
       -
               """Setup automatic certificate renewal.

     

       335
       -
               

     

       336
       -
               Args:

     

       337
       -
                   check_interval: Interval in seconds to check for renewal (default: 24 hours)

     

       338
       -
               """

     

       339
       -
               def renewal_loop():

     

       340
       -
                   """Background renewal loop."""

     

       341
       -
                   while True:

     

       342
       -
                       try:

     

       343
       -
                           if self.needs_renewal():

     

       344
       -
                               logger.info("Certificate renewal needed")

     

       345
       -
                               self.obtain_certificate()

     

       346
       -
                           else:

     

       347
       -
                               logger.debug("Certificate renewal not needed")

     

       348
       -
                       except Exception as e:

     

       349
       -
                           logger.error("Certificate renewal check failed", error=str(e))

     

       350
       -
                       

     

       351
       -
                       time.sleep(check_interval)

     

       352
       -
               

     

       353
       -
               renewal_thread = threading.Thread(target=renewal_loop, daemon=True)

     

       354
       -
               renewal_thread.start()

     

       355
       -
               logger.info("Started automatic certificate renewal", interval_hours=check_interval/3600)

+2 -17

netdata_zulip_bot/main.py

···

       42
        
       ZULIP_API_KEY=your-api-key-here

     

       43
        
       ZULIP_STREAM=netdata-alerts

     

       44
        
       

     

       45
       -
       # Server Configuration

     

       46
        
       SERVER_HOST=0.0.0.0

     

       47
       -
       SERVER_PORT=8443

     

       48
       -
       SERVER_DOMAIN=your-domain.com

     

       49
       -
       SERVER_ENABLE_MTLS=true

     

       50
       -
       

     

       51
       -
       # Automated SSL Certificate Configuration (Recommended)

     

       52
       -
       SERVER_AUTO_CERT=true

     

       53
       -
       SERVER_CERT_EMAIL=admin@example.com

     

       54
       -
       SERVER_CERT_PATH=./certs

     

       55
       -
       # Use Let's Encrypt staging server for testing

     

       56
       -
       SERVER_CERT_STAGING=false

     

       57
       -
       # Port for ACME HTTP-01 challenge (must be accessible from internet)

     

       58
       -
       SERVER_ACME_PORT=80

     

       59
       -
       

     

       60
       -
       # Manual SSL Certificate Configuration (if not using auto-cert)

     

       61
       -
       # SERVER_AUTO_CERT=false

     

       62
       -
       # SERVER_CERT_PATH=/etc/letsencrypt/live

     

       63
        
       """

     

       64
        
           

     

       65
        
           with open(".env.sample", 'w') as f:

···

       42
        
       ZULIP_API_KEY=your-api-key-here

     

       43
        
       ZULIP_STREAM=netdata-alerts

     

       44
        
       

     

       45
       +
       # Server Configuration (HTTP only, TLS handled by reverse proxy)

     

       46
        
       SERVER_HOST=0.0.0.0

     

       47
       +
       SERVER_PORT=8080

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       48
        
       """

     

       49
        
           

     

       50
        
           with open(".env.sample", 'w') as f:

+1 -8

netdata_zulip_bot/models.py

···

       82
        
       class ServerConfig(BaseModel):

     

       83
        
           """Server configuration."""

     

       84
        
           host: str = "0.0.0.0"

     

       85
       -
           port: int = 8443

     

       86
       -
           domain: str  # Required for Let's Encrypt

     

       87
       -
           cert_path: str = "./certs"  # Directory for storing certificates

     

       88
       -
           enable_mtls: bool = True

     

       89
       -
           auto_cert: bool = False  # Enable automatic certificate management

     

       90
       -
           cert_email: str = ""  # Email for Let's Encrypt account

     

       91
       -
           cert_staging: bool = False  # Use Let's Encrypt staging server

     

       92
       -
           acme_port: int = 80  # Port for ACME HTTP-01 challenge

     

       93
        
           

     

       94
        
           model_config = ConfigDict(env_prefix="SERVER_")

···

       82
        
       class ServerConfig(BaseModel):

     

       83
        
           """Server configuration."""

     

       84
        
           host: str = "0.0.0.0"

     

       85
       +
           port: int = 8080  # Default HTTP port

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       86
        
           

     

       87
        
           model_config = ConfigDict(env_prefix="SERVER_")

-38

netdata_zulip_bot/netdata_ca.py

···

       1
       -
       """Netdata Cloud CA certificate for mutual TLS authentication."""

     

       2
       -
       

     

       3
       -
       # This certificate is from the official Netdata documentation:

     

       4
       -
       # https://github.com/netdata/netdata/blob/master/integrations/cloud-notifications/metadata.yaml

     

       5
       -
       NETDATA_CA_CERT = """-----BEGIN CERTIFICATE-----

     

       6
       -
       MIIF0jCCA7qgAwIBAgIUDV0rS5jXsyNX33evHEQOwn9fPo0wDQYJKoZIhvcNAQEN

     

       7
       -
       BQAwgYAxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH

     

       8
       -
       Ew1TYW4gRnJhbmNpc2NvMRYwFAYDVQQKEw1OZXRkYXRhLCBJbmMuMRIwEAYDVQQL

     

       9
       -
       EwlDbG91ZCBTUkUxGDAWBgNVBAMTD05ldGRhdGEgUm9vdCBDQTAeFw0yMzAyMjIx

     

       10
       -
       MjQzMDBaFw0zMzAyMTkxMjQzMDBaMIGAMQswCQYDVQQGEwJVUzETMBEGA1UECBMK

     

       11
       -
       Q2FsaWZvcm5pYTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEWMBQGA1UEChMNTmV0

     

       12
       -
       ZGF0YSwgSW5jLjESMBAGA1UECxMJQ2xvdWQgU1JFMRgwFgYDVQQDEw9OZXRkYXRh

     

       13
       -
       IFJvb3QgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCwIg7z3R++

     

       14
       -
       ppQYYVVoMIDlhWO3qVTMsAQoJYEvVa6fqaImUBLW/k19LUaXgUJPohB7gBp1pkjs

     

       15
       -
       QfY5dBo8iFr7MDHtyiAFjcQV181sITTMBEJwp77R4slOXCvrreizhTt1gvf4S1zL

     

       16
       -
       qeHBYWEgH0RLrOAqD0jkOHwewVouO0k3Wf2lEbCq3qRk2HeDvkv0LR7sFC+dDms8

     

       17
       -
       fDHqb/htqhk+FAJELGRqLeaFq1Z5Eq1/9dk4SIeHgK5pdYqsjpBzOTmocgriw6he

     

       18
       -
       s7F3dOec1ZZdcBEAxOjbYt4e58JwuR81cWAVMmyot5JNCzYVL9e5Vc5n22qt2dmc

     

       19
       -
       Tzw2rLOPt9pT5bzbmyhcDuNg2Qj/5DySAQ+VQysx91BJRXyUimqE7DwQyLhpQU72

     

       20
       -
       jw29lf2RHdCPNmk8J1TNropmpz/aI7rkperPugdOmxzP55i48ECbvDF4Wtazi+l+

     

       21
       -
       4kx7ieeLfEQgixy4lRUUkrgJlIDOGbw+d2Ag6LtOgwBiBYnDgYpvLucnx5cFupPY

     

       22
       -
       Cy3VlJ4EKUeQQSsz5kVmvotk9MED4sLx1As8V4e5ViwI5dCsRfKny7BeJ6XNPLnw

     

       23
       -
       PtMh1hbiqCcDmB1urCqXcMle4sRhKccReYOwkLjLLZ80A+MuJuIEAUUuEPCwywzU

     

       24
       -
       R7pagYsmvNgmwIIuJtB6mIJBShC7TpJG+wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMC

     

       25
       -
       AQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU9IbvOsPSUrpr8H2zSafYVQ9e

     

       26
       -
       Ft8wDQYJKoZIhvcNAQENBQADggIBABQ08aI31VKZs8jzg+y/QM5cvzXlVhcpkZsY

     

       27
       -
       1VVBr0roSBw9Pld9SERrEHto8PVXbadRxeEs4sKivJBKubWAooQ6NTvEB9MHuGnZ

     

       28
       -
       VCU+N035Gq/mhBZgtIs/Zz33jTB2ju3G4Gm9VTZbVqd0OUxFs41Iqvi0HStC3/Io

     

       29
       -
       rKi7crubmp5f2cNW1HrS++ScbTM+VaKVgQ2Tg5jOjou8wtA+204iYXlFpw9Q0qnP

     

       30
       -
       qq6ix7TfLLeRVp6mauwPsAJUgHZluz7yuv3r7TBdukU4ZKUmfAGIPSebtB3EzXfH

     

       31
       -
       7Y326xzv0hEpjvDHLy6+yFfTdBSrKPsMHgc9bsf88dnypNYL8TUiEHlcTgCGU8ts

     

       32
       -
       ud8sWN2M5FEWbHPNYRVfH3xgY2iOYZzn0i+PVyGryOPuzkRHTxDLPIGEWE5susM4

     

       33
       -
       X4bnNJyKH1AMkBCErR34CLXtAe2ngJlV/V3D4I8CQFJdQkn9tuznohUU/j80xvPH

     

       34
       -
       FOcDGQYmh4m2aIJtlNVP6+/92Siugb5y7HfslyRK94+bZBg2D86TcCJWaaZOFUrR

     

       35
       -
       Y3WniYXsqM5/JI4OOzu7dpjtkJUYvwtg7Qb5jmm8Ilf5rQZJhuvsygzX6+WM079y

     

       36
       -
       nsjoQAm6OwpTN5362vE9SYu1twz7KdzBlUkDhePEOgQkWfLHBJWwB+PvB1j/cUA3

     

       37
       -
       5zrbwvQf

     

       38
       -
       -----END CERTIFICATE-----"""

+3 -82

netdata_zulip_bot/server.py

···

       1
        
       """FastAPI webhook server for receiving Netdata notifications."""

     

       2
        
       

     

       3
       -
       import ssl

     

       4
       -
       import tempfile

     

       5
       -
       from pathlib import Path

     

       6
        
       from typing import Dict, Any

     

       7
        
       

     

       8
        
       import structlog

     
···

       10
        
       from fastapi import FastAPI, HTTPException, Request, status

     

       11
        
       from fastapi.responses import JSONResponse

     

       12
        
       

     

       13
       -
       from .cert_manager import CertificateManager

     

       14
        
       from .formatter import ZulipMessageFormatter

     

       15
        
       from .models import WebhookPayload, ZulipConfig, ServerConfig

     

       16
       -
       from .netdata_ca import NETDATA_CA_CERT

     

       17
        
       from .zulip_client import ZulipNotifier

     

       18
        
       

     

       19
        
       logger = structlog.get_logger()

     
···

       32
        
               self.zulip_config = zulip_config

     

       33
        
               self.server_config = server_config

     

       34
        
               self.formatter = ZulipMessageFormatter()

     

       35
       -
               self.cert_manager = None

     

       36
       -
               

     

       37
       -
               # Initialize certificate manager if auto-cert is enabled

     

       38
       -
               if self.server_config.auto_cert:

     

       39
       -
                   self.cert_manager = CertificateManager(

     

       40
       -
                       domain=self.server_config.domain,

     

       41
       -
                       email=self.server_config.cert_email,

     

       42
       -
                       cert_dir=Path(self.server_config.cert_path),

     

       43
       -
                       staging=self.server_config.cert_staging,

     

       44
       -
                       port=self.server_config.acme_port

     

       45
       -
                   )

     

       46
        
               

     

       47
        
               # Initialize Zulip client

     

       48
        
               try:

     
···

       134
        
                       )

     

       135
        
                       raise

     

       136
        
           

     

       137
       -
           def get_ssl_context(self) -> ssl.SSLContext:

     

       138
       -
               """Create SSL context for HTTPS and mutual TLS."""

     

       139
       -
               context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)

     

       140
       -
               

     

       141
       -
               # Get certificate paths

     

       142
       -
               if self.cert_manager and self.server_config.auto_cert:

     

       143
       -
                   # Use automated certificates

     

       144
       -
                   try:

     

       145
       -
                       cert_file, key_file, fullchain_file = self.cert_manager.obtain_certificate()

     

       146
       -
                       logger.info(

     

       147
       -
                           "Using automated SSL certificate",

     

       148
       -
                           cert_file=str(cert_file),

     

       149
       -
                           key_file=str(key_file)

     

       150
       -
                       )

     

       151
       -
                   except Exception as e:

     

       152
       -
                       logger.error("Failed to obtain automated certificate", error=str(e))

     

       153
       -
                       raise

     

       154
       -
               else:

     

       155
       -
                   # Use manually provided certificates

     

       156
       -
                   cert_path = Path(self.server_config.cert_path) / self.server_config.domain

     

       157
       -
                   fullchain_file = cert_path / "fullchain.pem"

     

       158
       -
                   key_file = cert_path / "privkey.pem"

     

       159
       -
                   

     

       160
       -
                   if not fullchain_file.exists() or not key_file.exists():

     

       161
       -
                       logger.error(

     

       162
       -
                           "SSL certificate files not found",

     

       163
       -
                           cert_file=str(fullchain_file),

     

       164
       -
                           key_file=str(key_file)

     

       165
       -
                       )

     

       166
       -
                       raise FileNotFoundError(f"SSL certificate files not found at {cert_path}")

     

       167
       -
               

     

       168
       -
               context.load_cert_chain(str(fullchain_file), str(key_file))

     

       169
       -
               

     

       170
       -
               # Configure mutual TLS if enabled

     

       171
       -
               if self.server_config.enable_mtls:

     

       172
       -
                   # Use the hardcoded Netdata CA certificate

     

       173
       -
                   with tempfile.NamedTemporaryFile(mode='w', suffix='.pem', delete=False) as ca_file:

     

       174
       -
                       ca_file.write(NETDATA_CA_CERT)

     

       175
       -
                       ca_file_path = ca_file.name

     

       176
       -
                   

     

       177
       -
                   try:

     

       178
       -
                       context.load_verify_locations(ca_file_path)

     

       179
       -
                       context.verify_mode = ssl.CERT_REQUIRED

     

       180
       -
                       logger.info("Mutual TLS enabled with hardcoded Netdata CA certificate")

     

       181
       -
                   finally:

     

       182
       -
                       # Clean up the temporary file

     

       183
       -
                       Path(ca_file_path).unlink(missing_ok=True)

     

       184
       -
               else:

     

       185
       -
                   context.verify_mode = ssl.CERT_NONE

     

       186
       -
                   logger.info("Mutual TLS disabled")

     

       187
       -
               

     

       188
       -
               return context

     

       189
        
           

     

       190
        
           def run(self):

     

       191
       -
               """Run the webhook server with HTTPS and optional mutual TLS."""

     

       192
        
               try:

     

       193
       -
                   # Setup automatic certificate renewal if enabled

     

       194
       -
                   if self.cert_manager and self.server_config.auto_cert:

     

       195
       -
                       self.cert_manager.setup_auto_renewal()

     

       196
       -
                       logger.info("Automatic certificate renewal enabled")

     

       197
       -
                   

     

       198
       -
                   ssl_context = self.get_ssl_context()

     

       199
       -
                   

     

       200
        
                   logger.info(

     

       201
       -
                       "Starting Netdata Zulip webhook server",

     

       202
        
                       host=self.server_config.host,

     

       203
       -
                       port=self.server_config.port,

     

       204
       -
                       domain=self.server_config.domain,

     

       205
       -
                       mtls_enabled=self.server_config.enable_mtls,

     

       206
       -
                       auto_cert=self.server_config.auto_cert

     

       207
        
                   )

     

       208
        
                   

     

       209
        
                   uvicorn.run(

     

       210
        
                       self.app,

     

       211
        
                       host=self.server_config.host,

     

       212
        
                       port=self.server_config.port,

     

       213
       -
                       ssl_context=ssl_context,

     

       214
        
                       access_log=False,  # We handle logging in middleware

     

       215
        
                   )

     

       216

···

       1
        
       """FastAPI webhook server for receiving Netdata notifications."""

     

       2
        
       

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       3
        
       from typing import Dict, Any

     

       4
        
       

     

       5
        
       import structlog

     
···

       7
        
       from fastapi import FastAPI, HTTPException, Request, status

     

       8
        
       from fastapi.responses import JSONResponse

     

       9
        
       

     

       0
        
       
     

       10
        
       from .formatter import ZulipMessageFormatter

     

       11
        
       from .models import WebhookPayload, ZulipConfig, ServerConfig

     

       0
        
       
     

       12
        
       from .zulip_client import ZulipNotifier

     

       13
        
       

     

       14
        
       logger = structlog.get_logger()

     
···

       27
        
               self.zulip_config = zulip_config

     

       28
        
               self.server_config = server_config

     

       29
        
               self.formatter = ZulipMessageFormatter()

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       30
        
               

     

       31
        
               # Initialize Zulip client

     

       32
        
               try:

     
···

       118
        
                       )

     

       119
        
                       raise

     

       120
        
           

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       121
        
           

     

       122
        
           def run(self):

     

       123
       +
               """Run the webhook server (HTTP only, TLS handled by reverse proxy)."""

     

       124
        
               try:

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       125
        
                   logger.info(

     

       126
       +
                       "Starting Netdata Zulip webhook server (HTTP)",

     

       127
        
                       host=self.server_config.host,

     

       128
       +
                       port=self.server_config.port

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       129
        
                   )

     

       130
        
                   

     

       131
        
                   uvicorn.run(

     

       132
        
                       self.app,

     

       133
        
                       host=self.server_config.host,

     

       134
        
                       port=self.server_config.port,

     

       0
        
       
     

       135
        
                       access_log=False,  # We handle logging in middleware

     

       136
        
                   )

     

       137

-3

pyproject.toml

···

       13
        
           "zulip>=0.9.0",

     

       14
        
           "pydantic>=2.5.0",

     

       15
        
           "python-multipart>=0.0.6",

     

       16
       -
           "acme>=2.8.0",

     

       17
       -
           "josepy>=1.14.0",

     

       18
       -
           "cryptography>=41.0.0",

     

       19
        
           "python-dotenv>=1.0.0",

     

       20
        
           "structlog>=23.2.0",

     

       21
        
       ]

···

       13
        
           "zulip>=0.9.0",

     

       14
        
           "pydantic>=2.5.0",

     

       15
        
           "python-multipart>=0.0.6",

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       16
        
           "python-dotenv>=1.0.0",

     

       17
        
           "structlog>=23.2.0",

     

       18
        
       ]

-129

tests/test_cert_manager.py

···

       1
       -
       """Tests for the certificate manager module."""

     

       2
       -
       

     

       3
       -
       import tempfile

     

       4
       -
       from pathlib import Path

     

       5
       -
       from unittest.mock import Mock, patch, MagicMock

     

       6
       -
       

     

       7
       -
       import pytest

     

       8
       -
       

     

       9
       -
       from netdata_zulip_bot.cert_manager import CertificateManager

     

       10
       -
       

     

       11
       -
       

     

       12
       -
       class TestCertificateManager:

     

       13
       -
           """Test certificate manager functionality."""

     

       14
       -
           

     

       15
       -
           @pytest.fixture

     

       16
       -
           def temp_cert_dir(self):

     

       17
       -
               """Create a temporary directory for certificates."""

     

       18
       -
               with tempfile.TemporaryDirectory() as tmpdir:

     

       19
       -
                   yield Path(tmpdir)

     

       20
       -
           

     

       21
       -
           @pytest.fixture

     

       22
       -
           def cert_manager(self, temp_cert_dir):

     

       23
       -
               """Create a certificate manager instance."""

     

       24
       -
               return CertificateManager(

     

       25
       -
                   domain="test.example.com",

     

       26
       -
                   email="test@example.com",

     

       27
       -
                   cert_dir=temp_cert_dir,

     

       28
       -
                   staging=True,  # Always use staging for tests

     

       29
       -
                   port=8080

     

       30
       -
               )

     

       31
       -
           

     

       32
       -
           def test_initialization(self, cert_manager, temp_cert_dir):

     

       33
       -
               """Test certificate manager initialization."""

     

       34
       -
               assert cert_manager.domain == "test.example.com"

     

       35
       -
               assert cert_manager.email == "test@example.com"

     

       36
       -
               assert cert_manager.cert_dir == temp_cert_dir

     

       37
       -
               assert cert_manager.staging is True

     

       38
       -
               assert cert_manager.challenge_port == 8080

     

       39
       -
               

     

       40
       -
               # Check that paths are created correctly

     

       41
       -
               assert cert_manager.account_key_path == temp_cert_dir / "account_key.pem"

     

       42
       -
               assert cert_manager.cert_path == temp_cert_dir / "test.example.com_cert.pem"

     

       43
       -
               assert cert_manager.key_path == temp_cert_dir / "test.example.com_key.pem"

     

       44
       -
               assert cert_manager.fullchain_path == temp_cert_dir / "test.example.com_fullchain.pem"

     

       45
       -
           

     

       46
       -
           def test_cert_dir_creation(self, temp_cert_dir):

     

       47
       -
               """Test that certificate directory is created if it doesn't exist."""

     

       48
       -
               new_dir = temp_cert_dir / "nested" / "certs"

     

       49
       -
               cert_manager = CertificateManager(

     

       50
       -
                   domain="test.example.com",

     

       51
       -
                   email="test@example.com",

     

       52
       -
                   cert_dir=new_dir,

     

       53
       -
                   staging=True

     

       54
       -
               )

     

       55
       -
               assert new_dir.exists()

     

       56
       -
               assert new_dir.is_dir()

     

       57
       -
           

     

       58
       -
           @patch('netdata_zulip_bot.cert_manager.x509')

     

       59
       -
           def test_needs_renewal_no_cert(self, mock_x509, cert_manager):

     

       60
       -
               """Test that renewal is needed when certificate doesn't exist."""

     

       61
       -
               assert cert_manager.needs_renewal() is True

     

       62
       -
           

     

       63
       -
           @patch('netdata_zulip_bot.cert_manager.datetime')

     

       64
       -
           @patch('netdata_zulip_bot.cert_manager.x509')

     

       65
       -
           def test_needs_renewal_expired(self, mock_x509, mock_datetime, cert_manager):

     

       66
       -
               """Test that renewal is needed when certificate is expiring soon."""

     

       67
       -
               from datetime import datetime, timezone, timedelta

     

       68
       -
               

     

       69
       -
               # Create a mock certificate file

     

       70
       -
               cert_manager.cert_path.touch()

     

       71
       -
               

     

       72
       -
               # Mock certificate with 20 days remaining

     

       73
       -
               mock_cert = Mock()

     

       74
       -
               now = datetime(2024, 1, 1, tzinfo=timezone.utc)

     

       75
       -
               mock_cert.not_valid_after_utc = now + timedelta(days=20)

     

       76
       -
               mock_x509.load_pem_x509_certificate.return_value = mock_cert

     

       77
       -
               mock_datetime.now.return_value = now

     

       78
       -
               

     

       79
       -
               assert cert_manager.needs_renewal() is True

     

       80
       -
           

     

       81
       -
           @patch('netdata_zulip_bot.cert_manager.datetime')

     

       82
       -
           @patch('netdata_zulip_bot.cert_manager.x509')

     

       83
       -
           def test_needs_renewal_valid(self, mock_x509, mock_datetime, cert_manager):

     

       84
       -
               """Test that renewal is not needed when certificate is still valid."""

     

       85
       -
               from datetime import datetime, timezone, timedelta

     

       86
       -
               

     

       87
       -
               # Create a mock certificate file

     

       88
       -
               cert_manager.cert_path.touch()

     

       89
       -
               

     

       90
       -
               # Mock certificate with 60 days remaining

     

       91
       -
               mock_cert = Mock()

     

       92
       -
               now = datetime(2024, 1, 1, tzinfo=timezone.utc)

     

       93
       -
               mock_cert.not_valid_after_utc = now + timedelta(days=60)

     

       94
       -
               mock_x509.load_pem_x509_certificate.return_value = mock_cert

     

       95
       -
               mock_datetime.now.return_value = now

     

       96
       -
               

     

       97
       -
               assert cert_manager.needs_renewal() is False

     

       98
       -
           

     

       99
       -
           def test_generate_private_key(self, cert_manager):

     

       100
       -
               """Test private key generation."""

     

       101
       -
               key = cert_manager._generate_private_key()

     

       102
       -
               assert key is not None

     

       103
       -
               assert key.key_size == 2048

     

       104
       -
           

     

       105
       -
           @patch('netdata_zulip_bot.cert_manager.threading.Thread')

     

       106
       -
           def test_challenge_server_start(self, mock_thread, cert_manager):

     

       107
       -
               """Test that challenge server starts correctly."""

     

       108
       -
               cert_manager._start_challenge_server()

     

       109
       -
               

     

       110
       -
               # Verify thread was created and started

     

       111
       -
               mock_thread.assert_called_once()

     

       112
       -
               mock_thread.return_value.start.assert_called_once()

     

       113
       -
           

     

       114
       -
           def test_challenge_tokens_storage(self, cert_manager):

     

       115
       -
               """Test that challenge tokens are stored correctly."""

     

       116
       -
               cert_manager.challenge_tokens["test_token"] = "test_response"

     

       117
       -
               assert cert_manager.challenge_tokens["test_token"] == "test_response"

     

       118
       -
           

     

       119
       -
           @patch('netdata_zulip_bot.cert_manager.client.ClientV2')

     

       120
       -
           @patch('netdata_zulip_bot.cert_manager.client.ClientNetwork')

     

       121
       -
           def test_obtain_certificate_mock(self, mock_network, mock_client, cert_manager):

     

       122
       -
               """Test certificate obtaining with mocked ACME client."""

     

       123
       -
               # This is a simplified test that mocks the ACME interaction

     

       124
       -
               # In production, this would interact with Let's Encrypt staging server

     

       125
       -
               

     

       126
       -
               # Mock that certificate doesn't need renewal

     

       127
       -
               with patch.object(cert_manager, 'needs_renewal', return_value=False):

     

       128
       -
                   paths = cert_manager.obtain_certificate()

     

       129
       -
                   assert paths == (cert_manager.cert_path, cert_manager.key_path, cert_manager.fullchain_path)