Netdata.cloud bot for Zulip
anil.recoil.org
1# Netdata Zulip Bot - Development Instructions
2
3This repository implements a Zulip bot that receives incoming webhook notifications from Netdata Cloud and posts the resulting notifications to a Zulip topic.
4
5## Core Requirements
6
7### Technology Stack
8- **Language**: Python with `uv` package manager
9- **Framework**: Python `zulip_bots` PyPI package for Zulip integration
10- **Web Server**: FastAPI for webhook endpoint
11- **Reverse Proxy**: Caddy for HTTPS and mutual TLS handling
12- **Deployment**: Standalone service behind reverse proxy
13
14### Netdata Integration
15- **Webhook Format**: Follow the Netdata Cloud webhook notification format from:
16 - URL: `https://raw.githubusercontent.com/netdata/netdata/refs/heads/master/integrations/cloud-notifications/metadata.yaml`
17 - Section: `id: notify-cloud-webhook`
18- **Notification Types**: Handle both alert notifications and reachability notifications
19
20### Zulip Configuration
21- **Configuration Source**: The `.zuliprc` file should contain:
22 - Zulip site URL
23 - Bot email and API key
24 - Target stream/channel for posting messages
25- **Topic Organization**: Messages should be posted to topics based on severity:
26 - `critical`, `warning`, `clear` for alerts
27 - `reachability` for host status changes
28- **Message Format**: Rich markdown with:
29 - Alert details and timestamps
30 - Markdown-formatted alert URLs for easy access to Netdata Cloud
31
32### Security Requirements
33- **HTTP Only**: The bot service listens on HTTP internally
34- **Reverse Proxy**: Caddy handles HTTPS with Let's Encrypt certificates
35- **Mutual TLS**: Caddy validates Netdata's client certificates
36 - Client certificate validation at the reverse proxy level
37 - Netdata CA certificate configured in Caddyfile
38
39### Service Architecture
40- **Backend Service**: FastAPI bot listening on HTTP (port 8080)
41- **Reverse Proxy**: Caddy handling HTTPS, Let's Encrypt, and mutual TLS
42- **Webhook Endpoint**: `/webhook/netdata` for receiving notifications
43- **Health Check**: `/health` endpoint for monitoring
44- **Structured Logging**: JSON-structured logs for production monitoring
45
46## Implementation Notes
47
48### Configuration Management
49- Support both `.zuliprc` file and environment variables
50- Provide sample configuration files with `--create-config` flag
51- Server configuration via environment variables:
52 - `SERVER_HOST`: Bind address (default: 0.0.0.0)
53 - `SERVER_PORT`: HTTP port (default: 8080)
54- Reverse proxy configuration in `Caddyfile`
55
56### Message Processing
571. Receive Netdata webhook POST request
582. Validate and parse JSON payload
593. Determine notification type (alert vs reachability)
604. Format message with appropriate emoji and markdown
615. Send to configured Zulip stream/topic
62
63### Error Handling
64- Validate all incoming payloads
65- Log errors without exposing internal details
66- Return appropriate HTTP status codes
67- Implement retry logic for Zulip API failures
68
69## Testing
70
71Run tests with:
72```bash
73uv run python -m pytest tests/ -v
74```
75
76## Deployment
77
78The service should be deployable via:
79- Systemd service (see `examples/netdata-zulip-bot.service`)
80- Docker container (see `Dockerfile` and `docker-compose.yml`)
81- Automated setup script (`scripts/setup.sh`)
82- Caddy reverse proxy configuration (`Caddyfile`)
83
84### Reverse Proxy Setup
851. Install Caddy on your server
862. Update `Caddyfile` with your domain name
873. Place the Netdata CA certificate in `netdata-ca.pem`
884. Start both the bot service and Caddy
89
90## Development Commands
91
92- **Install dependencies**: `uv sync`
93- **Create config samples**: `uv run netdata-zulip-bot --create-config`
94- **Run tests**: `uv run python -m pytest tests/`
95- **Start service**: `uv run netdata-zulip-bot`
96
97## Important Reminders
98
99- Always validate Netdata webhook payloads before processing
100- Ensure Caddy and reverse proxy are properly configured before production deployment
101- Test mutual TLS authentication with actual Netdata Cloud webhooks
102- Monitor service logs for webhook processing errors
103- Keep Zulip API credentials secure and never commit them to the repository
104- Update the Netdata CA certificate in `netdata-ca.pem` as needed