Netdata.cloud bot for Zulip
anil.recoil.org
1# Netdata Zulip Bot
2
3*100% vibe coded, use at your peril*
4
5A webhook service that receives notifications from Netdata Cloud and forwards them to Zulip channels. Features HTTPS with Let's Encrypt certificates and mutual TLS authentication for secure communication with Netdata Cloud.
6
7## Features
8
9- 🔐 **HTTPS with Let's Encrypt**: Automatic SSL certificate management
10- 🤝 **Mutual TLS**: Secure authentication with Netdata Cloud
11- 📊 **Rich Formatting**: Beautiful Zulip messages with emojis and markdown
12- 🏷️ **Topic Organization**: Automatic topic routing by severity level
13- 📝 **Structured Logging**: JSON-structured logs for monitoring
14- ⚡ **High Performance**: FastAPI-based webhook endpoint
15
16## Quick Start
17
18### 1. Install Dependencies
19
20```bash
21# Using uv (recommended)
22uv sync
23
24# Or using pip
25pip install -e .
26```
27
28### 2. Create Configuration
29
30```bash
31# Generate sample configuration files
32netdata-zulip-bot --create-config
33
34# Copy and customize
35cp .zuliprc.sample ~/.zuliprc
36```
37
38### 3. Configure Zulip Settings
39
40Edit `~/.zuliprc`:
41
42```ini
43[api]
44site=https://yourorg.zulipchat.com
45email=netdata-bot@yourorg.zulipchat.com
46key=your-zulip-api-key
47stream=netdata-alerts
48```
49
50### 4. Set Server Environment Variables
51
52```bash
53export SERVER_DOMAIN=your-webhook-domain.com
54export SERVER_PORT=8443
55export SERVER_ENABLE_MTLS=true
56```
57
58### 5. Setup SSL Certificate
59
60```bash
61# Install certbot and obtain certificate
62sudo certbot certonly --standalone -d your-webhook-domain.com
63
64# Ensure certificate files are accessible
65sudo chown -R $USER:$USER /etc/letsencrypt/live/your-webhook-domain.com/
66```
67
68### 6. Run the Service
69
70```bash
71netdata-zulip-bot
72```
73
74## Configuration
75
76### Zulip Configuration
77
78The bot supports two configuration methods:
79
80#### Method 1: Zuliprc File (Recommended)
81
82Create `~/.zuliprc`:
83
84```ini
85[api]
86site=https://yourorg.zulipchat.com
87email=netdata-bot@yourorg.zulipchat.com
88key=your-zulip-api-key
89stream=netdata-alerts
90```
91
92#### Method 2: Environment Variables
93
94```bash
95export ZULIP_SITE=https://yourorg.zulipchat.com
96export ZULIP_EMAIL=netdata-bot@yourorg.zulipchat.com
97export ZULIP_API_KEY=your-api-key
98export ZULIP_STREAM=netdata-alerts
99```
100
101Use `--env-config` flag to use environment variables instead of zuliprc.
102
103### Server Configuration
104
105Set these environment variables:
106
107- `SERVER_DOMAIN`: Your public domain (required for Let's Encrypt)
108- `SERVER_HOST`: Bind address (default: `0.0.0.0`)
109- `SERVER_PORT`: HTTPS port (default: `8443`)
110- `SERVER_CERT_PATH`: Certificate path (default: `/etc/letsencrypt/live`)
111- `SERVER_ENABLE_MTLS`: Enable mutual TLS (default: `true`)
112
113## Message Format
114
115### Alert Notifications
116
117Messages are posted to topics based on severity level:
118
119- **Topic**: `critical`, `warning`, or `clear`
120- **Format**: Rich markdown with alert details, timestamps, and links
121
122Example:
123```
124🔴 **High CPU Usage**
125
126**Space:** production
127**Chart:** system.cpu
128**Context:** cpu utilization
129**Severity:** Critical
130**Time:** 2024-01-15 14:30:00 UTC
131
132**Details:** CPU usage has exceeded 90% for 5 minutes
133**Summary:** Critical alert: High CPU usage detected
134
135[View Alert](https://app.netdata.cloud/spaces/...)
136```
137
138### Reachability Notifications
139
140Messages are posted to the `reachability` topic:
141
142```
143❌ **Host Unreachable**
144
145**Host:** web-server-01
146**Status:** ❌ Unreachable
147**Severity:** Critical
148
149**Summary:** Host web-server-01 is no longer reachable
150
151[View Host](https://app.netdata.cloud/...)
152```
153
154## Deployment
155
156### Systemd Service
157
158Create `/etc/systemd/system/netdata-zulip-bot.service`:
159
160```ini
161[Unit]
162Description=Netdata Zulip Bot
163After=network.target
164
165[Service]
166Type=simple
167User=netdata-bot
168WorkingDirectory=/opt/netdata-zulip-bot
169Environment=SERVER_DOMAIN=your-domain.com
170ExecStart=/opt/netdata-zulip-bot/venv/bin/netdata-zulip-bot
171Restart=always
172RestartSec=5
173
174[Install]
175WantedBy=multi-user.target
176```
177
178Enable and start:
179```bash
180sudo systemctl enable netdata-zulip-bot
181sudo systemctl start netdata-zulip-bot
182```
183
184### Docker
185
186```dockerfile
187FROM python:3.11-slim
188
189WORKDIR /app
190COPY . .
191RUN pip install -e .
192
193EXPOSE 8443
194
195CMD ["netdata-zulip-bot"]
196```
197
198## Security
199
200### Mutual TLS Authentication
201
202The service supports mutual TLS to authenticate Netdata Cloud webhooks:
203
2041. **Server Certificate**: Automatically managed by Let's Encrypt
2052. **Client Verification**: Validates Netdata's client certificate
2063. **CA Certificate**: Built-in Netdata CA certificate for client validation
207
208### Webhook Endpoint Security
209
210- HTTPS-only communication
211- Request logging and monitoring
212- Payload validation and sanitization
213- Error handling without information disclosure
214
215## Monitoring
216
217The service provides structured JSON logging for easy monitoring:
218
219```json
220{
221 "timestamp": "2024-01-15T14:30:00.000Z",
222 "level": "info",
223 "event": "Message sent to Zulip",
224 "stream": "netdata-alerts",
225 "topic": "critical",
226 "message_id": 12345
227}
228```
229
230### Health Check
231
232```bash
233curl -k https://your-domain.com:8443/health
234```
235
236Response:
237```json
238{
239 "status": "healthy",
240 "service": "netdata-zulip-bot"
241}
242```
243
244## Development
245
246### Running Tests
247
248```bash
249pytest
250```
251
252### Code Formatting
253
254```bash
255black .
256ruff check .
257```
258
259### Local Development
260
261For development, you can disable HTTPS and mTLS:
262
263```bash
264export SERVER_ENABLE_MTLS=false
265# Use HTTP for testing (not recommended for production)
266```
267
268## Troubleshooting
269
270### Common Issues
271
2721. **Certificate Not Found**
273 - Ensure Let's Encrypt certificates exist at `/etc/letsencrypt/live/your-domain.com/`
274 - Check file permissions
275
2762. **Zulip Connection Failed**
277 - Verify API credentials in zuliprc
278 - Test connection with Zulip's API
279
2803. **Webhook Not Receiving Data**
281 - Check firewall settings for port 8443
282 - Verify domain DNS resolution
283 - Check Netdata Cloud webhook configuration
284
285### Logs
286
287View service logs:
288```bash
289sudo journalctl -u netdata-zulip-bot -f
290```
291
292## License
293
294MIT License - see LICENSE file for details.