# Netdata Zulip Bot

A production-ready webhook service that receives notifications from Netdata Cloud and forwards them to Zulip channels. Designed to run behind a reverse proxy (like Caddy) that handles HTTPS and mutual TLS authentication.

## Features

- 🔗 **Reverse Proxy Ready**: HTTP service designed to run behind Caddy/nginx
- 🤝 **Mutual TLS Support**: When configured with reverse proxy
- 📊 **Rich Formatting**: Beautiful Zulip messages with emojis and markdown
- 🏷️ **Topic Organization**: Automatic topic routing by severity level
- 📝 **Structured Logging**: JSON-structured logs for monitoring
- ⚡ **High Performance**: FastAPI-based webhook endpoint
- 🔧 **Flexible Configuration**: Support for .zuliprc files or environment variables
- ✅ **Webhook Verification**: Built-in Netdata challenge/response handling

## Quick Start

### 1. Install Dependencies

```bash
# Using uv (recommended)
uv sync
```

### 2. Create Configuration

```bash
# Generate sample configuration files
uv run netdata-zulip-bot --create-config

# Copy and customize
cp .zuliprc.sample ~/.zuliprc
cp .env.sample .env
```

### 3. Configure Zulip Settings

Edit `~/.zuliprc`:

```ini
[api]
site=https://yourorg.zulipchat.com
email=netdata-bot@yourorg.zulipchat.com  
key=your-zulip-api-key
stream=netdata-alerts
```

### 4. Configure Environment Variables

Edit `.env` file or set environment variables:

```bash
# Server configuration (HTTP only)
export SERVER_HOST=0.0.0.0
export SERVER_PORT=8080

# Required: Netdata webhook challenge secret
export SERVER_CHALLENGE_SECRET=your-challenge-secret-here

# Optional: Override Zulip stream
export ZULIP_STREAM=netdata-alerts
```

### 5. Run the Service

```bash
# Start the HTTP service
uv run netdata-zulip-bot

# Or with custom configuration
uv run netdata-zulip-bot --zuliprc /path/to/.zuliprc

# The service runs on HTTP (default: localhost:8080)
# Use a reverse proxy like Caddy for HTTPS and mutual TLS
```

## Configuration

### Zulip Configuration

The bot supports two configuration methods:

#### Method 1: Zuliprc File (Recommended)

Create `~/.zuliprc`:

```ini
[api]
site=https://yourorg.zulipchat.com
email=netdata-bot@yourorg.zulipchat.com
key=your-zulip-api-key
stream=netdata-alerts
```

#### Method 2: Environment Variables

```bash
export ZULIP_SITE=https://yourorg.zulipchat.com
export ZULIP_EMAIL=netdata-bot@yourorg.zulipchat.com
export ZULIP_API_KEY=your-api-key
export ZULIP_STREAM=netdata-alerts
```

Use the `--env-config` flag to use environment variables instead of zuliprc:

```bash
uv run netdata-zulip-bot --env-config
```

### Server Configuration

Set these environment variables:

- `SERVER_HOST`: Bind address (default: `0.0.0.0`)
- `SERVER_PORT`: HTTP port (default: `8080`)
- `SERVER_CHALLENGE_SECRET`: Netdata webhook challenge secret (required)

### Reverse Proxy Setup

The bot is designed to run behind a reverse proxy that handles HTTPS and mutual TLS:

#### Using Caddy (Recommended)

1. Update `Caddyfile` with your domain name
2. Place Netdata CA certificate in `netdata-ca.pem`
3. Run both services:

```bash
# Start the bot
uv run netdata-zulip-bot &

# Start Caddy
caddy run --config Caddyfile
```

#### Using Docker Compose

```bash
docker-compose up -d
```

## Message Format

### Alert Notifications

Messages are posted to topics based on severity level:

- **Topic**: `critical`, `warning`, or `clear`
- **Format**: Rich markdown with alert details, timestamps, and links

Example:
```
🔴 **High CPU Usage**

**Space:** production
**Chart:** system.cpu
**Context:** cpu utilization
**Severity:** Critical
**Time:** 2024-01-15 14:30:00 UTC

**Details:** CPU usage has exceeded 90% for 5 minutes

**Summary:** Critical alert: High CPU usage detected

[View Alert](https://app.netdata.cloud/spaces/...)
```

### Reachability Notifications

Messages are posted to the `reachability` topic:

```
❌ **Host Unreachable**

**Host:** web-server-01
**Status:** ❌ Unreachable  
**Severity:** Critical

**Summary:** Host web-server-01 is no longer reachable

[View Host](https://app.netdata.cloud/...)
```

## Deployment

### Systemd Service

See `examples/netdata-zulip-bot.service` for a complete systemd service configuration.

### Automated Setup

Use the provided setup script:

```bash
sudo ./scripts/setup.sh --domain your-domain.com --email admin@example.com
```

### Docker

The included `Dockerfile` and `docker-compose.yml` provide a complete setup with Caddy reverse proxy:

```bash
docker-compose up -d
```

## Security

### Architecture

The bot uses a security-focused architecture:

1. **HTTP Backend**: Simple HTTP service with no direct internet exposure
2. **Reverse Proxy**: Caddy handles HTTPS, certificates, and client authentication
3. **Mutual TLS**: Client certificate validation at the reverse proxy level

### Webhook Security

- **Challenge/Response**: Built-in Netdata webhook verification using HMAC-SHA256
- **Payload Validation**: Strict payload parsing and validation
- **Request Logging**: Comprehensive logging of all webhook requests
- **Error Handling**: Secure error responses without information disclosure

### SSL Certificate Management

SSL certificates are managed by the reverse proxy (Caddy):

1. **Automatic Provisioning**: Caddy obtains Let's Encrypt certificates
2. **Automatic Renewal**: Built-in certificate renewal
3. **Mutual TLS**: Client certificate validation using Netdata CA certificate

## Monitoring

The service provides structured JSON logging for easy monitoring:

```json
{
  "timestamp": "2024-01-15T14:30:00.000Z",
  "level": "info",
  "event": "Message sent to Zulip",
  "stream": "netdata-alerts",
  "topic": "critical",
  "message_id": 12345
}
```

### Health Check

```bash
# Direct HTTP check (backend service)
curl http://localhost:8080/health

# Through reverse proxy
curl https://your-domain.com/health
```

Response:
```json
{
  "status": "healthy",
  "service": "netdata-zulip-bot"
}
```

## Development

### Running Tests

```bash
uv run python -m pytest tests/ -v
```

### Code Formatting

```bash
uv run black .
uv run ruff check .
```

### Local Development

For development, you can run the HTTP service directly:

```bash
# Set required environment variables
export SERVER_CHALLENGE_SECRET=test-secret

# Run the service
uv run netdata-zulip-bot

# Test webhook endpoint
curl -X POST http://localhost:8080/webhook/netdata?crc_token=test123
```

## Troubleshooting

### Common Issues

1. **Configuration Issues**
   - Ensure `SERVER_CHALLENGE_SECRET` is set (required for Netdata webhook verification)
   - Verify `.zuliprc` file contains all required fields
   - Check that Zulip bot has permission to post to the configured stream

2. **Reverse Proxy Issues**
   - Ensure Caddy configuration uses correct domain name
   - Verify Netdata CA certificate is properly configured
   - Check that port 80 is accessible for Let's Encrypt challenges

3. **Webhook Not Receiving Data**
   - Verify Netdata Cloud webhook URL points to your reverse proxy
   - Check webhook challenge secret matches configuration
   - Review service logs for error messages

### Logs

View service logs:
```bash
sudo journalctl -u netdata-zulip-bot -f
```

## License

MIT License - see LICENSE file for details.