···
1
+
# Aggregator Setup Guide
3
+
This guide explains how to set up and register an aggregator with Coves instances.
7
+
- [Overview](#overview)
8
+
- [Architecture](#architecture)
9
+
- [Prerequisites](#prerequisites)
10
+
- [Quick Start](#quick-start)
11
+
- [Detailed Setup Steps](#detailed-setup-steps)
12
+
- [Authorization Process](#authorization-process)
13
+
- [Posting to Communities](#posting-to-communities)
14
+
- [Rate Limits](#rate-limits)
15
+
- [Security Best Practices](#security-best-practices)
16
+
- [Troubleshooting](#troubleshooting)
17
+
- [API Reference](#api-reference)
21
+
**Aggregators** are automated services that post content to Coves communities. They are similar to Bluesky's feed generators and labelers - self-managed external services that integrate with the platform.
23
+
**Key characteristics**:
24
+
- Self-owned: You create and manage your own PDS account
25
+
- Domain-verified: Prove ownership via `.well-known/atproto-did`
26
+
- Community-authorized: Moderators grant posting permission per-community
27
+
- Rate-limited: 10 posts per hour per community
29
+
**Example use cases**:
30
+
- RSS feed aggregators (tech news, blog posts)
31
+
- Social media cross-posters (Twitter → Coves)
32
+
- Event notifications (GitHub releases, weather alerts)
33
+
- Content curation bots (daily links, summaries)
40
+
┌──────────────────────────────────────────────────────────┐
41
+
│ 1. One-Time Setup │
42
+
├──────────────────────────────────────────────────────────┤
43
+
│ Aggregator creates PDS account │
45
+
│ Proves domain ownership (.well-known) │
47
+
│ Registers with Coves (enters users table) │
49
+
│ Writes service declaration │
51
+
│ Jetstream indexes into aggregators table │
52
+
└──────────────────────────────────────────────────────────┘
54
+
┌──────────────────────────────────────────────────────────┐
55
+
│ 2. Per-Community Authorization │
56
+
├──────────────────────────────────────────────────────────┤
57
+
│ Moderator writes authorization record │
59
+
│ Jetstream indexes into aggregator_authorizations │
60
+
└──────────────────────────────────────────────────────────┘
62
+
┌──────────────────────────────────────────────────────────┐
63
+
│ 3. Posting (Ongoing) │
64
+
├──────────────────────────────────────────────────────────┤
65
+
│ Aggregator calls post creation endpoint │
67
+
│ Handler validates: │
68
+
│ - Author in users table ✓ │
69
+
│ - Author in aggregators table ✓ │
70
+
│ - Authorization exists ✓ │
71
+
│ - Rate limit not exceeded ✓ │
73
+
│ Post written to community's PDS │
75
+
│ Jetstream indexes post │
76
+
└──────────────────────────────────────────────────────────┘
81
+
**users** - All actors (users, communities, aggregators)
83
+
CREATE TABLE users (
84
+
did TEXT PRIMARY KEY,
85
+
handle TEXT NOT NULL,
87
+
indexed_at TIMESTAMPTZ
91
+
**aggregators** - Aggregator-specific metadata
93
+
CREATE TABLE aggregators (
94
+
did TEXT PRIMARY KEY,
95
+
display_name TEXT NOT NULL,
98
+
config_schema JSONB,
100
+
maintainer_did TEXT,
101
+
record_uri TEXT NOT NULL UNIQUE,
102
+
record_cid TEXT NOT NULL,
103
+
created_at TIMESTAMPTZ,
104
+
indexed_at TIMESTAMPTZ
108
+
**aggregator_authorizations** - Community authorizations
110
+
CREATE TABLE aggregator_authorizations (
111
+
id BIGSERIAL PRIMARY KEY,
112
+
aggregator_did TEXT NOT NULL,
113
+
community_did TEXT NOT NULL,
114
+
enabled BOOLEAN NOT NULL DEFAULT true,
117
+
record_uri TEXT NOT NULL UNIQUE,
118
+
record_cid TEXT NOT NULL,
119
+
UNIQUE(aggregator_did, community_did)
125
+
1. **Domain ownership**: You must own a domain where you can host static files over HTTPS
126
+
2. **Web server**: Ability to serve the `.well-known/atproto-did` file
127
+
3. **Development tools**: `curl`, `jq`, basic shell scripting knowledge
128
+
4. **Email address**: For creating the PDS account
131
+
- Custom avatar image (PNG/JPEG/WebP, max 1MB)
132
+
- GitHub repository for source code transparency
136
+
We provide automated setup scripts:
139
+
cd scripts/aggregator-setup
141
+
# Make scripts executable
144
+
# Run setup scripts in order
145
+
./1-create-pds-account.sh
146
+
./2-setup-wellknown.sh
147
+
# (Upload .well-known to your web server)
148
+
./3-register-with-coves.sh
149
+
./4-create-service-declaration.sh
152
+
See [scripts/aggregator-setup/README.md](../../scripts/aggregator-setup/README.md) for detailed script documentation.
154
+
## Detailed Setup Steps
156
+
### Step 1: Create PDS Account
158
+
Your aggregator needs its own atProto identity (DID). The easiest way is to create an account on an existing PDS.
160
+
**Using an existing PDS (recommended)**:
163
+
curl -X POST https://bsky.social/xrpc/com.atproto.server.createAccount \
164
+
-H "Content-Type: application/json" \
166
+
"handle": "mynewsbot.bsky.social",
167
+
"email": "bot@example.com",
168
+
"password": "secure-password-here"
175
+
"accessJwt": "eyJ...",
176
+
"refreshJwt": "eyJ...",
177
+
"handle": "mynewsbot.bsky.social",
178
+
"did": "did:plc:abc123...",
183
+
**Save these credentials securely!** You'll need the DID and access token for all subsequent operations.
185
+
**Alternative**: Run your own PDS or use `did:web` (advanced).
187
+
### Step 2: Prove Domain Ownership
189
+
To register with Coves, you must prove you own a domain by serving your DID at `https://yourdomain.com/.well-known/atproto-did`.
191
+
**Create the file**:
194
+
mkdir -p .well-known
195
+
echo "did:plc:abc123..." > .well-known/atproto-did
198
+
**Upload to your web server** so it's accessible at:
200
+
https://rss-bot.example.com/.well-known/atproto-did
203
+
**Verify it works**:
205
+
curl https://rss-bot.example.com/.well-known/atproto-did
206
+
# Should return: did:plc:abc123...
209
+
**Nginx configuration example**:
211
+
location /.well-known/atproto-did {
212
+
alias /var/www/.well-known/atproto-did;
213
+
default_type text/plain;
214
+
add_header Access-Control-Allow-Origin *;
218
+
### Step 3: Register with Coves
220
+
Call the registration endpoint to register your aggregator DID with the Coves instance.
222
+
**Endpoint**: `POST /xrpc/social.coves.aggregator.register`
226
+
curl -X POST https://api.coves.social/xrpc/social.coves.aggregator.register \
227
+
-H "Content-Type: application/json" \
229
+
"did": "did:plc:abc123...",
230
+
"domain": "rss-bot.example.com"
234
+
**Response** (Success):
237
+
"did": "did:plc:abc123...",
238
+
"handle": "mynewsbot.bsky.social",
239
+
"message": "Aggregator registered successfully. Next step: create a service declaration record at at://did:plc:abc123.../social.coves.aggregator.service/self"
244
+
1. Coves fetches `https://rss-bot.example.com/.well-known/atproto-did`
245
+
2. Verifies it contains your DID
246
+
3. Resolves your DID to get handle and PDS URL
247
+
4. Inserts you into the `users` table
249
+
**You're now registered!** But you need to create a service declaration next.
251
+
### Step 4: Create Service Declaration
253
+
Write a `social.coves.aggregator.service` record to your repository. This contains metadata about your aggregator and gets indexed by Coves' Jetstream consumer.
255
+
**Endpoint**: `POST https://your-pds.com/xrpc/com.atproto.repo.createRecord`
259
+
curl -X POST https://bsky.social/xrpc/com.atproto.repo.createRecord \
260
+
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
261
+
-H "Content-Type: application/json" \
263
+
"repo": "did:plc:abc123...",
264
+
"collection": "social.coves.aggregator.service",
267
+
"$type": "social.coves.aggregator.service",
268
+
"did": "did:plc:abc123...",
269
+
"displayName": "RSS News Aggregator",
270
+
"description": "Aggregates tech news from various RSS feeds",
271
+
"sourceUrl": "https://github.com/yourname/rss-aggregator",
272
+
"maintainer": "did:plc:your-personal-did",
273
+
"createdAt": "2024-01-15T12:00:00Z"
281
+
"uri": "at://did:plc:abc123.../social.coves.aggregator.service/self",
282
+
"cid": "bafyrei..."
286
+
**Optional fields**:
287
+
- `avatar`: Blob reference to avatar image
288
+
- `configSchema`: JSON Schema for community-specific configuration
290
+
**Wait 5-10 seconds** for Jetstream to index your service declaration into the `aggregators` table.
292
+
## Authorization Process
294
+
Before you can post to a community, a moderator must authorize your aggregator.
296
+
### How Authorization Works
298
+
1. **Moderator decision**: Community moderator evaluates your aggregator
299
+
2. **Authorization record**: Moderator writes `social.coves.aggregator.authorization` to community's repo
300
+
3. **Jetstream indexing**: Record gets indexed into `aggregator_authorizations` table
301
+
4. **Posting enabled**: You can now post to that community
303
+
### Authorization Record Structure
305
+
**Location**: `at://{community_did}/social.coves.aggregator.authorization/{rkey}`
310
+
"$type": "social.coves.aggregator.authorization",
311
+
"aggregatorDid": "did:plc:abc123...",
312
+
"communityDid": "did:plc:community123...",
314
+
"createdBy": "did:plc:moderator...",
315
+
"createdAt": "2024-01-15T12:00:00Z",
317
+
"maxPostsPerHour": 5,
318
+
"allowedCategories": ["tech", "news"]
323
+
### Checking Your Authorizations
325
+
**Endpoint**: `GET /xrpc/social.coves.aggregator.getAuthorizations`
328
+
curl "https://api.coves.social/xrpc/social.coves.aggregator.getAuthorizations?aggregatorDid=did:plc:abc123...&enabledOnly=true"
334
+
"authorizations": [
336
+
"aggregatorDid": "did:plc:abc123...",
337
+
"communityDid": "did:plc:community123...",
338
+
"communityHandle": "~tech@coves.social",
340
+
"createdAt": "2024-01-15T12:00:00Z",
347
+
## Posting to Communities
349
+
Once authorized, you can post to communities using the standard post creation endpoint.
353
+
**Endpoint**: `POST /xrpc/social.coves.community.post.create`
357
+
curl -X POST https://api.coves.social/xrpc/social.coves.community.post.create \
358
+
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
359
+
-H "Content-Type: application/json" \
361
+
"communityDid": "did:plc:community123...",
363
+
"text": "New blog post: Understanding atProto Identity\nhttps://example.com/post",
364
+
"createdAt": "2024-01-15T12:00:00Z",
367
+
"index": { "byteStart": 50, "byteEnd": 75 },
370
+
"$type": "social.coves.richtext.facet#link",
371
+
"uri": "https://example.com/post"
383
+
"uri": "at://did:plc:abc123.../social.coves.community.post/3k...",
384
+
"cid": "bafyrei..."
388
+
### Post Validation
390
+
The handler validates:
391
+
1. **Authentication**: Valid JWT token
392
+
2. **Author exists**: DID in `users` table
393
+
3. **Is aggregator**: DID in `aggregators` table
394
+
4. **Authorization**: Active authorization for (aggregator, community)
395
+
5. **Rate limit**: Less than 10 posts/hour to this community
396
+
6. **Content**: Valid post structure per lexicon
400
+
**Per-community rate limit**: 10 posts per hour
402
+
This is tracked in the `aggregator_posts` table and enforced at the handler level.
404
+
**Why?**: Prevents spam while allowing useful bot activity.
406
+
**Best practices**:
407
+
- Batch similar content
408
+
- Post only high-quality content
409
+
- Respect community guidelines
410
+
- Monitor your posting rate
412
+
## Security Best Practices
414
+
### Credential Management
417
+
- Store credentials in environment variables or secret management
418
+
- Use HTTPS for all API calls
419
+
- Rotate access tokens regularly (use refresh tokens)
420
+
- Keep `aggregator-config.env` out of version control
423
+
- Hardcode credentials in source code
424
+
- Commit credentials to Git
425
+
- Share access tokens publicly
426
+
- Reuse personal credentials for bots
428
+
### Domain Security
431
+
- Use HTTPS for `.well-known` endpoint
432
+
- Keep domain under your control
433
+
- Monitor for unauthorized changes
434
+
- Use DNSSEC if possible
437
+
- Use HTTP (will fail verification)
438
+
- Use shared/untrusted hosting
439
+
- Allow others to modify `.well-known` files
440
+
- Use expired SSL certificates
442
+
### Content Security
445
+
- Validate all external content before posting
446
+
- Sanitize URLs and text
447
+
- Rate-limit your own posting
448
+
- Implement circuit breakers for failures
451
+
- Post unvalidated user input
452
+
- Include malicious links
454
+
- Bypass rate limits
458
+
### Registration Errors
460
+
#### Error: "DomainVerificationFailed"
462
+
**Cause**: `.well-known/atproto-did` not accessible or contains wrong DID
465
+
1. Verify file is accessible: `curl https://yourdomain.com/.well-known/atproto-did`
466
+
2. Check content matches your DID exactly (no extra whitespace)
467
+
3. Ensure HTTPS is working (not HTTP)
468
+
4. Check web server logs for access errors
469
+
5. Verify firewall rules allow HTTPS traffic
471
+
#### Error: "AlreadyRegistered"
473
+
**Cause**: This DID is already registered with this Coves instance
476
+
- This is safe to ignore if you're re-running setup
477
+
- If you need to update info, just create a new service declaration
478
+
- Contact instance admin if you need to remove registration
480
+
#### Error: "DIDResolutionFailed"
482
+
**Cause**: Could not resolve DID document from PLC directory
485
+
1. Verify DID exists: `curl https://plc.directory/{your-did}`
486
+
2. Wait 30 seconds and retry (PLC propagation delay)
487
+
3. Check PDS is accessible
488
+
4. Verify DID format is correct (must start with `did:plc:` or `did:web:`)
492
+
#### Error: "NotAuthorized"
494
+
**Cause**: No active authorization for this (aggregator, community) pair
497
+
1. Check authorizations: `GET /xrpc/social.coves.aggregator.getAuthorizations`
498
+
2. Contact community moderator to request authorization
499
+
3. Verify authorization wasn't disabled
500
+
4. Wait for Jetstream to index authorization (5-10 seconds)
502
+
#### Error: "RateLimitExceeded"
504
+
**Cause**: Exceeded 10 posts/hour to this community
507
+
1. Wait for the rate limit window to reset
508
+
2. Batch posts to stay under limit
509
+
3. Distribute posts across multiple communities
510
+
4. Implement posting queue in your aggregator
512
+
### Service Declaration Not Appearing
514
+
**Symptoms**: Service declaration created but not in `aggregators` table
517
+
1. Wait 5-10 seconds for Jetstream to index
518
+
2. Check Jetstream consumer logs for errors
519
+
3. Verify record was created: Check PDS at `at://your-did/social.coves.aggregator.service/self`
520
+
4. Verify `$type` field is exactly `"social.coves.aggregator.service"`
521
+
5. Check `displayName` is not empty (required field)
525
+
### Registration Endpoint
527
+
**`POST /xrpc/social.coves.aggregator.register`**
532
+
did: string // DID of aggregator (did:plc or did:web)
533
+
domain: string // Domain serving .well-known/atproto-did
540
+
did: string // Registered DID
541
+
handle: string // Handle from DID document
542
+
message: string // Next steps message
547
+
- `InvalidDID`: DID format invalid
548
+
- `DomainVerificationFailed`: .well-known verification failed
549
+
- `AlreadyRegistered`: DID already registered
550
+
- `DIDResolutionFailed`: Could not resolve DID
552
+
### Query Endpoints
554
+
**`GET /xrpc/social.coves.aggregator.getServices`**
556
+
Get aggregator service details.
559
+
- `dids`: Array of DIDs (comma-separated)
561
+
**`GET /xrpc/social.coves.aggregator.getAuthorizations`**
563
+
List communities that authorized an aggregator.
566
+
- `aggregatorDid`: Aggregator DID
567
+
- `enabledOnly`: Filter to enabled only (default: false)
569
+
**`GET /xrpc/social.coves.aggregator.listForCommunity`**
571
+
List aggregators authorized by a community.
574
+
- `communityDid`: Community DID
575
+
- `enabledOnly`: Filter to enabled only (default: false)
579
+
- [Aggregator PRD](PRD_AGGREGATORS.md) - Architecture and design decisions
580
+
- [atProto Guide](../../ATPROTO_GUIDE.md) - atProto fundamentals
581
+
- [Communities PRD](../PRD_COMMUNITIES.md) - Community system overview
582
+
- [Setup Scripts README](../../scripts/aggregator-setup/README.md) - Script documentation
586
+
For issues or questions:
588
+
1. Check this guide's troubleshooting section
589
+
2. Review the PRD and architecture docs
590
+
3. Check Coves GitHub issues
591
+
4. Ask in Coves developer community
bretton.dev