A community based topic aggregation platform built on atproto
bretton.dev
1# Aggregator Setup Guide
2
3This guide explains how to set up and register an aggregator with Coves instances.
4
5## Table of Contents
6
7- [Overview](#overview)
8- [Architecture](#architecture)
9- [Prerequisites](#prerequisites)
10- [Quick Start](#quick-start)
11- [Detailed Setup Steps](#detailed-setup-steps)
12- [Authorization Process](#authorization-process)
13- [Posting to Communities](#posting-to-communities)
14- [Rate Limits](#rate-limits)
15- [Security Best Practices](#security-best-practices)
16- [Troubleshooting](#troubleshooting)
17- [API Reference](#api-reference)
18
19## Overview
20
21**Aggregators** are automated services that post content to Coves communities. They are similar to Bluesky's feed generators and labelers - self-managed external services that integrate with the platform.
22
23**Key characteristics**:
24- Self-owned: You create and manage your own PDS account
25- Domain-verified: Prove ownership via `.well-known/atproto-did`
26- Community-authorized: Moderators grant posting permission per-community
27- Rate-limited: 10 posts per hour per community
28
29**Example use cases**:
30- RSS feed aggregators (tech news, blog posts)
31- Social media cross-posters (Twitter → Coves)
32- Event notifications (GitHub releases, weather alerts)
33- Content curation bots (daily links, summaries)
34
35## Architecture
36
37### Data Flow
38
39```
40┌──────────────────────────────────────────────────────────┐
41│ 1. One-Time Setup │
42├──────────────────────────────────────────────────────────┤
43│ Aggregator creates PDS account │
44│ ↓ │
45│ Proves domain ownership (.well-known) │
46│ ↓ │
47│ Registers with Coves (enters users table) │
48│ ↓ │
49│ Writes service declaration │
50│ ↓ │
51│ Jetstream indexes into aggregators table │
52└──────────────────────────────────────────────────────────┘
53
54┌──────────────────────────────────────────────────────────┐
55│ 2. Per-Community Authorization │
56├──────────────────────────────────────────────────────────┤
57│ Moderator writes authorization record │
58│ ↓ │
59│ Jetstream indexes into aggregator_authorizations │
60└──────────────────────────────────────────────────────────┘
61
62┌──────────────────────────────────────────────────────────┐
63│ 3. Posting (Ongoing) │
64├──────────────────────────────────────────────────────────┤
65│ Aggregator calls post creation endpoint │
66│ ↓ │
67│ Handler validates: │
68│ - Author in users table ✓ │
69│ - Author in aggregators table ✓ │
70│ - Authorization exists ✓ │
71│ - Rate limit not exceeded ✓ │
72│ ↓ │
73│ Post written to community's PDS │
74│ ↓ │
75│ Jetstream indexes post │
76└──────────────────────────────────────────────────────────┘
77```
78
79### Database Tables
80
81**users** - All actors (users, communities, aggregators)
82```sql
83CREATE TABLE users (
84 did TEXT PRIMARY KEY,
85 handle TEXT NOT NULL,
86 pds_url TEXT,
87 indexed_at TIMESTAMPTZ
88);
89```
90
91**aggregators** - Aggregator-specific metadata
92```sql
93CREATE TABLE aggregators (
94 did TEXT PRIMARY KEY,
95 display_name TEXT NOT NULL,
96 description TEXT,
97 avatar_url TEXT,
98 config_schema JSONB,
99 source_url TEXT,
100 maintainer_did TEXT,
101 record_uri TEXT NOT NULL UNIQUE,
102 record_cid TEXT NOT NULL,
103 created_at TIMESTAMPTZ,
104 indexed_at TIMESTAMPTZ
105);
106```
107
108**aggregator_authorizations** - Community authorizations
109```sql
110CREATE TABLE aggregator_authorizations (
111 id BIGSERIAL PRIMARY KEY,
112 aggregator_did TEXT NOT NULL,
113 community_did TEXT NOT NULL,
114 enabled BOOLEAN NOT NULL DEFAULT true,
115 config JSONB,
116 created_by TEXT,
117 record_uri TEXT NOT NULL UNIQUE,
118 record_cid TEXT NOT NULL,
119 UNIQUE(aggregator_did, community_did)
120);
121```
122
123## Prerequisites
124
1251. **Domain ownership**: You must own a domain where you can host static files over HTTPS
1262. **Web server**: Ability to serve the `.well-known/atproto-did` file
1273. **Development tools**: `curl`, `jq`, basic shell scripting knowledge
1284. **Email address**: For creating the PDS account
129
130**Optional**:
131- Custom avatar image (PNG/JPEG/WebP, max 1MB)
132- GitHub repository for source code transparency
133
134## Quick Start
135
136We provide automated setup scripts:
137
138```bash
139cd scripts/aggregator-setup
140
141# Make scripts executable
142chmod +x *.sh
143
144# Run setup scripts in order
145./1-create-pds-account.sh
146./2-setup-wellknown.sh
147# (Upload .well-known to your web server)
148./3-register-with-coves.sh
149./4-create-service-declaration.sh
150```
151
152See [scripts/aggregator-setup/README.md](../../scripts/aggregator-setup/README.md) for detailed script documentation.
153
154## Detailed Setup Steps
155
156### Step 1: Create PDS Account
157
158Your aggregator needs its own atProto identity (DID). The easiest way is to create an account on an existing PDS.
159
160**Using an existing PDS (recommended)**:
161
162```bash
163curl -X POST https://bsky.social/xrpc/com.atproto.server.createAccount \
164 -H "Content-Type: application/json" \
165 -d '{
166 "handle": "mynewsbot.bsky.social",
167 "email": "bot@example.com",
168 "password": "secure-password-here"
169 }'
170```
171
172**Response**:
173```json
174{
175 "accessJwt": "eyJ...",
176 "refreshJwt": "eyJ...",
177 "handle": "mynewsbot.bsky.social",
178 "did": "did:plc:abc123...",
179 "didDoc": {...}
180}
181```
182
183**Save these credentials securely!** You'll need the DID and access token for all subsequent operations.
184
185**Alternative**: Run your own PDS or use `did:web` (advanced).
186
187### Step 2: Prove Domain Ownership
188
189To register with Coves, you must prove you own a domain by serving your DID at `https://yourdomain.com/.well-known/atproto-did`.
190
191**Create the file**:
192
193```bash
194mkdir -p .well-known
195echo "did:plc:abc123..." > .well-known/atproto-did
196```
197
198**Upload to your web server** so it's accessible at:
199```
200https://rss-bot.example.com/.well-known/atproto-did
201```
202
203**Verify it works**:
204```bash
205curl https://rss-bot.example.com/.well-known/atproto-did
206# Should return: did:plc:abc123...
207```
208
209**Nginx configuration example**:
210```nginx
211location /.well-known/atproto-did {
212 alias /var/www/.well-known/atproto-did;
213 default_type text/plain;
214 add_header Access-Control-Allow-Origin *;
215}
216```
217
218### Step 3: Register with Coves
219
220Call the registration endpoint to register your aggregator DID with the Coves instance.
221
222**Endpoint**: `POST /xrpc/social.coves.aggregator.register`
223
224**Request**:
225```bash
226curl -X POST https://api.coves.social/xrpc/social.coves.aggregator.register \
227 -H "Content-Type: application/json" \
228 -d '{
229 "did": "did:plc:abc123...",
230 "domain": "rss-bot.example.com"
231 }'
232```
233
234**Response** (Success):
235```json
236{
237 "did": "did:plc:abc123...",
238 "handle": "mynewsbot.bsky.social",
239 "message": "Aggregator registered successfully. Next step: create a service declaration record at at://did:plc:abc123.../social.coves.aggregator.service/self"
240}
241```
242
243**What happens**:
2441. Coves fetches `https://rss-bot.example.com/.well-known/atproto-did`
2452. Verifies it contains your DID
2463. Resolves your DID to get handle and PDS URL
2474. Inserts you into the `users` table
248
249**You're now registered!** But you need to create a service declaration next.
250
251### Step 4: Create Service Declaration
252
253Write a `social.coves.aggregator.service` record to your repository. This contains metadata about your aggregator and gets indexed by Coves' Jetstream consumer.
254
255**Endpoint**: `POST https://your-pds.com/xrpc/com.atproto.repo.createRecord`
256
257**Request**:
258```bash
259# Note: This calls the PDS directly, so it uses Bearer authorization (not DPoP)
260curl -X POST https://bsky.social/xrpc/com.atproto.repo.createRecord \
261 -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
262 -H "Content-Type: application/json" \
263 -d '{
264 "repo": "did:plc:abc123...",
265 "collection": "social.coves.aggregator.service",
266 "rkey": "self",
267 "record": {
268 "$type": "social.coves.aggregator.service",
269 "did": "did:plc:abc123...",
270 "displayName": "RSS News Aggregator",
271 "description": "Aggregates tech news from various RSS feeds",
272 "sourceUrl": "https://github.com/yourname/rss-aggregator",
273 "maintainer": "did:plc:your-personal-did",
274 "createdAt": "2024-01-15T12:00:00Z"
275 }
276 }'
277```
278
279**Response**:
280```json
281{
282 "uri": "at://did:plc:abc123.../social.coves.aggregator.service/self",
283 "cid": "bafyrei..."
284}
285```
286
287**Optional fields**:
288- `avatar`: Blob reference to avatar image
289- `configSchema`: JSON Schema for community-specific configuration
290
291**Wait 5-10 seconds** for Jetstream to index your service declaration into the `aggregators` table.
292
293## Authorization Process
294
295Before you can post to a community, a moderator must authorize your aggregator.
296
297### How Authorization Works
298
2991. **Moderator decision**: Community moderator evaluates your aggregator
3002. **Authorization record**: Moderator writes `social.coves.aggregator.authorization` to community's repo
3013. **Jetstream indexing**: Record gets indexed into `aggregator_authorizations` table
3024. **Posting enabled**: You can now post to that community
303
304### Authorization Record Structure
305
306**Location**: `at://{community_did}/social.coves.aggregator.authorization/{rkey}`
307
308**Example**:
309```json
310{
311 "$type": "social.coves.aggregator.authorization",
312 "aggregatorDid": "did:plc:abc123...",
313 "communityDid": "did:plc:community123...",
314 "enabled": true,
315 "createdBy": "did:plc:moderator...",
316 "createdAt": "2024-01-15T12:00:00Z",
317 "config": {
318 "maxPostsPerHour": 5,
319 "allowedCategories": ["tech", "news"]
320 }
321}
322```
323
324### Checking Your Authorizations
325
326**Endpoint**: `GET /xrpc/social.coves.aggregator.getAuthorizations`
327
328```bash
329curl "https://api.coves.social/xrpc/social.coves.aggregator.getAuthorizations?aggregatorDid=did:plc:abc123...&enabledOnly=true"
330```
331
332**Response**:
333```json
334{
335 "authorizations": [
336 {
337 "aggregatorDid": "did:plc:abc123...",
338 "communityDid": "did:plc:community123...",
339 "communityHandle": "~tech@coves.social",
340 "enabled": true,
341 "createdAt": "2024-01-15T12:00:00Z",
342 "config": {...}
343 }
344 ]
345}
346```
347
348## Posting to Communities
349
350Once authorized, you can post to communities using the standard post creation endpoint.
351
352### Create Post
353
354**Endpoint**: `POST /xrpc/social.coves.community.post.create`
355
356**Request**:
357```bash
358# Note: This calls the Coves API, so it uses DPoP authorization
359curl -X POST https://api.coves.social/xrpc/social.coves.community.post.create \
360 -H "Authorization: DPoP YOUR_ACCESS_TOKEN" \
361 -H "Content-Type: application/json" \
362 -d '{
363 "communityDid": "did:plc:community123...",
364 "post": {
365 "text": "New blog post: Understanding atProto Identity\nhttps://example.com/post",
366 "createdAt": "2024-01-15T12:00:00Z",
367 "facets": [
368 {
369 "index": { "byteStart": 50, "byteEnd": 75 },
370 "features": [
371 {
372 "$type": "social.coves.richtext.facet#link",
373 "uri": "https://example.com/post"
374 }
375 ]
376 }
377 ]
378 }
379 }'
380```
381
382**Response**:
383```json
384{
385 "uri": "at://did:plc:abc123.../social.coves.community.post/3k...",
386 "cid": "bafyrei..."
387}
388```
389
390### Post Validation
391
392The handler validates:
3931. **Authentication**: Valid JWT token
3942. **Author exists**: DID in `users` table
3953. **Is aggregator**: DID in `aggregators` table
3964. **Authorization**: Active authorization for (aggregator, community)
3975. **Rate limit**: Less than 10 posts/hour to this community
3986. **Content**: Valid post structure per lexicon
399
400### Rate Limits
401
402**Per-community rate limit**: 10 posts per hour
403
404This is tracked in the `aggregator_posts` table and enforced at the handler level.
405
406**Why?**: Prevents spam while allowing useful bot activity.
407
408**Best practices**:
409- Batch similar content
410- Post only high-quality content
411- Respect community guidelines
412- Monitor your posting rate
413
414## Security Best Practices
415
416### Credential Management
417
418✅ **DO**:
419- Store credentials in environment variables or secret management
420- Use HTTPS for all API calls
421- Rotate access tokens regularly (use refresh tokens)
422- Keep `aggregator-config.env` out of version control
423
424❌ **DON'T**:
425- Hardcode credentials in source code
426- Commit credentials to Git
427- Share access tokens publicly
428- Reuse personal credentials for bots
429
430### Domain Security
431
432✅ **DO**:
433- Use HTTPS for `.well-known` endpoint
434- Keep domain under your control
435- Monitor for unauthorized changes
436- Use DNSSEC if possible
437
438❌ **DON'T**:
439- Use HTTP (will fail verification)
440- Use shared/untrusted hosting
441- Allow others to modify `.well-known` files
442- Use expired SSL certificates
443
444### Content Security
445
446✅ **DO**:
447- Validate all external content before posting
448- Sanitize URLs and text
449- Rate-limit your own posting
450- Implement circuit breakers for failures
451
452❌ **DON'T**:
453- Post unvalidated user input
454- Include malicious links
455- Spam communities
456- Bypass rate limits
457
458## Troubleshooting
459
460### Registration Errors
461
462#### Error: "DomainVerificationFailed"
463
464**Cause**: `.well-known/atproto-did` not accessible or contains wrong DID
465
466**Solutions**:
4671. Verify file is accessible: `curl https://yourdomain.com/.well-known/atproto-did`
4682. Check content matches your DID exactly (no extra whitespace)
4693. Ensure HTTPS is working (not HTTP)
4704. Check web server logs for access errors
4715. Verify firewall rules allow HTTPS traffic
472
473#### Error: "AlreadyRegistered"
474
475**Cause**: This DID is already registered with this Coves instance
476
477**Solutions**:
478- This is safe to ignore if you're re-running setup
479- If you need to update info, just create a new service declaration
480- Contact instance admin if you need to remove registration
481
482#### Error: "DIDResolutionFailed"
483
484**Cause**: Could not resolve DID document from PLC directory
485
486**Solutions**:
4871. Verify DID exists: `curl https://plc.directory/{your-did}`
4882. Wait 30 seconds and retry (PLC propagation delay)
4893. Check PDS is accessible
4904. Verify DID format is correct (must start with `did:plc:` or `did:web:`)
491
492### Posting Errors
493
494#### Error: "NotAuthorized"
495
496**Cause**: No active authorization for this (aggregator, community) pair
497
498**Solutions**:
4991. Check authorizations: `GET /xrpc/social.coves.aggregator.getAuthorizations`
5002. Contact community moderator to request authorization
5013. Verify authorization wasn't disabled
5024. Wait for Jetstream to index authorization (5-10 seconds)
503
504#### Error: "RateLimitExceeded"
505
506**Cause**: Exceeded 10 posts/hour to this community
507
508**Solutions**:
5091. Wait for the rate limit window to reset
5102. Batch posts to stay under limit
5113. Distribute posts across multiple communities
5124. Implement posting queue in your aggregator
513
514### Service Declaration Not Appearing
515
516**Symptoms**: Service declaration created but not in `aggregators` table
517
518**Solutions**:
5191. Wait 5-10 seconds for Jetstream to index
5202. Check Jetstream consumer logs for errors
5213. Verify record was created: Check PDS at `at://your-did/social.coves.aggregator.service/self`
5224. Verify `$type` field is exactly `"social.coves.aggregator.service"`
5235. Check `displayName` is not empty (required field)
524
525## API Reference
526
527### Registration Endpoint
528
529**`POST /xrpc/social.coves.aggregator.register`**
530
531**Input**:
532```typescript
533{
534 did: string // DID of aggregator (did:plc or did:web)
535 domain: string // Domain serving .well-known/atproto-did
536}
537```
538
539**Output**:
540```typescript
541{
542 did: string // Registered DID
543 handle: string // Handle from DID document
544 message: string // Next steps message
545}
546```
547
548**Errors**:
549- `InvalidDID`: DID format invalid
550- `DomainVerificationFailed`: .well-known verification failed
551- `AlreadyRegistered`: DID already registered
552- `DIDResolutionFailed`: Could not resolve DID
553
554### Query Endpoints
555
556**`GET /xrpc/social.coves.aggregator.getServices`**
557
558Get aggregator service details.
559
560**Parameters**:
561- `dids`: Array of DIDs (comma-separated)
562
563**`GET /xrpc/social.coves.aggregator.getAuthorizations`**
564
565List communities that authorized an aggregator.
566
567**Parameters**:
568- `aggregatorDid`: Aggregator DID
569- `enabledOnly`: Filter to enabled only (default: false)
570
571**`GET /xrpc/social.coves.aggregator.listForCommunity`**
572
573List aggregators authorized by a community.
574
575**Parameters**:
576- `communityDid`: Community DID
577- `enabledOnly`: Filter to enabled only (default: false)
578
579## Further Reading
580
581- [Aggregator PRD](PRD_AGGREGATORS.md) - Architecture and design decisions
582- [atProto Guide](../../ATPROTO_GUIDE.md) - atProto fundamentals
583- [Communities PRD](../PRD_COMMUNITIES.md) - Community system overview
584- [Setup Scripts README](../../scripts/aggregator-setup/README.md) - Script documentation
585
586## Support
587
588For issues or questions:
589
5901. Check this guide's troubleshooting section
5912. Review the PRD and architecture docs
5923. Check Coves GitHub issues
5934. Ask in Coves developer community