A community based topic aggregation platform built on atproto
bretton.dev
1# Communities PRD: Federated Forum System
2
3**Status:** In Development
4**Owner:** Platform Team
5**Last Updated:** 2025-10-10
6
7## Overview
8
9Coves communities are federated, instance-scoped forums built on atProto. Each community is identified by a scoped handle (`!gaming@coves.social`) and owns its own atProto repository, enabling true portability and decentralized governance.
10
11## Architecture Evolution
12
13### ✅ V2 Architecture (Current - 2025-10-10)
14
15**Communities own their own repositories:**
16- Each community has its own DID (`did:plc:xxx`)
17- Each community owns its own atProto repository (`at://community_did/...`)
18- Each community has its own PDS account (managed by Coves backend)
19- Communities are truly portable - can migrate between instances by updating DID document
20
21**Repository Structure:**
22```
23Repository: at://did:plc:community789/social.coves.community.profile/self
24Owner: did:plc:community789 (community owns itself)
25Hosted By: did:web:coves.social (instance manages credentials)
26```
27
28**Key Benefits:**
29- ✅ True atProto compliance (matches feed generators, labelers)
30- ✅ Portable URIs (never change when migrating instances)
31- ✅ Self-owned identity model
32- ✅ Standard rkey="self" for singleton profiles
33
34---
35
36## ✅ Completed Features (2025-10-10)
37
38### Core Infrastructure
39- [x] **V2 Architecture:** Communities own their own repositories
40- [x] **PDS Account Provisioning:** Automatic account creation for each community
41- [x] **Credential Management:** Secure storage of community PDS credentials
42- [x] **Encryption at Rest:** PostgreSQL pgcrypto for sensitive credentials
43- [x] **Write-Forward Pattern:** Service → PDS → Firehose → AppView
44- [x] **Jetstream Consumer:** Real-time indexing from firehose
45- [x] **V2 Validation:** Strict rkey="self" enforcement (no V1 compatibility)
46
47### Security & Data Protection
48- [x] **Encrypted Credentials:** Access/refresh tokens encrypted in database
49- [x] **Credential Persistence:** PDS credentials survive server restarts
50- [x] **JSON Exclusion:** Credentials never exposed in API responses (`json:"-"` tags)
51- [x] **Password Hashing:** bcrypt for PDS account passwords
52- [x] **Timeout Handling:** 30s timeout for write operations, 10s for reads
53
54### Database Schema
55- [x] **Communities Table:** Full metadata with V2 credential columns
56- [x] **Subscriptions Table:** Lightweight feed following
57- [x] **Memberships Table:** Active participation tracking
58- [x] **Moderation Table:** Local moderation actions
59- [x] **Encryption Keys Table:** Secure key management for pgcrypto
60- [x] **Indexes:** Optimized for search, visibility filtering, and lookups
61
62### Service Layer
63- [x] **CreateCommunity:** Provisions PDS account, creates record, persists credentials
64- [x] **UpdateCommunity:** Uses community's own credentials (not instance credentials)
65- [x] **GetCommunity:** Fetches from AppView DB with decrypted credentials
66- [x] **ListCommunities:** Pagination, filtering, sorting
67- [x] **SearchCommunities:** Full-text search on name/description
68- [x] **Subscribe/Unsubscribe:** Create subscription records
69- [x] **Handle Validation:** Scoped handle format (`!name@instance`)
70- [x] **DID Generation:** Uses `did:plc` for portability
71
72### Jetstream Consumer
73- [x] **Profile Events:** Create, update, delete community profiles
74- [x] **Subscription Events:** Index user subscriptions to communities
75- [x] **V2 Enforcement:** Reject non-"self" rkeys (no V1 communities)
76- [x] **Self-Ownership Validation:** Verify owner_did == did
77- [x] **Error Handling:** Graceful handling of malformed events
78
79### Testing Coverage
80- [x] **Integration Tests:** Full CRUD operations
81- [x] **Credential Tests:** Persistence, encryption, decryption
82- [x] **V2 Validation Tests:** Rkey enforcement, self-ownership
83- [x] **Consumer Tests:** Firehose event processing
84- [x] **Repository Tests:** Database operations
85- [x] **Unit Tests:** Service layer logic, timeout handling
86
87---
88
89## 🚧 In Progress / Needs Testing
90
91### XRPC API Endpoints
92**Status:** Handlers exist, need comprehensive E2E testing
93
94- [ ] `social.coves.community.create` - **Handler exists**, needs E2E test with real PDS
95- [ ] `social.coves.community.get` - **Handler exists**, needs E2E test
96- [ ] `social.coves.community.update` - **Handler exists**, needs E2E test with community credentials
97- [ ] `social.coves.community.list` - **Handler exists**, needs E2E test with pagination
98- [ ] `social.coves.community.search` - **Handler exists**, needs E2E test with queries
99- [ ] `social.coves.community.subscribe` - **Handler exists**, needs E2E test
100- [ ] `social.coves.community.unsubscribe` - **Handler exists**, needs E2E test
101
102**What's needed:**
103- E2E tests that verify complete flow: HTTP → Service → PDS → Firehose → Consumer → DB → HTTP response
104- Test with real PDS instance (not mocked)
105- Verify Jetstream consumer picks up events in real-time
106
107### Posts in Communities
108**Status:** Lexicon designed, implementation TODO
109
110- [ ] Extend `social.coves.post` lexicon with `community` field
111- [ ] Create post endpoint (with community membership validation?)
112- [ ] Feed generation for community posts
113- [ ] Post consumer (index community posts from firehose)
114- [ ] Community post count tracking
115
116**What's needed:**
117- Decide membership requirements for posting
118- Design feed generation algorithm
119- Implement post indexing in consumer
120- Add tests for post creation/listing
121
122---
123
124## ⏳ TODO Before V1 Production Launch
125
126### Critical Security & Authorization
127- [ ] **OAuth Middleware:** Protect create/update/delete endpoints
128- [ ] **Authorization Checks:** Verify user is community creator/moderator
129- [ ] **Rate Limiting:** Prevent community creation spam (e.g., 5 per user per hour)
130- [ ] **Handle Collision Detection:** Prevent duplicate community handles
131- [ ] **DID Validation:** Verify DIDs before accepting create requests
132- [ ] **Token Refresh Logic:** Handle expired PDS access tokens
133
134### Community Discovery & Visibility
135- [ ] **Visibility Enforcement:** Respect public/unlisted/private settings in listings
136- [ ] **Federation Config:** Honor `allowExternalDiscovery` flag
137- [ ] **Search Relevance:** Implement ranking algorithm (members, activity, etc.)
138- [ ] **Directory Endpoint:** Public community directory with filters
139
140### Membership & Participation
141- [ ] **Membership Tracking:** Auto-create membership on first post
142- [ ] **Reputation System:** Track user participation per community
143- [ ] **Subscription → Membership Flow:** Define conversion logic
144- [ ] **Member Lists:** Endpoint to list community members
145- [ ] **Moderator Assignment:** Allow creators to add moderators
146
147### Moderation (Basic)
148- [ ] **Delist Community:** Remove from search/directory
149- [ ] **Quarantine Community:** Show warning label
150- [ ] **Remove Community:** Hide from instance AppView
151- [ ] **Moderation Audit Log:** Track all moderation actions
152- [ ] **Admin Endpoints:** Instance operator tools
153
154### Token Refresh & Resilience
155- [ ] **Refresh Token Logic:** Auto-refresh expired PDS access tokens
156- [ ] **Retry Mechanism:** Retry failed PDS calls with backoff
157- [ ] **Credential Rotation:** Periodic password rotation for security
158- [ ] **Error Recovery:** Graceful degradation if PDS is unavailable
159
160### Performance & Scaling
161- [ ] **Database Indexes:** Verify all common queries are indexed
162- [ ] **Query Optimization:** Review N+1 query patterns
163- [ ] **Caching Strategy:** Cache frequently accessed communities
164- [ ] **Pagination Limits:** Enforce max results per request
165- [ ] **Connection Pooling:** Optimize PDS HTTP client reuse
166
167### Documentation & Deployment
168- [ ] **API Documentation:** OpenAPI/Swagger specs for all endpoints
169- [ ] **Deployment Guide:** Production setup instructions
170- [ ] **Migration Guide:** How to upgrade from test to production
171- [ ] **Monitoring Guide:** Metrics and alerting setup
172- [ ] **Security Checklist:** Pre-launch security audit
173
174---
175
176## Out of Scope (Future Versions)
177
178### V3: Federation & Discovery
179- [ ] Cross-instance community search
180- [ ] Federated moderation signals
181- [ ] Trust networks between instances
182- [ ] Moderation signal subscription
183
184### V4: Community Governance
185- [ ] Community-owned governance (voting on moderators)
186- [ ] Migration voting (community votes to move instances)
187- [ ] Custom domain DIDs (`did:web:gaming.community`)
188- [ ] Governance thresholds and time locks
189
190---
191
192## Recent Critical Fixes (2025-10-10)
193
194### Security & Credential Management
195**Issue:** PDS credentials were created but never persisted
196**Fix:** Service layer now immediately persists credentials via `repo.Create()`
197**Impact:** Communities can now be updated after creation (credentials survive restarts)
198
199**Issue:** Credentials stored in plaintext in PostgreSQL
200**Fix:** Added pgcrypto encryption for access/refresh tokens
201**Impact:** Database compromise no longer exposes active tokens
202
203**Issue:** UpdateCommunity used instance credentials instead of community credentials
204**Fix:** Changed to use `existing.DID` and `existing.PDSAccessToken`
205**Impact:** Updates now correctly authenticate as the community itself
206
207### V2 Architecture Enforcement
208**Issue:** Consumer accepted V1 communities with TID-based rkeys
209**Fix:** Strict validation - only rkey="self" accepted
210**Impact:** No legacy V1 data in production
211
212**Issue:** PDS write operations timed out (10s too short)
213**Fix:** Dynamic timeout - writes get 30s, reads get 10s
214**Impact:** Community creation no longer fails on slow PDS operations
215
216---
217
218## Lexicon Summary
219
220### `social.coves.community.profile`
221**Status:** ✅ Implemented and tested
222
223**Required Fields:**
224- `handle` - Scoped handle (`!gaming@coves.social`)
225- `atprotoHandle` - Real atProto handle (`gaming.communities.coves.social`)
226- `name` - Community name
227- `createdBy` - DID of user who created community
228- `hostedBy` - DID of hosting instance
229- `visibility` - `"public"`, `"unlisted"`, or `"private"`
230- `federation.allowExternalDiscovery` - Boolean
231
232**Optional Fields:**
233- `displayName` - Display name for UI
234- `description` - Community description
235- `descriptionFacets` - Rich text annotations
236- `avatar` - Blob reference for avatar image
237- `banner` - Blob reference for banner image
238- `moderationType` - `"moderator"` or `"sortition"`
239- `contentWarnings` - Array of content warning types
240- `memberCount` - Cached count
241- `subscriberCount` - Cached count
242
243### `social.coves.community.subscription`
244**Status:** ✅ Schema exists, consumer TODO
245
246**Fields:**
247- `community` - DID of community being subscribed to
248- `subscribedAt` - Timestamp
249
250### `social.coves.post` (Community Extension)
251**Status:** ⏳ TODO
252
253**New Field:**
254- `community` - Optional DID of community this post belongs to
255
256---
257
258## Success Metrics
259
260### Pre-Launch Checklist
261- [ ] All XRPC endpoints have E2E tests
262- [ ] OAuth authentication working on all protected endpoints
263- [ ] Rate limiting prevents abuse
264- [ ] Communities can be created, updated, searched, and subscribed to
265- [ ] Jetstream consumer indexes events in < 1 second
266- [ ] Database handles 10,000+ communities without performance issues
267- [ ] Security audit completed
268
269### V1 Launch Goals
270- Communities can be created with scoped handles
271- Posts can be made to communities (when implemented)
272- Community discovery works on local instance
273- All three visibility levels function correctly
274- Basic moderation (delist/remove) works
275
276---
277
278## Technical Decisions Log
279
280### 2025-10-10: V2 Architecture Completed
281- Migrated from instance-owned to community-owned repositories
282- Each community now has own PDS account
283- Credentials encrypted at rest using pgcrypto
284- Strict V2 enforcement (no V1 compatibility)
285
286### 2025-10-08: DID Architecture & atProto Compliance
287- Migrated from `did:coves` to `did:plc` (portable DIDs)
288- Added required `did` field to lexicon
289- Fixed critical `record_uri` bug
290- Matches Bluesky feed generator pattern
291
292---
293
294## References
295
296- atProto Lexicon Spec: https://atproto.com/specs/lexicon
297- DID Web Spec: https://w3c-ccg.github.io/did-method-web/
298- Bluesky Handle System: https://atproto.com/specs/handle
299- PLC Directory: https://plc.directory