Communities PRD: Federated Forum System#

Status: In Development Owner: Platform Team Last Updated: 2025-10-17

Overview#

Coves communities are federated, instance-scoped forums built on atProto. Each community is identified by a scoped handle (!gaming@coves.social) and owns its own atProto repository, enabling true portability and decentralized governance.

Architecture Evolution#

✅ V2 Architecture (Current - 2025-10-10)#

Communities own their own repositories:

Each community has its own DID (did:plc:xxx)
Each community owns its own atProto repository (at://community_did/...)
Each community has its own PDS account (managed by Coves backend)
Communities are truly portable - can migrate between instances by updating DID document

Repository Structure:

Repository:  at://did:plc:community789/social.coves.community.profile/self
Owner:       did:plc:community789 (community owns itself)
Hosted By:   did:web:coves.social (instance manages credentials)

Key Benefits:

✅ True atProto compliance (matches feed generators, labelers)
✅ Portable URIs (never change when migrating instances)
✅ Self-owned identity model
✅ Standard rkey="self" for singleton profiles

✅ Completed Features (Updated 2025-10-17)#

Core Infrastructure#

V2 Architecture: Communities own their own repositories
PDS Account Provisioning: Automatic account creation for each community
Credential Management: Secure storage of community PDS credentials
Token Refresh: Automatic refresh of expired access tokens (completed 2025-10-17)
Encryption at Rest: PostgreSQL pgcrypto for sensitive credentials
Write-Forward Pattern: Service → PDS → Firehose → AppView
Jetstream Consumer: Real-time indexing from firehose
V2 Validation: Strict rkey="self" enforcement (no V1 compatibility)

Security & Data Protection#

Encrypted Credentials: Access/refresh tokens encrypted in database
Credential Persistence: PDS credentials survive server restarts
Automatic Token Refresh: Tokens refresh 5 minutes before expiration (completed 2025-10-17)
Password Fallback: Re-authentication when refresh tokens expire
Concurrency Safety: Per-community mutex prevents refresh race conditions
JSON Exclusion: Credentials never exposed in API responses (json:"-" tags)
Password Encryption: Encrypted (not hashed) for session creation fallback
Timeout Handling: 30s timeout for write operations, 10s for reads

Database Schema#

Communities Table: Full metadata with V2 credential columns
Subscriptions Table: Lightweight feed following
Memberships Table: Active participation tracking
Moderation Table: Local moderation actions
Encryption Keys Table: Secure key management for pgcrypto
Indexes: Optimized for search, visibility filtering, and lookups

Service Layer#

CreateCommunity: Provisions PDS account, creates record, persists credentials
UpdateCommunity: Uses community's own credentials (not instance credentials)
GetCommunity: Fetches from AppView DB with decrypted credentials
ListCommunities: Pagination, filtering, sorting
SearchCommunities: Full-text search on name/description
Subscribe/Unsubscribe: Create subscription records
Handle Validation: Scoped handle format (!name@instance)
DID Generation: Uses did:plc for portability

Jetstream Consumer#

Profile Events: Create, update, delete community profiles
Subscription Events: Index user subscriptions to communities
V2 Enforcement: Reject non-"self" rkeys (no V1 communities)
Self-Ownership Validation: Verify owner_did == did
Error Handling: Graceful handling of malformed events

Testing Coverage#

Integration Tests: Full CRUD operations
Credential Tests: Persistence, encryption, decryption
Token Refresh Tests: JWT parsing, credential updates, concurrency (completed 2025-10-17)
V2 Validation Tests: Rkey enforcement, self-ownership
Consumer Tests: Firehose event processing
Repository Tests: Database operations
Unit Tests: Service layer logic, timeout handling

🚧 In Progress / Needs Testing#

XRPC API Endpoints#

Status: All core endpoints E2E tested! ✅

✅ E2E Tested (via community_e2e_test.go):

social.coves.community.create - Full E2E test with real PDS
social.coves.community.get - E2E test validates HTTP endpoint
social.coves.community.list - E2E test with pagination/filtering
social.coves.community.update - E2E test verifies write-forward + PDS update
social.coves.community.subscribe - E2E test verifies subscription in user's repo
social.coves.community.unsubscribe - E2E test verifies PDS deletion

📍 Post-Alpha:

social.coves.community.search - Handler exists, defer E2E testing to post-alpha

✅ OAuth Authentication Complete (2025-10-16):

User access tokens now flow through middleware → handlers → service
Subscribe/unsubscribe operations use correct user-scoped credentials
All E2E tests validate real PDS authentication with user tokens

⚠️ Alpha Blockers (Must Complete Before Alpha Launch)#

Critical Missing Features#

Community Blocking: ✅ COMPLETE - Users can block communities from their feeds
- ✅ Lexicon: social.coves.community.block record type implemented
- ✅ Service: BlockCommunity() / UnblockCommunity() / GetBlockedCommunities() / IsBlocked()
- ✅ Handlers: Block/unblock endpoints implemented
- ✅ Repository: Full blocking methods with indexing
- ✅ Jetstream Consumer: Real-time indexing of block events
- ✅ Integration tests: Comprehensive coverage
- Completed: 2025-10-16
- Impact: Users can now hide unwanted communities from their feeds

✅ Critical Infrastructure - RESOLVED (2025-10-16)#

✅ Subscription Indexing & ContentVisibility - COMPLETE
- Status: Subscriptions now fully indexed in AppView with feed slider support
- Completed: 2025-10-16
- What Was Fixed:
  1. ✅ Fixed critical collection name bug (social.coves.actor.subscription → social.coves.community.subscription)
  2. ✅ Implemented ContentVisibility (1-5 slider) across all layers (handler, service, consumer, repository)
  3. ✅ Production Jetstream consumer now running (cmd/server/main.go:220-243)
  4. ✅ Migration 008 adds content_visibility column with defaults and constraints
  5. ✅ Atomic subscriber count updates (SubscribeWithCount/UnsubscribeWithCount)
  6. ✅ DELETE operations properly handled (unsubscribe indexing)
  7. ✅ Idempotent operations (safe for Jetstream event replays)
  8. ✅ atProto naming compliance: singular namespace + subject field
- Impact:
  - ✅ Users CAN subscribe/unsubscribe (writes to their PDS repo)
  - ✅ AppView INDEXES subscriptions from Jetstream in real-time
  - ✅ Can query user's subscriptions (data persisted with contentVisibility)
  - ✅ Feed generation ENABLED (know who's subscribed with visibility preferences)
  - ✅ Subscriber counts accurate (atomic updates)
- Testing:
  - ✅ 13 comprehensive integration tests (subscription_indexing_test.go) - ALL PASSING
  - ✅ Enhanced E2E tests verify complete flow (HTTP → PDS → Jetstream → AppView)
  - ✅ ContentVisibility clamping tested (0→1, 10→5, defaults to 3)
  - ✅ Idempotency verified (duplicate events handled gracefully)
- Files:
  - Implementation Doc: docs/IMPLEMENTATION_SUBSCRIPTION_INDEXING.md
  - Lexicon: internal/atproto/lexicon/social/coves/community/subscription.json
  - Consumer: internal/atproto/jetstream/community_consumer.go
  - Connector: internal/atproto/jetstream/community_jetstream_connector.go
  - Migration: internal/db/migrations/008_add_content_visibility_to_subscriptions.sql
  - Tests: tests/integration/subscription_indexing_test.go

Critical Security (High Priority)#

OAuth Authentication: ✅ COMPLETE - User access tokens flow end-to-end
- ✅ Middleware stores user access token in context
- ✅ Handlers extract and pass token to service
- ✅ Service uses user token for user repo operations (subscribe/unsubscribe)
- ✅ All E2E tests pass with real PDS authentication
- Completed: 2025-10-16
Token Refresh Logic: ✅ COMPLETE - Auto-refresh expired PDS access tokens
- ✅ Automatic token refresh before PDS operations (5-minute buffer)
- ✅ Password fallback when refresh tokens expire (~2 months)
- ✅ Concurrency-safe with per-community mutex locking
- ✅ Atomic credential updates in database
- ✅ Integration tests and structured logging
- Completed: 2025-10-17
- See: IMPLEMENTATION_TOKEN_REFRESH.md

📍 Beta Features (High Priority - Post Alpha)#

Content Rules System#

Status: Lexicon complete (2025-10-18), implementation TODO Priority: CRITICAL for Alpha - Enables community content policies

Communities can define content posting restrictions via the contentRules object in their profile:

Key Features:

✅ Lexicon: contentRules object defined in social.coves.community.profile
Validation: Post creation validates against community rules
AppView indexing: Index post characteristics (embed_type, text_length)
Error handling: Clear ContentRuleViolation errors

Example Use Cases:

Text-only communities: "AskCoves" requires text, blocks all embeds
Image communities: "CovesPics" requires at least 1 image
No restrictions: General communities allow all content types

See: PRD_GOVERNANCE.md - Content Rules System for full details

Blob Upload Proxy System#

Status: Design documented, implementation TODO Priority: CRITICAL for Beta - Required for image/video posts in communities

Problem: Users on external PDSs cannot directly upload blobs to community-owned PDS repositories because they lack authentication credentials for the community's PDS.

Solution: Coves AppView acts as an authenticated proxy for blob uploads:

Flow:

User uploads blob to Coves AppView via social.coves.blob.uploadForCommunity
AppView validates user can post to community (not banned, community accessible)
AppView uses community's PDS credentials to upload blob via com.atproto.repo.uploadBlob
AppView returns CID to user
User creates post record referencing the CID
Post and blob both live in community's PDS

Implementation Checklist:

Handler: social.coves.blob.uploadForCommunity endpoint
Validation: Check user authorization to post in community
Credential Management: Reuse community token refresh logic
Upload Proxy: Forward blob to community's PDS with community credentials
Security: Size limits, content-type validation, rate limiting
Testing: E2E test with federated user uploading to community

Why This Approach:

✅ Works with federated users (any PDS)
✅ Reuses existing community credential infrastructure
✅ Matches V2 architecture (AppView orchestrates, communities own data)
✅ Blobs stored on correct PDS (community's repository)
❌ AppView becomes upload intermediary (bandwidth cost)

Alternative Considered: Direct user uploads to community PDS

Rejected: Would require creating temporary user accounts on every community PDS (complex, insecure)

See: Design discussion in context of ATProto blob architecture

Posts in Communities#

Status: Lexicon designed, implementation TODO Priority: HIGHEST for Beta 1

social.coves.post already has community field ✅
Removed postType enum in favor of content rules ✅ (2025-10-18)
Create post endpoint with content rules validation
Feed generation for community posts
Post consumer (index community posts from firehose)
Community post count tracking
Decide membership requirements for posting

Without posts, communities exist but can't be used!

Depends on: Blob Upload Proxy System (for image/video posts)

📍 Beta Features (Lower Priority)#

Membership System#

Status: Lexicon exists, design decisions needed Deferred: Answer design questions before implementing

Decide: Auto-join on first post vs explicit join?
Decide: Reputation tracking in lexicon vs AppView only?
Implement membership record creation (if explicit join)
Member lists endpoint
Reputation tracking (if in lexicon)

Community Management#

Community Deletion: Soft delete / permanent delete
Wiki System: Lexicon exists, no implementation
Advanced Rules: Separate rules records, moderation config
Moderator Management: Assign/remove moderators (governance work)
Categories: REMOVE from lexicon and code (not needed)

User Features#

Saved Items: Save posts/comments for later
User Flairs: Per-community user flair (design TBD)

Instance Moderation#

Delist Community: Remove from search/directory
Quarantine Community: Show warning label
Remove Community: Hide from instance AppView
Moderation Audit Log: Track all moderation actions

⏳ TODO Before V1 Production Launch#

Community Discovery & Visibility#

Visibility Enforcement: Respect public/unlisted/private settings in listings
Federation Config: Honor allowExternalDiscovery flag
Search Relevance: Implement ranking algorithm (members, activity, etc.)
Directory Endpoint: Public community directory with filters
Rate Limiting: Prevent community creation spam (e.g., 5 per user per hour)
Handle Collision Detection: Prevent duplicate community handles
DID Validation: Verify DIDs before accepting create requests

Token Refresh & Resilience#

Retry Mechanism: Retry failed PDS calls with backoff
Credential Rotation: Periodic password rotation for security
Error Recovery: Graceful degradation if PDS is unavailable

Performance & Scaling#

Database Indexes: Verify all common queries are indexed
Query Optimization: Review N+1 query patterns
Caching Strategy: Cache frequently accessed communities
Pagination Limits: Enforce max results per request
Connection Pooling: Optimize PDS HTTP client reuse

Documentation & Deployment#

API Documentation: OpenAPI/Swagger specs for all endpoints
Deployment Guide: Production setup instructions
Migration Guide: How to upgrade from test to production
Monitoring Guide: Metrics and alerting setup
Security Checklist: Pre-launch security audit

Infrastructure & DNS#

DNS Wildcard Setup: Configure *.community.coves.social for community handle resolution
Well-Known Endpoint: Implement .well-known/atproto-did handler for *.community.coves.social subdomains

Out of Scope (Future Versions)#

V3: Federation & Discovery#

Cross-instance community search
Federated moderation signals
Trust networks between instances
Moderation signal subscription

V4: Community Governance#

Community-owned governance (voting on moderators)
Migration voting (community votes to move instances)
Custom domain DIDs (did:web:gaming.community)
Governance thresholds and time locks

Recent Critical Fixes (2025-10-10)#

Security & Credential Management#

Issue: PDS credentials were created but never persisted Fix: Service layer now immediately persists credentials via repo.Create() Impact: Communities can now be updated after creation (credentials survive restarts)

Issue: Credentials stored in plaintext in PostgreSQL Fix: Added pgcrypto encryption for access/refresh tokens Impact: Database compromise no longer exposes active tokens

Issue: UpdateCommunity used instance credentials instead of community credentials Fix: Changed to use existing.DID and existing.PDSAccessToken Impact: Updates now correctly authenticate as the community itself

V2 Architecture Enforcement#

Issue: Consumer accepted V1 communities with TID-based rkeys Fix: Strict validation - only rkey="self" accepted Impact: No legacy V1 data in production

Issue: PDS write operations timed out (10s too short) Fix: Dynamic timeout - writes get 30s, reads get 10s Impact: Community creation no longer fails on slow PDS operations

Lexicon Summary#

`social.coves.community.profile`#

Status: ✅ Implemented and tested

Required Fields:

handle - atProto handle (DNS-resolvable, e.g., gaming.community.coves.social)
name - Short community name for !mentions (e.g., gaming)
createdBy - DID of user who created community
hostedBy - DID of hosting instance
visibility - "public", "unlisted", or "private"
federation.allowExternalDiscovery - Boolean

Note: The !gaming@coves.social format is derived client-side from name + instance for UI display. The handle field contains only the DNS-resolvable atProto handle.

Optional Fields:

displayName - Display name for UI
description - Community description
descriptionFacets - Rich text annotations
avatar - Blob reference for avatar image
banner - Blob reference for banner image
moderationType - "moderator" or "sortition"
contentWarnings - Array of content warning types
contentRules - Content posting restrictions (see PRD_GOVERNANCE.md)
- allowedEmbedTypes - Array of allowed embed types (images, video, external, record)
- requireText - Whether text content is required
- minTextLength / maxTextLength - Text length constraints
- requireTitle - Whether title is required
- minImages / maxImages - Image count constraints
- allowFederated - Whether federated posts allowed
memberCount - Cached count
subscriberCount - Cached count

`social.coves.community.subscription`#

Status: ✅ Schema exists, consumer TODO

Fields:

community - DID of community being subscribed to
subscribedAt - Timestamp

`social.coves.post` (Community Extension)#

Status: ⏳ TODO

New Field:

community - Optional DID of community this post belongs to

Success Metrics#

Pre-Launch Checklist#

All XRPC endpoints have E2E tests
OAuth authentication working on all protected endpoints
Rate limiting prevents abuse
Communities can be created, updated, searched, and subscribed to
Jetstream consumer indexes events in < 1 second
Database handles 10,000+ communities without performance issues
Security audit completed

V1 Launch Goals#

Communities can be created with scoped handles
Posts can be made to communities (when implemented)
Community discovery works on local instance
All three visibility levels function correctly
Basic moderation (delist/remove) works

Technical Decisions Log#

2025-10-18: Content Rules Over Post Type Enum#

Decision: Remove postType enum from post creation; use flexible contentRules in community profile instead

Rationale:

postType enum forced users to explicitly select type (bad UX - app should infer from structure)
Enum was rigid - couldn't support nuanced rules like "text required, images optional"
Content rules are more extensible (add new constraints without changing post lexicon)
Follows atProto philosophy: describe data structure, not UI intent
Enables both community restrictions ("text only") AND user filtering ("show videos only")

Implementation:

Community profile contains optional contentRules object
Post validation checks structure against rules at creation time
AppView indexes post characteristics (embed_type, text_length) for filtering
Errors use ContentRuleViolation instead of InvalidPostType

Examples:

"AskCoves": allowedEmbedTypes: [] + requireText: true + minTextLength: 50
"CovesPics": allowedEmbedTypes: ["images"] + minImages: 1
General communities: contentRules: null (no restrictions)

Trade-offs Accepted:

Validation logic more complex than simple enum check (but more powerful)
Communities can't programmatically restrict to exact "article" vs "text" types (but structure-based rules are better)

See: PRD_GOVERNANCE.md - Content Rules System

2025-10-11: Single Handle Field (atProto-Compliant)#

Decision: Use single handle field containing DNS-resolvable atProto handle; remove atprotoHandle field