A community based topic aggregation platform built on atproto
bretton.dev
1# AT Protocol Implementation Guide
2
3This guide provides comprehensive information about implementing AT Protocol (atproto) in the Coves platform.
4
5## Table of Contents
6- [Core Concepts](#core-concepts)
7- [Architecture Overview](#architecture-overview)
8- [Lexicons](#lexicons)
9- [XRPC](#xrpc)
10- [Data Storage](#data-storage)
11- [Identity & Authentication](#identity--authentication)
12- [Firehose & Sync](#firehose--sync)
13- [Go Implementation Patterns](#go-implementation-patterns)
14- [Best Practices](#best-practices)
15
16## Core Concepts
17
18### What is AT Protocol?
19AT Protocol is a federated social networking protocol that enables:
20- **Decentralized identity** - Users own their identity (DID) and can move between providers
21- **Data portability** - Users can export and migrate their full social graph and content
22- **Interoperability** - Different apps can interact with the same underlying data
23
24### Key Components
251. **DIDs (Decentralized Identifiers)** - Persistent user identifiers (e.g., `did:plc:xyz123`)
262. **Handles** - Human-readable names that resolve to DIDs (e.g., `alice.bsky.social`)
273. **Repositories** - User data stored in PDS'
284. **Lexicons** - Schema definitions for data types and API methods
295. **XRPC** - The RPC protocol for client-server communication
306. **Firehose** - Real-time event stream of repository changes
31
32## Architecture Overview
33
34### Coves Architecture Pattern
35Coves uses a simplified, single-database architecture that leverages existing atProto infrastructure:
36
37#### Components
38
391. **PDS (Personal Data Server)**
40 - Managed by Bluesky's official PDS implementation
41 - Handles user repositories, DIDs, and CAR file storage
42 - Users can use our PDS or any external PDS (federated)
43 - Emits events to the Relay/firehose
44
452. **Relay (BigSky)**
46 - Aggregates firehose events from multiple PDSs
47 - For development: subscribes only to local dev PDS
48 - For production: can subscribe to multiple PDSs or public relay
49
503. **AppView Database (Single PostgreSQL)**
51 - **Purpose**: Denormalized, indexed data optimized for Coves queries
52 - **Storage**: PostgreSQL with Coves-specific schema
53 - **Contains**:
54 - Indexed posts, communities, feeds
55 - User read states and preferences
56 - PDS metadata and record references
57 - **Properties**:
58 - Eventually consistent with PDS repositories
59 - Can be rebuilt from firehose replay
60 - Application-specific aggregations
61
624. **Coves AppView (Go Application)**
63 - Subscribes to Relay firehose
64 - Indexes relevant records into PostgreSQL
65 - Serves XRPC queries for Coves features
66 - Implements custom feed algorithms
67
68### Data Flow
69
70```
71Write Path:
72Client → PDS (via XRPC) → Repository Record Created
73 ↓
74 Firehose Event
75 ↓
76 Relay aggregates events
77 ↓
78 Coves AppView subscribes
79 ↓
80 Index in PostgreSQL
81
82Read Path:
83Client → Coves AppView (via XRPC) → PostgreSQL Query → Response
84```
85
86**Key Point**: Coves AppView only reads from the firehose and indexes data. It does NOT write to CAR files or manage repositories directly - the PDS handles that.
87
88## Lexicons
89
90### What are Lexicons?
91Lexicons are JSON schema files that define:
92- Data types (records stored in repositories)
93- API methods (queries and procedures)
94- Input/output schemas for API calls
95
96### Lexicon Structure
97```json
98{
99 "lexicon": 1,
100 "id": "social.coves.community.profile",
101 "defs": {
102 "main": {
103 "type": "record",
104 "key": "self",
105 "record": {
106 "type": "object",
107 "required": ["name", "createdAt"],
108 "properties": {
109 "name": {"type": "string", "maxLength": 64},
110 "description": {"type": "string", "maxLength": 256},
111 "rules": {"type": "array", "items": {"type": "string"}},
112 "createdAt": {"type": "string", "format": "datetime"}
113 }
114 }
115 }
116 }
117}
118```
119
120### Lexicon Types
121
122#### 1. Record Types
123Define data structures stored in user repositories:
124```json
125{
126 "type": "record",
127 "key": "tid|rkey|literal",
128 "record": { /* schema */ }
129}
130```
131
132#### 2. Query Types (Read-only)
133Define read operations that don't modify state:
134```json
135{
136 "type": "query",
137 "parameters": { /* input schema */ },
138 "output": { /* response schema */ }
139}
140```
141
142#### 3. Procedure Types (Write)
143Define operations that modify repositories:
144```json
145{
146 "type": "procedure",
147 "input": { /* request body schema */ },
148 "output": { /* response schema */ }
149}
150```
151
152### Naming Conventions
153- Use reverse-DNS format: `social.coves.community.profile`
154- Queries often start with `get`, `list`, or `search`
155- Procedures often start with `create`, `update`, or `delete`
156- Keep names descriptive but concise
157
158
159## Identity & Authentication
160
161### DIDs (Decentralized Identifiers)
162- Permanent, unique identifiers for users
163- Two types supported:
164 - `did:plc:*` - Hosted by PLC Directory
165 - `did:web:*` - Self-hosted
166
167### Handle Resolution
168Handles resolve to DIDs via:
1691. DNS TXT record: `_atproto.alice.com → did:plc:xyz`
1702. HTTPS well-known: `https://alice.com/.well-known/atproto-did`
171
172### Authentication Flow
1731. Client creates session with OAuth
174
175## Firehose & Sync
176
177### Firehose Events
178Real-time stream of repository changes:
179- Commit events (creates, updates, deletes)
180- Identity events (handle changes)
181- Account events (status changes)
182
183### Subscribing to Firehose
184Connect via WebSocket to `com.atproto.sync.subscribeRepos`:
185```
186wss://bsky.network/xrpc/com.atproto.sync.subscribeRepos
187```
188
189### Processing Events
190- Events include full record data and operation type
191- Process events to update AppView database
192- Handle out-of-order events with sequence numbers
193
194## Go Implementation Patterns
195
196### Using Indigo Library
197Bluesky's official Go implementation provides:
198- Lexicon code generation
199- XRPC client/server
200- Firehose subscription
201
202### Code Generation
203Generate Go types from Lexicons:
204```bash
205go run github.com/bluesky-social/indigo/cmd/lexgen \
206 --package coves \
207 --prefix social.coves \
208 --outdir api/coves \
209 lexicons/social/coves/*.json
210```
211
212### Repository Operations
213```go
214// Write to repository
215rkey := models.GenerateTID()
216err := repoStore.CreateRecord(ctx, userDID, "social.coves.post", rkey, &Post{
217 Text: "Hello",
218 CreatedAt: time.Now().Format(time.RFC3339),
219})
220
221// Read from repository
222records, err := repoStore.ListRecords(ctx, userDID, "social.coves.post", limit, cursor)
223```
224
225### XRPC Handler Pattern
226```go
227func (s *Server) HandleGetCommunity(ctx context.Context) error {
228 // 1. Parse and validate input
229 id := xrpc.QueryParam(ctx, "id")
230
231 // 2. Call service layer
232 community, err := s.communityService.GetByID(ctx, id)
233 if err != nil {
234 return err
235 }
236
237 // 3. Return response
238 return xrpc.WriteJSONResponse(ctx, community)
239}
240```
241
242## Best Practices
243
244### 1. Lexicon Design
245- Keep schemas focused and single-purpose
246- Use references (`$ref`) for shared types
247- Version carefully - Lexicons are contracts
248- Document thoroughly with descriptions
249
250### 2. Data Modeling
251- Store minimal data in repositories
252- Denormalize extensively in AppView
253- Use record keys that are meaningful
254- Plan for data portability
255
256### 3. Performance
257- Batch firehose processing
258- Use database transactions wisely
259- Index AppView tables appropriately
260- Cache frequently accessed data
261
262### 4. Error Handling
263- Use standard XRPC error codes
264- Provide meaningful error messages
265- Handle network failures gracefully
266- Implement proper retry logic
267
268### 5. Security
269- Validate all inputs against Lexicons
270- Verify signatures on repository data
271- Rate limit API endpoints
272- Sanitize user-generated content
273
274### 6. Federation
275- Design for multi-instance deployment
276- Handle remote user identities
277- Respect instance-specific policies
278- Plan for cross-instance data sync
279
280## Common Patterns
281
282### Handling User Content
283- Always validate against Lexicon schemas
284- Store in user's repository via CAR files
285- Index in AppView for efficient queries
286- Emit firehose events for subscribers
287
288## Resources
289
290### Official Documentation
291- [ATProto Specifications](https://atproto.com/specs)
292- [Lexicon Documentation](https://atproto.com/specs/lexicon)
293- [XRPC Specification](https://atproto.com/specs/xrpc)
294
295### Reference Implementations
296- [Indigo (Go)](https://github.com/bluesky-social/indigo)
297- [ATProto SDK (TypeScript)](https://github.com/bluesky-social/atproto)
298
299### Tools
300- [Lexicon CLI](https://github.com/bluesky-social/atproto/tree/main/packages/lex-cli)
301- [goat CLI](https://github.com/bluesky-social/indigo/tree/main/cmd/goat)