A community based topic aggregation platform built on atproto
bretton.dev
1# AT Protocol Implementation Guide
2
3This guide provides comprehensive information about implementing AT Protocol (atproto) in the Coves platform.
4
5## Table of Contents
6- [Core Concepts](#core-concepts)
7- [Architecture Overview](#architecture-overview)
8- [Lexicons](#lexicons)
9- [XRPC](#xrpc)
10- [Data Storage](#data-storage)
11- [Identity & Authentication](#identity--authentication)
12- [Firehose & Sync](#firehose--sync)
13- [Go Implementation Patterns](#go-implementation-patterns)
14- [Best Practices](#best-practices)
15
16## Core Concepts
17
18### What is AT Protocol?
19AT Protocol is a federated social networking protocol that enables:
20- **Decentralized identity** - Users own their identity (DID) and can move between providers
21- **Data portability** - Users can export and migrate their full social graph and content
22- **Interoperability** - Different apps can interact with the same underlying data
23
24### Key Components
251. **DIDs (Decentralized Identifiers)** - Persistent user identifiers (e.g., `did:plc:xyz123`)
262. **Handles** - Human-readable names that resolve to DIDs (e.g., `alice.bsky.social`)
273. **Repositories** - User data stored as signed Merkle trees in CAR files
284. **Lexicons** - Schema definitions for data types and API methods
295. **XRPC** - The RPC protocol for client-server communication
306. **Firehose** - Real-time event stream of repository changes
31
32## Architecture Overview
33
34### Two-Database Pattern
35AT Protocol requires two distinct data stores:
36
37#### 1. Repository Database (Source of Truth)
38- **Purpose**: Stores user-generated content as immutable, signed records
39- **Storage**: CAR files containing Merkle trees + PostgreSQL metadata
40- **Access**: Through XRPC procedures that modify repositories
41- **Properties**:
42 - Append-only (soft deletes via tombstones)
43 - Cryptographically verifiable
44 - User-controlled and portable
45
46#### 2. AppView Database (Query Layer)
47- **Purpose**: Denormalized, indexed data optimized for queries
48- **Storage**: PostgreSQL with application-specific schema
49- **Access**: Through XRPC queries (read-only)
50- **Properties**:
51 - Eventually consistent with repositories
52 - Can be rebuilt from repository data
53 - Application-specific aggregations
54
55### Data Flow
56
57```
58Write Path:
59Client → XRPC Procedure → Service → Write Repo → CAR Store
60 ↓
61 Firehose Event
62 ↓
63 AppView Indexer
64 ↓
65 AppView Database
66
67Read Path:
68Client → XRPC Query → Service → Read Repo → AppView Database
69```
70
71## Lexicons
72
73### What are Lexicons?
74Lexicons are JSON schema files that define:
75- Data types (records stored in repositories)
76- API methods (queries and procedures)
77- Input/output schemas for API calls
78
79### Lexicon Structure
80```json
81{
82 "lexicon": 1,
83 "id": "social.coves.community.profile",
84 "defs": {
85 "main": {
86 "type": "record",
87 "key": "self",
88 "record": {
89 "type": "object",
90 "required": ["name", "createdAt"],
91 "properties": {
92 "name": {"type": "string", "maxLength": 64},
93 "description": {"type": "string", "maxLength": 256},
94 "rules": {"type": "array", "items": {"type": "string"}},
95 "createdAt": {"type": "string", "format": "datetime"}
96 }
97 }
98 }
99 }
100}
101```
102
103### Lexicon Types
104
105#### 1. Record Types
106Define data structures stored in user repositories:
107```json
108{
109 "type": "record",
110 "key": "tid|rkey|literal",
111 "record": { /* schema */ }
112}
113```
114
115#### 2. Query Types (Read-only)
116Define read operations that don't modify state:
117```json
118{
119 "type": "query",
120 "parameters": { /* input schema */ },
121 "output": { /* response schema */ }
122}
123```
124
125#### 3. Procedure Types (Write)
126Define operations that modify repositories:
127```json
128{
129 "type": "procedure",
130 "input": { /* request body schema */ },
131 "output": { /* response schema */ }
132}
133```
134
135### Naming Conventions
136- Use reverse-DNS format: `social.coves.community.profile`
137- Queries often start with `get`, `list`, or `search`
138- Procedures often start with `create`, `update`, or `delete`
139- Keep names descriptive but concise
140
141## XRPC
142
143### What is XRPC?
144XRPC (Cross-Protocol RPC) is AT Protocol's HTTP-based RPC system:
145- All methods live under `/xrpc/` path
146- Method names map directly to Lexicon IDs
147- Supports both JSON and binary data
148
149### Request Format
150```
151# Query (GET)
152GET /xrpc/social.coves.community.getCommunity?id=123
153
154# Procedure (POST)
155POST /xrpc/social.coves.community.createPost
156Content-Type: application/json
157Authorization: Bearer <token>
158
159{"text": "Hello, Coves!"}
160```
161
162### Authentication
163- Uses Bearer tokens in Authorization header
164- Tokens are JWTs signed by the user's signing key
165- Service auth for server-to-server calls
166
167## Data Storage
168
169### CAR Files
170Content Addressable archive files store repository data:
171- Contains IPLD blocks forming a Merkle tree
172- Each block identified by CID (Content IDentifier)
173- Enables cryptographic verification and efficient sync
174
175### Record Keys (rkeys)
176- Unique identifiers for records within a collection
177- Can be TIDs (timestamp-based) or custom strings
178- Must match pattern: `[a-zA-Z0-9._~-]{1,512}`
179
180### Repository Structure
181```
182Repository (did:plc:user123)
183├── social.coves.post
184│ ├── 3kkreaz3amd27 (TID)
185│ └── 3kkreaz3amd28 (TID)
186├── social.coves.community.member
187│ ├── community123
188│ └── community456
189└── app.bsky.actor.profile
190 └── self
191```
192
193## Identity & Authentication
194
195### DIDs (Decentralized Identifiers)
196- Permanent, unique identifiers for users
197- Two types supported:
198 - `did:plc:*` - Hosted by PLC Directory
199 - `did:web:*` - Self-hosted
200
201### Handle Resolution
202Handles resolve to DIDs via:
2031. DNS TXT record: `_atproto.alice.com → did:plc:xyz`
2042. HTTPS well-known: `https://alice.com/.well-known/atproto-did`
205
206### Authentication Flow
2071. Client creates session with identifier/password
2082. Server returns access/refresh tokens
2093. Client uses access token for API requests
2104. Refresh when access token expires
211
212## Firehose & Sync
213
214### Firehose Events
215Real-time stream of repository changes:
216- Commit events (creates, updates, deletes)
217- Identity events (handle changes)
218- Account events (status changes)
219
220### Subscribing to Firehose
221Connect via WebSocket to `com.atproto.sync.subscribeRepos`:
222```
223wss://bsky.network/xrpc/com.atproto.sync.subscribeRepos
224```
225
226### Processing Events
227- Events include full record data and operation type
228- Process events to update AppView database
229- Handle out-of-order events with sequence numbers
230
231## Go Implementation Patterns
232
233### Using Indigo Library
234Bluesky's official Go implementation provides:
235- Lexicon code generation
236- CAR file handling
237- XRPC client/server
238- Firehose subscription
239
240### Code Generation
241Generate Go types from Lexicons:
242```bash
243go run github.com/bluesky-social/indigo/cmd/lexgen \
244 --package coves \
245 --prefix social.coves \
246 --outdir api/coves \
247 lexicons/social/coves/*.json
248```
249
250### Repository Operations
251```go
252// Write to repository
253rkey := models.GenerateTID()
254err := repoStore.CreateRecord(ctx, userDID, "social.coves.post", rkey, &Post{
255 Text: "Hello",
256 CreatedAt: time.Now().Format(time.RFC3339),
257})
258
259// Read from repository
260records, err := repoStore.ListRecords(ctx, userDID, "social.coves.post", limit, cursor)
261```
262
263### XRPC Handler Pattern
264```go
265func (s *Server) HandleGetCommunity(ctx context.Context) error {
266 // 1. Parse and validate input
267 id := xrpc.QueryParam(ctx, "id")
268
269 // 2. Call service layer
270 community, err := s.communityService.GetByID(ctx, id)
271 if err != nil {
272 return err
273 }
274
275 // 3. Return response
276 return xrpc.WriteJSONResponse(ctx, community)
277}
278```
279
280## Best Practices
281
282### 1. Lexicon Design
283- Keep schemas focused and single-purpose
284- Use references (`$ref`) for shared types
285- Version carefully - Lexicons are contracts
286- Document thoroughly with descriptions
287
288### 2. Data Modeling
289- Store minimal data in repositories
290- Denormalize extensively in AppView
291- Use record keys that are meaningful
292- Plan for data portability
293
294### 3. Performance
295- Batch firehose processing
296- Use database transactions wisely
297- Index AppView tables appropriately
298- Cache frequently accessed data
299
300### 4. Error Handling
301- Use standard XRPC error codes
302- Provide meaningful error messages
303- Handle network failures gracefully
304- Implement proper retry logic
305
306### 5. Security
307- Validate all inputs against Lexicons
308- Verify signatures on repository data
309- Rate limit API endpoints
310- Sanitize user-generated content
311
312### 6. Federation
313- Design for multi-instance deployment
314- Handle remote user identities
315- Respect instance-specific policies
316- Plan for cross-instance data sync
317
318## Common Patterns
319
320### Handling User Content
321- Always validate against Lexicon schemas
322- Store in user's repository via CAR files
323- Index in AppView for efficient queries
324- Emit firehose events for subscribers
325
326## Resources
327
328### Official Documentation
329- [ATProto Specifications](https://atproto.com/specs)
330- [Lexicon Documentation](https://atproto.com/specs/lexicon)
331- [XRPC Specification](https://atproto.com/specs/xrpc)
332
333### Reference Implementations
334- [Indigo (Go)](https://github.com/bluesky-social/indigo)
335- [ATProto SDK (TypeScript)](https://github.com/bluesky-social/atproto)
336
337### Tools
338- [Lexicon CLI](https://github.com/bluesky-social/atproto/tree/main/packages/lex-cli)
339- [goat CLI](https://github.com/bluesky-social/indigo/tree/main/cmd/goat)