A community based topic aggregation platform built on atproto
bretton.dev
1# Coves Local Development Guide
2
3Complete guide for setting up and running the Coves atProto development environment.
4
5## Table of Contents
6- [Quick Start](#quick-start)
7- [Architecture Overview](#architecture-overview)
8- [Prerequisites](#prerequisites)
9- [Setup Instructions](#setup-instructions)
10- [Using the Makefile](#using-the-makefile)
11- [Development Workflow](#development-workflow)
12- [Troubleshooting](#troubleshooting)
13- [Environment Variables](#environment-variables)
14
15## Quick Start
16
17```bash
18# 1. Start the PostgreSQL database
19make dev-db-up
20
21# 2. Start the PDS
22make dev-up
23
24# 3. View logs
25make dev-logs
26
27# 4. Check status
28make dev-status
29
30# 5. When done
31make dev-down
32```
33
34## Architecture Overview
35
36Coves uses a simplified single-database architecture with direct PDS firehose subscription:
37
38```
39┌─────────────────────────────────────────────┐
40│ Coves Local Development Stack │
41├─────────────────────────────────────────────┤
42│ │
43│ ┌──────────────┐ │
44│ │ PDS │ │
45│ │ :3001 │ │
46│ │ │ │
47│ │ Firehose───────────┐ │
48│ └──────────────┘ │ │
49│ │ │
50│ ▼ │
51│ ┌──────────────┐ │
52│ │ Coves AppView│ │
53│ │ (Go) │ │
54│ │ :8081 │ │
55│ └──────┬───────┘ │
56│ │ │
57│ ┌──────▼───────┐ │
58│ │ PostgreSQL │ │
59│ │ :5433 │ │
60│ └──────────────┘ │
61│ │
62└─────────────────────────────────────────────┘
63
64Your Production PDS (:3000) ← Runs independently
65```
66
67### Components
68
691. **PDS (Port 3001)** - Bluesky's Personal Data Server with:
70 - User repositories and CAR files (stored in Docker volume)
71 - Internal SQLite database for PDS metadata
72 - Firehose WebSocket: `ws://localhost:3001/xrpc/com.atproto.sync.subscribeRepos`
732. **PostgreSQL (Port 5433)** - Database for Coves AppView data only
743. **Coves AppView (Port 8081)** - Your Go application that:
75 - Subscribes directly to PDS firehose
76 - Indexes Coves-specific data to PostgreSQL
77
78**Key Points:**
79- ✅ Ports chosen to avoid conflicts with production PDS on :3000
80- ✅ PDS is self-contained with its own SQLite database and CAR storage
81- ✅ PostgreSQL is only used by the Coves AppView for indexing
82- ✅ AppView subscribes directly to PDS firehose (no relay needed)
83- ✅ Simple, clean architecture for local development
84
85## Prerequisites
86
87- **Docker & Docker Compose** - For running containerized services
88- **Go 1.22+** - For building the Coves AppView
89- **PostgreSQL client** (optional) - For database inspection
90- **Make** (optional but recommended) - For convenient commands
91
92## Setup Instructions
93
94### Step 1: Start the Database
95
96The PostgreSQL database must be running first:
97
98```bash
99# Start the database
100make dev-db-up
101
102# Verify it's running
103make dev-status
104```
105
106**Connection Details:**
107- Host: `localhost`
108- Port: `5433`
109- Database: `coves_dev`
110- User: `dev_user`
111- Password: `dev_password`
112
113### Step 2: Start the PDS
114
115Start the Personal Data Server:
116
117```bash
118# Start PDS
119make dev-up
120
121# View logs (follows in real-time)
122make dev-logs
123```
124
125Wait for health checks to pass (~10-30 seconds).
126
127### Step 3: Verify Services
128
129```bash
130# Check PDS is running
131make dev-status
132
133# Test PDS health endpoint
134curl http://localhost:3001/xrpc/_health
135
136# Test PDS firehose endpoint (should get WebSocket upgrade response)
137curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" \
138 http://localhost:3001/xrpc/com.atproto.sync.subscribeRepos
139```
140
141### Step 4: Run Coves AppView (When Ready)
142
143When you have a Dockerfile for the AppView:
144
1451. Uncomment the `appview` service in `docker-compose.dev.yml`
1462. Restart the stack: `make dev-down && make dev-up`
147
148Or run the AppView locally:
149
150```bash
151# Set environment variables
152export DATABASE_URL="postgresql://dev_user:dev_password@localhost:5433/coves_dev?sslmode=disable"
153export FIREHOSE_URL="ws://localhost:3001/xrpc/com.atproto.sync.subscribeRepos"
154export PDS_URL="http://localhost:3001"
155export PORT=8081
156
157# Run the AppView
158go run ./cmd/server
159```
160
161## Using the Makefile
162
163The Makefile provides convenient commands for development. Run `make help` to see all available commands:
164
165### General Commands
166
167```bash
168make help # Show all available commands with descriptions
169```
170
171### Development Stack Commands
172
173```bash
174make dev-up # Start PDS for local development
175make dev-down # Stop the stack
176make dev-logs # Tail logs from PDS
177make dev-status # Show status of containers
178make dev-reset # Nuclear option - remove all data and volumes
179```
180
181### Database Commands
182
183```bash
184make dev-db-up # Start PostgreSQL database
185make dev-db-down # Stop PostgreSQL database
186make dev-db-reset # Reset database (delete all data)
187make db-shell # Open psql shell to the database
188```
189
190### Testing Commands
191
192```bash
193make test # Run all tests (starts test DB, runs migrations, executes tests)
194make test-db-reset # Reset test database (clean slate)
195make test-db-stop # Stop test database
196```
197
198**See [TESTING_SUMMARY.md](../TESTING_SUMMARY.md) for complete testing documentation.**
199
200### Workflow Commands
201
202```bash
203make fresh-start # Complete fresh start (reset everything)
204make quick-restart # Quick restart (keeps data)
205```
206
207### Build Commands
208
209```bash
210make build # Build the Coves server binary
211make run # Run the Coves server
212make clean # Clean build artifacts
213```
214
215### Utilities
216
217```bash
218make validate-lexicon # Validate all Lexicon schemas
219make docs # Show documentation file locations
220```
221
222## Development Workflow
223
224### Typical Development Session
225
226```bash
227# 1. Start fresh environment
228make fresh-start
229
230# 2. Work on code...
231
232# 3. Restart services as needed
233make quick-restart
234
235# 4. View logs
236make dev-logs
237
238# 5. Run tests
239make test
240
241# 6. Clean up when done
242make dev-down
243```
244
245### Testing Lexicon Changes
246
247```bash
248# 1. Edit Lexicon files in internal/atproto/lexicon/
249
250# 2. Validate schemas
251make validate-lexicon
252
253# 3. Restart services to pick up changes
254make quick-restart
255```
256
257### Database Inspection
258
259```bash
260# Open PostgreSQL shell
261make db-shell
262
263# Or use psql directly
264PGPASSWORD=dev_password psql -h localhost -p 5433 -U dev_user -d coves_dev
265```
266
267### Viewing Logs
268
269```bash
270# Follow all logs
271make dev-logs
272
273# Or use docker-compose directly
274docker-compose -f docker-compose.dev.yml logs -f pds
275docker-compose -f docker-compose.dev.yml logs -f relay
276```
277
278## Troubleshooting
279
280### Port Already in Use
281
282**Problem:** Error binding to port 3000, 5433, etc.
283
284**Solution:**
285- The dev environment uses non-standard ports to avoid conflicts
286- PDS: 3001 (not 3000)
287- PostgreSQL: 5433 (not 5432)
288- Relay: 2471 (not 2470)
289- AppView: 8081 (not 8080)
290
291If you still have conflicts, check what's using the port:
292
293```bash
294# Check what's using a port
295lsof -i :3001
296lsof -i :5433
297
298# Kill the process
299kill -9 <PID>
300```
301
302### Database Connection Failed
303
304**Problem:** Services can't connect to PostgreSQL
305
306**Solution:**
307
308```bash
309# Ensure database is running
310make dev-db-up
311
312# Check database logs
313cd internal/db/local_dev_db_compose && docker-compose logs
314
315# Verify connection manually
316PGPASSWORD=dev_password psql -h localhost -p 5433 -U dev_user -d coves_dev
317```
318
319### PDS Health Check Failing
320
321**Problem:** PDS container keeps restarting
322
323**Solution:**
324
325```bash
326# Check PDS logs
327docker-compose -f docker-compose.dev.yml logs pds
328
329# Common issues:
330# 1. Database not accessible - ensure DB is running
331# 2. Invalid environment variables - check .env.dev
332# 3. Port conflict - ensure port 3001 is free
333```
334
335### AppView Not Receiving Firehose Events
336
337**Problem:** AppView isn't receiving events from PDS firehose
338
339**Solution:**
340
341```bash
342# Check PDS logs for firehose activity
343docker-compose -f docker-compose.dev.yml logs pds
344
345# Verify firehose endpoint is accessible
346curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" \
347 http://localhost:3001/xrpc/com.atproto.sync.subscribeRepos
348
349# Check AppView is connecting to correct URL:
350# FIREHOSE_URL=ws://localhost:3001/xrpc/com.atproto.sync.subscribeRepos
351```
352
353### Fresh Start Not Working
354
355**Problem:** `make fresh-start` fails
356
357**Solution:**
358
359```bash
360# Manually clean everything
361docker-compose -f docker-compose.dev.yml down -v
362cd internal/db/local_dev_db_compose && docker-compose down -v
363docker volume prune -f
364docker network prune -f
365
366# Then start fresh
367make dev-db-up
368sleep 2
369make dev-up
370```
371
372### Production PDS Interference
373
374**Problem:** Dev environment conflicts with your production PDS
375
376**Solution:**
377- Dev PDS runs on port 3001 (production is 3000)
378- Dev services use different handle domain (`.local.coves.dev`)
379- They should not interfere unless you have custom networking
380
381```bash
382# Verify production PDS is still accessible
383curl http://localhost:3000/xrpc/_health
384
385# Verify dev PDS is separate
386curl http://localhost:3001/xrpc/_health
387```
388
389## Environment Variables
390
391All configuration is in [.env.dev](../.env.dev) - a single file for both development and testing:
392
393### Development Database (Port 5433)
394```bash
395POSTGRES_HOST=localhost
396POSTGRES_PORT=5433
397POSTGRES_DB=coves_dev
398POSTGRES_USER=dev_user
399POSTGRES_PASSWORD=dev_password
400```
401
402### Test Database (Port 5434)
403```bash
404POSTGRES_TEST_DB=coves_test
405POSTGRES_TEST_USER=test_user
406POSTGRES_TEST_PASSWORD=test_password
407POSTGRES_TEST_PORT=5434
408```
409
410### PDS Configuration
411```bash
412PDS_HOSTNAME=localhost
413PDS_PORT=3001
414PDS_JWT_SECRET=local-dev-jwt-secret-change-in-production
415PDS_ADMIN_PASSWORD=admin
416PDS_SERVICE_HANDLE_DOMAINS=.local.coves.dev
417```
418
419### AppView Configuration
420```bash
421APPVIEW_PORT=8081
422FIREHOSE_URL=ws://localhost:3001/xrpc/com.atproto.sync.subscribeRepos
423PDS_URL=http://localhost:3001
424```
425
426### Development Settings
427```bash
428ENV=development
429LOG_LEVEL=debug
430```
431
432**No separate `.env.test` file needed!** All configuration (dev + test) is in `.env.dev`.
433
434## Next Steps
435
4361. **Build the Firehose Subscriber** - Create the AppView component that subscribes to the relay
4372. **Define Custom Lexicons** - Create Coves-specific schemas in `internal/atproto/lexicon/social/coves/`
4383. **Implement XRPC Handlers** - Build the API endpoints for Coves features
4394. **Create Integration Tests** - Use Testcontainers to test the full stack
440
441## Additional Resources
442
443- [CLAUDE.md](../CLAUDE.md) - Build guidelines and security practices
444- [ATPROTO_GUIDE.md](../ATPROTO_GUIDE.md) - Comprehensive atProto implementation guide
445- [PROJECT_STRUCTURE.md](../PROJECT_STRUCTURE.md) - Project organization
446- [PRD.md](../PRD.md) - Product requirements and roadmap
447
448## Getting Help
449
450- Check logs: `make dev-logs`
451- View status: `make dev-status`
452- Reset everything: `make fresh-start`
453- Inspect database: `make db-shell`
454
455For issues with atProto concepts, see [ATPROTO_GUIDE.md](../ATPROTO_GUIDE.md).
456
457For build process questions, see [CLAUDE.md](../CLAUDE.md).