···
1
-
# Repository Testing Summary
1
+
# Coves Testing Guide
3
-
## Lexicon Schema Validation
3
+
This document explains how testing works in Coves, including setup, running tests, and understanding the test infrastructure.
5
-
We use Indigo's lexicon validator to ensure all AT Protocol schemas are valid and properly structured.
7
-
### Validation Components:
8
-
1. **Schema Validation** (`cmd/validate-lexicon/main.go`)
9
-
- Validates all 57 lexicon schema files are valid JSON
10
-
- Checks cross-references between schemas
11
-
- Validates test data files against their schemas
7
+
Coves uses a unified testing approach with:
8
+
- **Single configuration file**: [.env.dev](.env.dev) for all environments (dev + test)
9
+
- **Isolated test database**: PostgreSQL on port 5434 (separate from dev on 5433)
10
+
- **Makefile commands**: Simple `make test` command handles everything
11
+
- **Docker Compose profiles**: Test database spins up automatically
16
+
# Run all tests (starts test DB, runs migrations, executes tests)
19
+
# Reset test database (clean slate)
22
+
# Stop test database
26
+
## Test Infrastructure
13
-
2. **Test Data** (`tests/lexicon-test-data/`)
14
-
- Contains example records for validation testing
15
-
- Files use naming convention:
16
-
- `*-valid*.json` - Should pass validation
17
-
- `*-invalid-*.json` - Should fail validation (tests error detection)
18
-
- Currently covers 5 record types:
19
-
- social.coves.actor.profile
20
-
- social.coves.community.profile
21
-
- social.coves.post.record
22
-
- social.coves.interaction.vote
23
-
- social.coves.moderation.ban
28
+
### Configuration (.env.dev)
25
-
3. **Validation Library** (`internal/validation/lexicon.go`)
26
-
- Wrapper around Indigo's ValidateRecord
27
-
- Provides type-specific validation methods
28
-
- Supports multiple input formats
30
+
All test configuration lives in [.env.dev](.env.dev):
30
-
### Running Lexicon Validation
32
-
# Full validation (schemas + test data) - DEFAULT
33
+
# Test Database Configuration
34
+
POSTGRES_TEST_DB=coves_test
35
+
POSTGRES_TEST_USER=test_user
36
+
POSTGRES_TEST_PASSWORD=test_password
37
+
POSTGRES_TEST_PORT=5434
40
+
**No separate `.env.test` file needed!** Everything is in `.env.dev`.
44
+
The test database runs in Docker via [docker-compose.dev.yml](docker-compose.dev.yml):
46
+
- **Service**: `postgres-test` (profile: `test`)
47
+
- **Port**: 5434 (separate from dev database on 5433)
48
+
- **Automatic startup**: The Makefile handles starting/stopping
49
+
- **Isolated data**: Completely separate from development database
51
+
### Running Tests Manually
53
+
If you need to run tests without the Makefile:
56
+
# 1. Load environment variables
57
+
set -a && source .env.dev && set +a
59
+
# 2. Start test database
60
+
docker-compose -f docker-compose.dev.yml --env-file .env.dev --profile test up -d postgres-test
62
+
# 3. Wait for it to be ready
66
+
goose -dir internal/db/migrations postgres \
67
+
"postgresql://$POSTGRES_TEST_USER:$POSTGRES_TEST_PASSWORD@localhost:$POSTGRES_TEST_PORT/$POSTGRES_TEST_DB?sslmode=disable" up
72
+
# 6. Stop test database (optional)
73
+
docker-compose -f docker-compose.dev.yml --env-file .env.dev --profile test stop postgres-test
76
+
**Note**: The Makefile automatically loads `.env.dev` variables, so `make test` is simpler than running manually.
82
+
Test individual components in isolation.
84
+
**Example**: [internal/core/users/service_test.go](internal/core/users/service_test.go)
87
+
# Run unit tests for a specific package
88
+
go test -v ./internal/core/users/...
91
+
### 2. Integration Tests
93
+
Test full request/response flows with a real database.
95
+
**Location**: [tests/integration/](tests/integration/)
98
+
- Start test database
100
+
- Execute HTTP requests against real handlers
103
+
**Example**: [tests/integration/integration_test.go](tests/integration/integration_test.go)
106
+
# Run integration tests
109
+
go test -v ./tests/integration/...
112
+
### 3. Lexicon Validation Tests
114
+
Validate AT Protocol Lexicon schemas and test data.
117
+
- **Schemas**: [internal/atproto/lexicon/](internal/atproto/lexicon/) - 57 lexicon schema files
118
+
- **Test Data**: [tests/lexicon-test-data/](tests/lexicon-test-data/) - Example records for validation
119
+
- **Validator**: [cmd/validate-lexicon/](cmd/validate-lexicon/) - Validation tool
120
+
- **Library**: [internal/validation/](internal/validation/) - Validation helpers
122
+
**Running validation**:
125
+
# Full validation (schemas + test data)
go run cmd/validate-lexicon/main.go
35
-
# Schemas only (skip test data validation)
128
+
# Schemas only (skip test data)
go run cmd/validate-lexicon/main.go --schemas-only
go run cmd/validate-lexicon/main.go -v
41
-
# Strict validation mode
go run cmd/validate-lexicon/main.go --strict
45
-
### Test Coverage Warning
46
-
The validator explicitly outputs which record types have test data and which don't. This prevents false confidence from passing tests when schemas lack test coverage.
138
+
**Test data naming convention**:
139
+
- `*-valid*.json` - Should pass validation
140
+
- `*-invalid-*.json` - Should fail validation (tests error detection)
142
+
**Current coverage** (as of last update):
143
+
- ✅ social.coves.actor.profile
144
+
- ✅ social.coves.community.profile
145
+
- ✅ social.coves.post.record
146
+
- ✅ social.coves.interaction.vote
147
+
- ✅ social.coves.moderation.ban
149
+
## Database Migrations
151
+
Migrations are managed with [goose](https://github.com/pressly/goose) and stored in [internal/db/migrations/](internal/db/migrations/).
153
+
### Running Migrations
156
+
# Development database
159
+
# Test database (automatically run by `make test`)
50
-
📋 Validation Summary:
51
-
Valid test files: 5/5 passed
52
-
Invalid test files: 2/2 correctly rejected
54
-
✅ All test files behaved as expected!
163
+
### Creating Migrations
56
-
📊 Test Data Coverage Summary:
57
-
- Records with test data: 5 types
58
-
- Valid test files: 5
59
-
- Invalid test files: 2 (for error validation)
166
+
# Create a new migration
167
+
goose -dir internal/db/migrations create migration_name sql
61
-
Tested record types:
62
-
✓ social.coves.actor.profile
63
-
✓ social.coves.community.profile
64
-
✓ social.coves.post.record
65
-
✓ social.coves.interaction.vote
66
-
✓ social.coves.moderation.ban
170
+
# internal/db/migrations/YYYYMMDDHHMMSS_migration_name.sql
68
-
⚠️ Record types without test data:
69
-
- social.coves.actor.membership
70
-
- social.coves.actor.subscription
71
-
- social.coves.community.rules
173
+
### Migration Best Practices
175
+
- **Always test migrations** on test database first
176
+
- **Write both Up and Down** migrations
177
+
- **Keep migrations atomic** - one logical change per migration
178
+
- **Test rollback** - verify the Down migration works
179
+
- **Don't modify old migrations** - create new ones instead
181
+
## Test Database Management
74
-
Coverage: 5/16 record types have test data (31.2%)
186
+
# Complete reset (deletes all data)
190
+
### Connecting to Test Database
79
-
# Run lexicon validation tests
80
-
go test -v ./tests/lexicon_validation_test.go
194
+
PGPASSWORD=test_password psql -h localhost -p 5434 -U test_user -d coves_test
82
-
# Run validation library tests
83
-
go test -v ./internal/validation/...
196
+
# Using docker exec
197
+
docker exec -it coves-test-postgres psql -U test_user -d coves_test
86
-
## Test Infrastructure Setup
87
-
- Created Docker Compose configuration for isolated test database on port 5434
88
-
- Test database is completely separate from development (5433) and production (5432)
89
-
- Configuration location: `/internal/db/test_db_compose/docker-compose.yml`
200
+
### Inspecting Test Data
91
-
## Repository Service Implementation
92
-
Successfully integrated Indigo's carstore for ATProto repository management:
95
-
1. **CarStore Wrapper** (`/internal/atproto/carstore/carstore.go`)
96
-
- Wraps Indigo's carstore implementation
97
-
- Manages CAR file storage with PostgreSQL metadata
206
+
-- View table schema
99
-
2. **RepoStore** (`/internal/atproto/carstore/repo_store.go`)
100
-
- Combines CarStore with UserMapping for DID-based access
101
-
- Handles DID to UID conversions transparently
210
+
SELECT * FROM users;
103
-
3. **UserMapping** (`/internal/atproto/carstore/user_mapping.go`)
104
-
- Maps ATProto DIDs to numeric UIDs (required by Indigo)
105
-
- Auto-creates user_maps table via GORM
212
+
-- Check migrations
213
+
SELECT * FROM goose_db_version;
107
-
4. **Repository Service** (`/internal/core/repository/service.go`)
108
-
- Updated to use Indigo's carstore instead of custom implementation
109
-
- Handles empty repositories gracefully
110
-
- Placeholder CID for empty repos until records are added
113
-
All repository tests passing:
114
-
- ✅ CreateRepository - Creates user mapping and repository record
115
-
- ✅ ImportExport - Handles empty CAR data correctly
116
-
- ✅ DeleteRepository - Removes repository and carstore data
117
-
- ✅ CompactRepository - Runs garbage collection
118
-
- ✅ UserMapping - DID to UID mapping works correctly
218
+
### Integration Test Template
120
-
## Implementation Notes
121
-
1. **Empty Repositories**: Since Indigo's carstore expects actual CAR data, we handle empty repositories by:
122
-
- Creating user mapping only
123
-
- Using placeholder CID
124
-
- Returning empty byte array on export
125
-
- Actual CAR data will be created when records are added
221
+
package integration
127
-
2. **Database Tables**: Indigo's carstore auto-creates:
128
-
- `user_maps` (DID ↔ UID mapping)
129
-
- `car_shards` (CAR file metadata)
130
-
- `block_refs` (IPLD block references)
132
-
3. **Migration**: Created migration to drop our custom block_refs table to avoid conflicts
228
+
func TestYourFeature(t *testing.T) {
229
+
db := setupTestDB(t)
135
-
To fully utilize the carstore, implement:
136
-
1. Record CRUD operations using carstore's DeltaSession
137
-
2. Proper CAR file generation when adding records
138
-
3. Commit tracking with proper signatures
139
-
4. Repository versioning and history
232
+
// Wire up dependencies
233
+
repo := postgres.NewYourRepository(db)
234
+
service := yourpackage.NewYourService(repo)
236
+
// Create test router
237
+
r := chi.NewRouter()
238
+
r.Mount("/api/path", routes.YourRoutes(service))
241
+
req := httptest.NewRequest("GET", "/api/path", nil)
242
+
w := httptest.NewRecorder()
243
+
r.ServeHTTP(w, req)
246
+
if w.Code != http.StatusOK {
247
+
t.Errorf("Expected 200, got %d", w.Code)
252
+
### Test Helper: setupTestDB
254
+
The `setupTestDB` function (in [tests/integration/integration_test.go](tests/integration/integration_test.go)):
255
+
- Reads config from environment variables (set by `.env.dev`)
256
+
- Connects to test database
258
+
- Cleans up test data
259
+
- Returns ready-to-use `*sql.DB`
261
+
## Common Test Commands
143
-
# Start test database
144
-
cd internal/db/test_db_compose
145
-
docker-compose up -d
267
+
# Run specific test package
268
+
go test -v ./internal/core/users/...
270
+
# Run specific test
271
+
go test -v ./tests/integration/... -run TestCreateUser
273
+
# Run with coverage
274
+
go test -v -cover ./...
276
+
# Run with race detector
277
+
go test -v -race ./...
283
+
make test-db-reset && make test
147
-
# Run repository tests
148
-
TEST_DATABASE_URL="postgres://test_user:test_password@localhost:5434/coves_test?sslmode=disable" \
149
-
go test -v ./internal/core/repository/...
288
+
### Test database won't start
151
-
# Stop test database
152
-
docker-compose down
291
+
# Check if port 5434 is in use
294
+
# Check container logs
295
+
docker logs coves-test-postgres
298
+
docker-compose -f docker-compose.dev.yml --profile test down -v
302
+
### Migrations failing
305
+
# Load environment and check migration status
306
+
set -a && source .env.dev && set +a
307
+
goose -dir internal/db/migrations postgres \
308
+
"postgresql://$POSTGRES_TEST_USER:$POSTGRES_TEST_PASSWORD@localhost:$POSTGRES_TEST_PORT/$POSTGRES_TEST_DB?sslmode=disable" status
314
+
### Tests can't connect to database
317
+
# Verify test database is running
318
+
docker ps | grep coves-test-postgres
320
+
# Verify environment variables
321
+
set -a && source .env.dev && set +a && echo "Test DB Port: $POSTGRES_TEST_PORT"
323
+
# Test connection manually
324
+
PGPASSWORD=test_password psql -h localhost -p 5434 -U test_user -d coves_test -c "SELECT 1"
327
+
### Lexicon validation errors
330
+
# Check schema syntax
331
+
cat internal/atproto/lexicon/path/to/schema.json | jq .
333
+
# Validate specific schema
334
+
go run cmd/validate-lexicon/main.go -v
336
+
# Check test data format
337
+
cat tests/lexicon-test-data/your-test.json | jq .
342
+
### Test Organization
343
+
- ✅ Unit tests live next to the code they test (`*_test.go`)
344
+
- ✅ Integration tests live in `tests/integration/`
345
+
- ✅ Test data lives in `tests/lexicon-test-data/`
346
+
- ✅ One test file per feature/endpoint
349
+
- ✅ Use `@example.com` emails for test users (auto-cleaned by setupTestDB)
350
+
- ✅ Clean up data in tests (or rely on setupTestDB cleanup)
351
+
- ✅ Don't rely on specific test execution order
352
+
- ✅ Each test should be independent
355
+
- ✅ Always use the test database (port 5434)
356
+
- ✅ Never connect to development database (port 5433) in tests
357
+
- ✅ Use transactions for fast test cleanup (where applicable)
358
+
- ✅ Test with realistic data sizes
361
+
- ✅ Create both valid and invalid test cases
362
+
- ✅ Cover all required fields
363
+
- ✅ Test edge cases (empty strings, max lengths, etc.)
364
+
- ✅ Update tests when schemas change
366
+
## CI/CD Integration
368
+
When setting up CI/CD, the test pipeline should:
371
+
# Example GitHub Actions workflow
373
+
- name: Start test database
375
+
docker-compose -f docker-compose.dev.yml --env-file .env.dev --profile test up -d postgres-test
378
+
- name: Run migrations
379
+
run: make test-db-migrate # (add this to Makefile if needed)
385
+
run: docker-compose -f docker-compose.dev.yml --profile test down -v
388
+
## Related Documentation
390
+
- [Makefile](Makefile) - All test commands
391
+
- [.env.dev](.env.dev) - Test configuration
392
+
- [docker-compose.dev.yml](docker-compose.dev.yml) - Test database setup
393
+
- [LOCAL_DEVELOPMENT.md](docs/LOCAL_DEVELOPMENT.md) - Full development setup
394
+
- [CLAUDE.md](CLAUDE.md) - Build guidelines
398
+
If tests are failing:
399
+
1. Check [Troubleshooting](#troubleshooting) section above
400
+
2. Verify `.env.dev` has correct test database config
401
+
3. Run `make test-db-reset` for a clean slate
402
+
4. Check test database logs: `docker logs coves-test-postgres`
404
+
For questions about:
405
+
- **Test infrastructure**: This document
406
+
- **AT Protocol testing**: [ATPROTO_GUIDE.md](ATPROTO_GUIDE.md)
407
+
- **Development setup**: [LOCAL_DEVELOPMENT.md](docs/LOCAL_DEVELOPMENT.md)
bretton.dev