code
Clone this repository
https://tangled.org/bretton.dev/coves
git@knot.bretton.dev:bretton.dev/coves
For self-hosted knots, clone URLs may differ based on your setup.
Implement blob upload service to fetch images from URLs and upload them to
PDS as atproto blobs, enabling proper thumbnail storage for external embeds.
**Service Features:**
- UploadBlobFromURL: Fetch image from URL → validate → upload to PDS
- UploadBlob: Upload raw binary data to PDS with authentication
- Size limit: 1MB per image (atproto recommendation)
- Supported MIME types: image/jpeg, image/png, image/webp
- MIME type normalization (image/jpg → image/jpeg)
- Timeout handling (10s for fetch, 30s for upload)
**Security & Validation:**
- Input validation (empty checks, nil guards)
- Size validation before network calls
- MIME type validation before reading data
- HTTP status code checking with sanitized error logs
- Proper error wrapping for debugging
**Federated Support:**
- Uses community's PDS URL when available
- Fallback to service default PDS
- Community authentication via PDSAccessToken
**Flow:**
1. Client posts external embed with URI (no thumb)
2. Unfurl service fetches metadata from oEmbed/OpenGraph
3. Blob service downloads thumbnail from metadata.thumbnailURL
4. Upload to community's PDS via com.atproto.repo.uploadBlob
5. Return BlobRef with CID for inclusion in post record
**BlobRef Type:**
```go
type BlobRef struct {
Type string `json:"$type"` // "blob"
Ref map[string]string `json:"ref"` // {"$link": "bafyrei..."}
MimeType string `json:"mimeType"` // "image/jpeg"
Size int `json:"size"` // bytes
}
```
This enables automatic thumbnail upload when users post links to
Streamable, YouTube, Reddit, Kagi Kite, or any URL with OpenGraph metadata.
Add PostgreSQL-backed cache for oEmbed and OpenGraph unfurl results to reduce
external API calls and improve performance.
**Database Layer:**
- Migration 017: Create unfurl_cache table with JSONB metadata storage
- Index on expires_at for efficient TTL-based cleanup
- Store provider, metadata, and thumbnail_url with expiration
**Repository Layer:**
- Repository interface with Get/Set operations
- PostgreSQL implementation with JSON marshaling
- Automatic TTL handling via PostgreSQL intervals
- Returns nil on cache miss (not an error)
**Error Types:**
- ErrNotFound: Cache miss or expired entry
- ErrInvalidURL: Invalid URL format
- ErrInvalidTTL: Non-positive TTL value
Design decisions:
- JSONB metadata column for flexible schema evolution
- Separate thumbnail_url for potential query optimization
- ON CONFLICT for upsert behavior (update on re-fetch)
- TTL-based expiration (default: 24 hours)
Part of URL unfurling feature to auto-populate external embeds with rich
metadata from supported providers (Streamable, YouTube, Reddit, Kagi, etc.).
Related: Circuit breaker pattern prevents cascading failures when providers
go down (already implemented in previous commits).
Update integration tests to reflect new validation order and circuit
breaker integration in unfurl service.
Changes in post_creation_test.go:
- Fix content length validation test expectations
- Update validation order: basic input before DID authentication
- Adjust test assertions to match new error flow
Changes in post_unfurl_test.go:
- Update Kagi provider test to expect circuit breaker wrapper
- Fix provider name expectations in unfurl tests
- Ensure tests align with circuit breaker integration
These changes ensure all integration tests pass with the new validation
flow and circuit breaker implementation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Restore full aggregator authorization checks while maintaining the
special case for Kagi aggregator's thumbnail URL handling.
Changes:
- Restore aggregator DID validation in post creation flow
- Add distinction between Kagi (trusted) and other aggregators
- Map aggregator authorization errors to 403 Forbidden
- Maintain validation order: basic input -> DID auth -> aggregator check
- Keep Kagi special case for thumbnail URL transformation
Security improvements:
- All aggregator posts now require valid aggregator DID registration
- Kagi aggregator identified via KAGI_AGGREGATOR_DID environment variable
- Non-Kagi aggregators must follow standard thumbnail validation rules
- Unauthorized aggregator attempts return 403 with clear error message
This ensures only authorized aggregators can create posts while allowing
Kagi's existing thumbnail URL workflow to continue working.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement circuit breaker pattern to handle external provider failures
gracefully and prevent cascading failures when unfurl services are down.
Changes:
- Add circuit_breaker.go with state management (Closed, Open, HalfOpen)
- Implement automatic recovery with exponential backoff
- Add comprehensive circuit breaker unit tests
- Integrate circuit breaker into unfurl service
- Fix defer response.Body.Close() errors in providers
- Fix linting issues in kagi_test.go and opengraph_test.go
The circuit breaker tracks failures per provider and automatically opens
when failure threshold is reached, preventing wasted requests to failing
services. After a cooldown period, it transitions to half-open to test
if the service has recovered.
Configuration:
- Failure threshold: 5 consecutive failures
- Timeout: 10 seconds
- Reset timeout: 60 seconds (before attempting recovery)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add community handles to all feed responses and refactor feed repositories
Features:
- Add handle field to communityRef lexicon and struct
- Select community handle in all feed SQL queries (community, timeline, discover)
- Populate handle in comment service post views
- Refactor feed_repo.go to use feedRepoBase (68% code reduction)
- Add HMAC-signed cursors for security
Improvements:
- Improved error handling for missing communities (ERROR log + fallback)
- Moved nullStringPtr helper to correct location
- Apply gofumpt formatting to entire codebase
All tests passing, linter checks pass, production-ready.
- Run gofumpt -l -w on all Go files
- Fix import statement formatting (blank lines between groups)
- Auto-fix via make lint-fix
- All linter checks now pass
No functional changes, formatting only