code
Clone this repository
https://tangled.org/bretton.dev/coves
git@knot.bretton.dev:bretton.dev/coves
For self-hosted knots, clone URLs may differ based on your setup.
Add two new scripts for generating realistic test data:
- generate_deep_thread.go: Creates deeply nested comment threads (100 levels)
for testing threading logic, depth limits, and performance
- generate_nba_comments.go: Generates NBA-themed comments with realistic
basketball discussion content for UX testing and demos
Both scripts:
- Insert directly into PostgreSQL (bypassing Jetstream for speed)
- Create realistic comment trees with varied content
- Useful for stress testing, performance validation, and demos
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove deprecated 'version' field from docker-compose.dev.yml.
Docker Compose v2 ignores this field and it's no longer needed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add source_name field to Perspective model for better attribution
- Extract source name from HTML anchor tags in parser
- Display source name in rich text (e.g., "The Straits Times" vs generic "Source")
- Improve spacing in highlights, perspectives, and sources lists (double newlines)
- Better visual separation between list items
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Streamable and other providers return protocol-relative URLs (//cdn.example.com)
in their oEmbed thumbnail_url field. These fail when passed to the blob service
because http.NewRequest requires a full URL with scheme.
Fix:
- Add normalizeURL() helper to convert // → https://
- Apply to both oEmbed and OpenGraph thumbnail URLs
- Add comprehensive tests (6 test cases)
Example transformation:
//cdn-cf-east.streamable.com/image/abc.jpg
→ https://cdn-cf-east.streamable.com/image/abc.jpg
This ensures Streamable video thumbnails are properly downloaded and uploaded
as blobs, fixing missing thumbnails in video embeds.
Affects: All users posting Streamable/YouTube/Reddit links (not Kagi-specific)
Tested: Unit tests pass, build successful
Update Kagi News aggregator client for nested external embed structure,
add unfurling roadmap items, and include utility scripts for testing.
**Kagi News Aggregator Updates:**
- Update client to use nested "external" structure (social.coves.embed.external)
- Update tests for new embed format
- Update README with latest API requirements
- Fix E2E tests for lexicon schema changes
**PRD Backlog Additions:**
60 new lines documenting unfurling feature:
- URL metadata enrichment via oEmbed/OpenGraph
- Blob upload for thumbnails
- Cache strategy (24h TTL)
- Circuit breaker for provider failures
- Supported providers (Streamable, YouTube, Reddit, Kagi)
**.env.dev Updates:**
- Add unfurl service configuration
- Add blob upload settings
- Update PDS URLs for testing
**Development Scripts:**
- generate_test_comments.go: Generate test comment data
- post_streamable.py: Test Streamable unfurling E2E
**Summary:**
- Aggregator client: Updated for nested embed structure
- Documentation: Added unfurling feature to backlog
- Scripts: Testing utilities for development
All aggregator tests updated to match new lexicon schema.
Ready for deployment with backward-compatible migration path.
Add extensive test coverage for external embed thumb validation and blob
reference transformation in feed responses.
**Thumb Validation Tests** (post_thumb_validation_test.go):
7 test cases covering strict blob reference validation:
1. ❌ Reject thumb as URL string (must be blob ref)
2. ❌ Reject thumb missing $type field
3. ❌ Reject thumb missing ref field
4. ❌ Reject thumb missing mimeType field
5. ✅ Accept valid blob reference
6. ✅ Accept missing thumb (unfurl will handle)
7. Security: Prevents URL injection attacks via thumb field
**Feed Blob Transform Tests** (feed_test.go):
6 test cases for GetCommunityFeed blob URL transformation:
1. Transforms blob refs to PDS URLs
2. Preserves community PDSURL in PostView
3. Generates correct getBlob endpoint URLs
4. Handles posts without embeds
5. Handles posts without thumbs
6. End-to-end feed query validation
**Integration Test Updates:**
- Update post creation tests for content length validation
- Update post handler tests with proper context setup
- Update E2E tests for nested external embed structure
- Add helper for creating communities with PDS credentials
- Add createTestUser helper for unique test isolation
**Test Isolation:**
- Use unique DIDs per test (via t.Name() suffix)
- Prevent cross-test data contamination
- Proper cleanup with defer db.Close()
**Example Validation:**
```go
// ❌ This should fail validation:
"thumb": "https://example.com/thumb.jpg" // URL string
// ✅ This should pass validation:
"thumb": {
"$type": "blob",
"ref": {"$link": "bafyrei..."},
"mimeType": "image/jpeg",
"size": 52813
}
```
**Coverage:**
- Blob validation: 7 test cases
- Blob transformation: 6 test cases
- Feed integration: 99 added lines in feed_test.go
- Total: 13 new test scenarios
This ensures security (no URL injection), correctness (proper blob format),
and functionality (URLs work in API responses).
Update external embed lexicon to use proper nested structure with dedicated
external object, aligning with atproto conventions and enabling better validation.
**Schema Changes:**
1. Main object now requires "external" property (was flat structure)
2. Add dedicated "external" definition with link metadata
3. Update embedType known values:
- OLD: ["article", "image", "video-stream"]
- NEW: ["article", "image", "video", "website"]
- Removed "video-stream" (use "video" instead)
- Added "website" for generic pages
**Before (flat structure):**
```json
{
"$type": "social.coves.embed.external",
"uri": "https://example.com",
"title": "Example",
"thumb": {...}
}
```
**After (nested structure):**
```json
{
"$type": "social.coves.embed.external",
"external": {
"uri": "https://example.com",
"title": "Example",
"thumb": {...}
}
}
```
**Rationale:**
- Follows atproto pattern (app.bsky.embed.external uses same structure)
- Enables future extensibility (can add external-level metadata)
- Clearer separation between embed type and embedded content
- Better validation with required "external" property
**embedType Values:**
- "article": Blog posts, news articles (rich text content)
- "image": Image galleries, photos (visual content)
- "video": Video embeds from Streamable, YouTube, etc.
- "website": Generic web pages without specific type
This aligns our lexicon with atproto best practices and prepares for
potential federation with other atproto implementations.
Breaking change: Clients must update to use nested structure.