···
1
+
# Federation PRD: Cross-Instance Posting (Beta)
3
+
**Status:** Planning - Beta
4
+
**Target:** Beta Release
6
+
**Last Updated:** 2025-11-16
12
+
Enable Lemmy-style federation where users on any Coves instance can post to communities hosted on other instances, while maintaining community ownership and moderation control.
14
+
### Problem Statement
16
+
**Current (Alpha):**
17
+
- Posts to communities require community credentials
18
+
- Users can only post to communities on their home instance
19
+
- No true federation across instances
22
+
- User A@coves.social can post to !gaming@covesinstance.com
23
+
- Communities maintain full moderation control
24
+
- Content lives in community repositories (not user repos)
25
+
- Seamless UX - users don't think about federation
32
+
1. **Enable cross-instance posting** - Users can post to any community on any federated instance
33
+
2. **Preserve community ownership** - Posts live in community repos, not user repos
34
+
3. **atProto-native implementation** - Use `com.atproto.server.getServiceAuth` pattern
35
+
4. **Maintain security** - No compromise on auth, validation, or moderation
37
+
### Non-Goals (Future Versions)
38
+
- Automatic instance discovery (Beta: manual allowlist)
39
+
- Cross-instance moderation delegation
40
+
- Content mirroring/replication
41
+
- User migration between instances
45
+
## Technical Approach
47
+
### Architecture: atProto Service Auth
49
+
Use atProto's native service authentication delegation pattern:
52
+
┌─────────────┐ ┌──────────────────┐ ┌─────────────┐
53
+
│ User A │ │ coves.social │ │ covesinstance│
54
+
│ @coves.soc │────────▶│ AppView │────────▶│ .com PDS │
55
+
└─────────────┘ (1) └──────────────────┘ (2) └─────────────┘
56
+
JWT auth Request Service Auth Validate
58
+
│◀──────────────────────┘
62
+
┌──────────────────┐
66
+
└──────────────────┘
69
+
┌──────────────────┐
72
+
└──────────────────┘
74
+
┌────────────┴────────────┐
76
+
┌──────────────┐ ┌──────────────┐
77
+
│ coves.social │ │covesinstance │
78
+
│ AppView │ │ .com AppView│
79
+
│ (indexes) │ │ (indexes) │
80
+
└──────────────┘ └──────────────┘
85
+
**Step 1: User Authentication (Unchanged)**
86
+
- User authenticates with their home instance (coves.social)
87
+
- Receives JWT token for API requests
89
+
**Step 2: Service Auth Request (New)**
90
+
- When posting to remote community, AppView requests service auth token
91
+
- Endpoint: `POST {remote-pds}/xrpc/com.atproto.server.getServiceAuth`
95
+
"aud": "did:plc:community123", // Community DID
96
+
"exp": 1234567890, // Token expiration
97
+
"lxm": "social.coves.community.post.create" // Authorized method
101
+
**Step 3: Service Auth Validation (New - PDS Side)**
102
+
- Remote PDS validates request:
103
+
- Is requesting service trusted? (instance allowlist)
104
+
- Is user banned from community?
105
+
- Does community allow remote posts?
106
+
- Rate limiting checks
107
+
- Returns scoped token valid for specific community + operation
109
+
**Step 4: Post Creation (Modified)**
110
+
- AppView uses service auth token to write to remote PDS
111
+
- Same `com.atproto.repo.createRecord` endpoint as current implementation
112
+
- Post record written to community's repository
114
+
**Step 5: Indexing (Unchanged)**
115
+
- PDS broadcasts to firehose
116
+
- All AppViews index via Jetstream consumers
120
+
## Implementation Details
122
+
### Phase 1: Service Detection (Local vs Remote)
124
+
**File:** `internal/core/posts/service.go`
127
+
func (s *postService) CreatePost(ctx context.Context, req CreatePostRequest) (*CreatePostResponse, error) {
128
+
// ... existing validation ...
130
+
community, err := s.communityService.GetByDID(ctx, communityDID)
135
+
// NEW: Route based on community location
136
+
if s.isLocalCommunity(community) {
137
+
return s.createLocalPost(ctx, community, req)
139
+
return s.createFederatedPost(ctx, community, req)
142
+
func (s *postService) isLocalCommunity(community *communities.Community) bool {
143
+
localPDSHost := extractHost(s.pdsURL)
144
+
communityPDSHost := extractHost(community.PDSURL)
145
+
return localPDSHost == communityPDSHost
149
+
### Phase 2: Service Auth Client
151
+
**New File:** `internal/atproto/service_auth/client.go`
154
+
type ServiceAuthClient interface {
155
+
// RequestServiceAuth obtains a scoped token for writing to remote community
156
+
RequestServiceAuth(ctx context.Context, opts ServiceAuthOptions) (*ServiceAuthToken, error)
159
+
type ServiceAuthOptions struct {
160
+
RemotePDSURL string // Remote PDS endpoint
161
+
CommunityDID string // Target community DID
162
+
UserDID string // Author DID (for validation)
163
+
Method string // "social.coves.community.post.create"
164
+
ExpiresIn int // Token lifetime (seconds)
167
+
type ServiceAuthToken struct {
168
+
Token string // JWT token for auth
169
+
ExpiresAt time.Time // When token expires
172
+
func (c *serviceAuthClient) RequestServiceAuth(ctx context.Context, opts ServiceAuthOptions) (*ServiceAuthToken, error) {
173
+
endpoint := fmt.Sprintf("%s/xrpc/com.atproto.server.getServiceAuth", opts.RemotePDSURL)
175
+
payload := map[string]interface{}{
176
+
"aud": opts.CommunityDID,
177
+
"exp": time.Now().Add(time.Duration(opts.ExpiresIn) * time.Second).Unix(),
178
+
"lxm": opts.Method,
181
+
// Sign request with our instance DID credentials
182
+
signedReq, err := c.signRequest(payload)
184
+
return nil, fmt.Errorf("failed to sign service auth request: %w", err)
187
+
resp, err := c.httpClient.Post(endpoint, signedReq)
189
+
return nil, fmt.Errorf("service auth request failed: %w", err)
192
+
return parseServiceAuthResponse(resp)
196
+
### Phase 3: Federated Post Creation
198
+
**File:** `internal/core/posts/service.go`
201
+
func (s *postService) createFederatedPost(ctx context.Context, community *communities.Community, req CreatePostRequest) (*CreatePostResponse, error) {
202
+
// 1. Request service auth token from remote PDS
203
+
token, err := s.serviceAuthClient.RequestServiceAuth(ctx, service_auth.ServiceAuthOptions{
204
+
RemotePDSURL: community.PDSURL,
205
+
CommunityDID: community.DID,
206
+
UserDID: req.AuthorDID,
207
+
Method: "social.coves.community.post.create",
208
+
ExpiresIn: 300, // 5 minutes
211
+
// Handle specific errors
212
+
if isUnauthorized(err) {
213
+
return nil, ErrNotAuthorizedRemote
216
+
return nil, ErrBannedRemote
218
+
return nil, fmt.Errorf("failed to obtain service auth: %w", err)
221
+
// 2. Build post record (same as local)
222
+
postRecord := PostRecord{
223
+
Type: "social.coves.community.post",
224
+
Community: community.DID,
225
+
Author: req.AuthorDID,
227
+
Content: req.Content,
228
+
// ... other fields ...
229
+
CreatedAt: time.Now().UTC().Format(time.RFC3339),
232
+
// 3. Write to remote PDS using service auth token
233
+
uri, cid, err := s.createPostOnRemotePDS(ctx, community.PDSURL, community.DID, postRecord, token.Token)
235
+
return nil, fmt.Errorf("failed to write to remote PDS: %w", err)
238
+
log.Printf("[FEDERATION] User %s posted to remote community %s: %s",
239
+
req.AuthorDID, community.DID, uri)
241
+
return &CreatePostResponse{
247
+
func (s *postService) createPostOnRemotePDS(
248
+
ctx context.Context,
250
+
communityDID string,
252
+
serviceAuthToken string,
253
+
) (uri, cid string, err error) {
254
+
endpoint := fmt.Sprintf("%s/xrpc/com.atproto.repo.createRecord", pdsURL)
256
+
payload := map[string]interface{}{
257
+
"repo": communityDID,
258
+
"collection": "social.coves.community.post",
262
+
jsonData, _ := json.Marshal(payload)
263
+
req, _ := http.NewRequestWithContext(ctx, "POST", endpoint, bytes.NewBuffer(jsonData))
265
+
// Use service auth token instead of community credentials
266
+
req.Header.Set("Authorization", "Bearer "+serviceAuthToken)
267
+
req.Header.Set("Content-Type", "application/json")
269
+
// ... execute request, parse response ...
270
+
return uri, cid, nil
274
+
### Phase 4: PDS Service Auth Validation (PDS Extension)
276
+
**Note:** This requires extending the PDS. Options:
277
+
1. Contribute to official atproto PDS
278
+
2. Run modified PDS fork
279
+
3. Use PDS middleware/proxy
281
+
**Conceptual Implementation:**
284
+
// PDS validates service auth requests before issuing tokens
285
+
func (h *ServiceAuthHandler) HandleGetServiceAuth(w http.ResponseWriter, r *http.Request) {
286
+
var req ServiceAuthRequest
287
+
json.NewDecoder(r.Body).Decode(&req)
289
+
// 1. Verify requesting service is trusted
290
+
requestingDID := extractDIDFromJWT(r.Header.Get("Authorization"))
291
+
if !h.isTrustedInstance(requestingDID) {
292
+
writeError(w, http.StatusForbidden, "UntrustedInstance", "Instance not in allowlist")
296
+
// 2. Validate community exists on this PDS
297
+
community, err := h.getCommunityByDID(req.Aud)
299
+
writeError(w, http.StatusNotFound, "CommunityNotFound", "Community not hosted here")
303
+
// 3. Check user not banned (query from AppView or local moderation records)
304
+
if h.isUserBanned(req.UserDID, req.Aud) {
305
+
writeError(w, http.StatusForbidden, "Banned", "User banned from community")
309
+
// 4. Check community settings (allows remote posts?)
310
+
if !community.AllowFederatedPosts {
311
+
writeError(w, http.StatusForbidden, "FederationDisabled", "Community doesn't accept federated posts")
315
+
// 5. Rate limiting (per user, per community, per instance)
316
+
if h.exceedsRateLimit(req.UserDID, req.Aud, requestingDID) {
317
+
writeError(w, http.StatusTooManyRequests, "RateLimited", "Too many requests")
321
+
// 6. Generate scoped token
322
+
token := h.issueServiceAuthToken(ServiceAuthTokenOptions{
323
+
Audience: req.Aud, // Community DID
324
+
Subject: requestingDID, // Requesting instance DID
325
+
Method: req.Lxm, // Authorized method
326
+
ExpiresAt: time.Unix(req.Exp, 0),
327
+
Scopes: []string{"write:posts"},
330
+
json.NewEncoder(w).Encode(map[string]string{
338
+
## Database Schema Changes
340
+
### New Table: `instance_federation`
342
+
Tracks trusted instances and federation settings:
345
+
CREATE TABLE instance_federation (
346
+
id SERIAL PRIMARY KEY,
347
+
instance_did TEXT NOT NULL UNIQUE,
348
+
instance_domain TEXT NOT NULL,
349
+
trust_level TEXT NOT NULL, -- 'trusted', 'limited', 'blocked'
350
+
allowed_methods TEXT[] NOT NULL DEFAULT '{}',
351
+
rate_limit_posts_per_hour INTEGER NOT NULL DEFAULT 100,
352
+
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
353
+
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
357
+
CREATE INDEX idx_instance_federation_did ON instance_federation(instance_did);
358
+
CREATE INDEX idx_instance_federation_trust ON instance_federation(trust_level);
361
+
### New Table: `federation_rate_limits`
363
+
Track federated post rate limits:
366
+
CREATE TABLE federation_rate_limits (
367
+
id SERIAL PRIMARY KEY,
368
+
user_did TEXT NOT NULL,
369
+
community_did TEXT NOT NULL,
370
+
instance_did TEXT NOT NULL,
371
+
window_start TIMESTAMPTZ NOT NULL,
372
+
post_count INTEGER NOT NULL DEFAULT 1,
373
+
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
374
+
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
376
+
UNIQUE(user_did, community_did, instance_did, window_start)
379
+
CREATE INDEX idx_federation_rate_limits_lookup
380
+
ON federation_rate_limits(user_did, community_did, instance_did, window_start);
383
+
### Update Table: `communities`
385
+
Add federation settings:
388
+
ALTER TABLE communities
389
+
ADD COLUMN allow_federated_posts BOOLEAN NOT NULL DEFAULT true,
390
+
ADD COLUMN federation_mode TEXT NOT NULL DEFAULT 'open';
391
+
-- federation_mode: 'open' (any instance), 'allowlist' (trusted only), 'local' (no federation)
396
+
## Security Considerations
398
+
### 1. Instance Trust Model
400
+
**Allowlist Approach (Beta):**
401
+
- Manual approval of federated instances
402
+
- Admin UI to manage instance trust levels
403
+
- Default: block all, explicit allow
406
+
- `trusted` - Full federation, normal rate limits
407
+
- `limited` - Federation allowed, strict rate limits
408
+
- `blocked` - No federation
410
+
### 2. User Ban Synchronization
412
+
**Challenge:** Remote instance needs to check local bans
415
+
1. **Service auth validation** - PDS queries AppView for ban status
416
+
2. **Ban records in PDS** - Moderation records stored in community repo
417
+
3. **Cached ban list** - Remote instances cache ban lists (with TTL)
419
+
**Beta Approach:** Option 1 (service auth validation queries AppView)
421
+
### 3. Rate Limiting
423
+
**Multi-level rate limits:**
424
+
- Per user per community: 10 posts/hour
425
+
- Per instance per community: 100 posts/hour
426
+
- Per user across all communities: 50 posts/hour
428
+
**Implementation:** In-memory + PostgreSQL for persistence
430
+
### 4. Content Validation
432
+
**Same validation as local posts:**
433
+
- Lexicon validation
434
+
- Content length limits
438
+
**Additional federation checks:**
439
+
- Verify author DID is valid
440
+
- Verify requesting instance signature
441
+
- Verify token scopes match operation
447
+
### New Endpoint: `social.coves.federation.getTrustedInstances`
449
+
**Purpose:** List instances this instance federates with
455
+
"id": "social.coves.federation.getTrustedInstances",
460
+
"encoding": "application/json",
463
+
"required": ["instances"],
467
+
"items": { "$ref": "#instanceView" }
475
+
"required": ["did", "domain", "trustLevel"],
477
+
"did": { "type": "string" },
478
+
"domain": { "type": "string" },
479
+
"trustLevel": { "type": "string" },
480
+
"allowedMethods": { "type": "array", "items": { "type": "string" } }
487
+
### Modified Endpoint: `social.coves.community.post.create`
490
+
- No API contract changes
491
+
- Internal routing: local vs federated
493
+
- `FederationFailed` - Remote instance unreachable
494
+
- `RemoteNotAuthorized` - Remote instance rejected auth
495
+
- `RemoteBanned` - User banned on remote community
501
+
### Happy Path: Cross-Instance Post
503
+
1. User on coves.social navigates to !gaming@covesinstance.com
504
+
2. Clicks "Create Post"
505
+
3. Fills out post form (title, content, etc.)
507
+
5. **Behind the scenes:**
508
+
- coves.social requests service auth from covesinstance.com
509
+
- covesinstance.com validates and issues token
510
+
- coves.social writes post using token
511
+
- Post appears in feed within seconds (via firehose)
512
+
6. **User sees:** Post published successfully
513
+
7. Post appears in:
514
+
- covesinstance.com feeds (native community)
515
+
- coves.social discover/all feeds (indexed via firehose)
516
+
- User's profile on coves.social
521
+
- Error: "You are banned from !gaming@covesinstance.com"
522
+
- Suggestion: "Contact community moderators for more information"
524
+
**Instance Blocked:**
525
+
- Error: "This community does not accept posts from your instance"
526
+
- Suggestion: "Contact community administrators or create a local account"
528
+
**Federation Unavailable:**
529
+
- Error: "Unable to connect to covesinstance.com. Try again later."
530
+
- Fallback: Allow saving as draft (future feature)
533
+
- Error: "You're posting too quickly. Please wait before posting again."
534
+
- Show: Countdown until next post allowed
538
+
## Testing Requirements
542
+
1. **Service Detection:**
543
+
- `isLocalCommunity()` correctly identifies local vs remote
544
+
- Handles edge cases (different ports, subdomains)
546
+
2. **Service Auth Client:**
547
+
- Correctly formats service auth requests
548
+
- Handles token expiration
549
+
- Retries on transient failures
551
+
3. **Federated Post Creation:**
552
+
- Uses service auth token instead of community credentials
553
+
- Falls back gracefully on errors
554
+
- Logs federation events
556
+
### Integration Tests
558
+
1. **Local Post (Regression):**
559
+
- Posting to local community still works
560
+
- No performance degradation
562
+
2. **Federated Post:**
563
+
- User can post to remote community
564
+
- Service auth token requested correctly
565
+
- Post written to remote PDS
566
+
- Post indexed by both AppViews
568
+
3. **Authorization Failures:**
569
+
- Banned users rejected at service auth stage
570
+
- Untrusted instances rejected
571
+
- Expired tokens rejected
573
+
4. **Rate Limiting:**
574
+
- Per-user rate limits enforced
575
+
- Per-instance rate limits enforced
576
+
- Rate limit resets correctly
578
+
### End-to-End Tests
580
+
1. **Cross-Instance User Journey:**
581
+
- Set up two instances (instance-a, instance-b)
582
+
- Create community on instance-b
583
+
- User on instance-a posts to instance-b community
584
+
- Verify post appears on both instances
586
+
2. **Moderation Enforcement:**
587
+
- Ban user on remote instance
588
+
- Verify user can't post from any instance
590
+
- Verify user can post again
592
+
3. **Instance Blocklist:**
593
+
- Block instance-a on instance-b
594
+
- Verify users from instance-a can't post to instance-b communities
595
+
- Unblock instance-a
596
+
- Verify posting works again
600
+
## Migration Path (Alpha → Beta)
602
+
### Phase 1: Backend Implementation (No User Impact)
603
+
1. Add service auth client
604
+
2. Add local vs remote detection
605
+
3. Deploy with feature flag `ENABLE_FEDERATION=false`
607
+
### Phase 2: Database Migration
608
+
1. Add federation tables
609
+
2. Seed with initial trusted instances (manual)
610
+
3. Add community federation flags (default: allow)
612
+
### Phase 3: Soft Launch
613
+
1. Enable federation for single test instance
614
+
2. Monitor service auth requests/errors
615
+
3. Validate rate limiting works
617
+
### Phase 4: Beta Rollout
618
+
1. Enable `ENABLE_FEDERATION=true` for all instances
619
+
2. Admin UI for managing trusted instances
620
+
3. Community settings for federation preferences
622
+
### Phase 5: Documentation & Onboarding
623
+
1. Instance operator guide: "How to federate with other instances"
624
+
2. Community moderator guide: "Federation settings"
625
+
3. User guide: "Posting across instances"
629
+
## Metrics & Success Criteria
631
+
### Performance Metrics
632
+
- Service auth request latency: p95 < 200ms
633
+
- Federated post creation time: p95 < 2 seconds (vs 500ms local)
634
+
- Service auth token cache hit rate: > 80%
636
+
### Adoption Metrics
637
+
- % of posts that are federated: Target 20% by end of Beta
638
+
- Number of federated instances: Target 5+ by end of Beta
639
+
- Cross-instance engagement (comments, votes): Monitor trend
641
+
### Reliability Metrics
642
+
- Service auth success rate: > 99%
643
+
- Federated post success rate: > 95%
644
+
- Service auth token validation errors: < 1%
646
+
### Security Metrics
647
+
- Unauthorized access attempts: Monitor & alert
648
+
- Rate limit triggers: Track per instance
649
+
- Ban evasion attempts: Zero tolerance
655
+
If federation causes critical issues:
657
+
1. **Immediate:** Set `ENABLE_FEDERATION=false` via env var
658
+
2. **Fallback:** All posts route through local-only flow
659
+
3. **Investigation:** Review logs for service auth failures
660
+
4. **Fix Forward:** Deploy patch, re-enable gradually
662
+
**No data loss:** Posts are written to PDS, indexed via firehose regardless of federation method.
668
+
1. **Instance Discovery:** How do users find communities on other instances?
669
+
- Beta: Manual (users share links)
670
+
- Future: Instance directory, community search across instances
672
+
2. **Service Auth Token Caching:** Should AppViews cache service auth tokens?
673
+
- Pros: Reduce latency, fewer PDS requests
674
+
- Cons: Stale permissions, ban enforcement delay
675
+
- **Decision needed:** Cache with short TTL (5 minutes)?
677
+
3. **PDS Implementation:** Who implements service auth validation?
678
+
- Option A: Contribute to official PDS (long timeline)
679
+
- Option B: Run forked PDS (maintenance burden)
680
+
- Option C: Proxy/middleware (added complexity)
681
+
- **Decision needed:** Start with Option B, migrate to Option A?
683
+
4. **Federation Symmetry:** If instance-a trusts instance-b, does instance-b auto-trust instance-a?
684
+
- Beta: No (asymmetric trust)
685
+
- Future: Mutual federation agreements?
687
+
5. **Cross-Instance Moderation:** Should bans propagate across instances?
688
+
- Beta: No (each instance decides)
689
+
- Future: Shared moderation lists?
693
+
## Future Enhancements (Post-Beta)
695
+
1. **Service Auth Token Caching:** Reduce latency for frequent posters
696
+
2. **Batch Service Auth:** Request tokens for multiple communities at once
697
+
3. **Instance Discovery API:** Automatic instance detection/registration
698
+
4. **Federation Analytics:** Dashboard showing cross-instance activity
699
+
5. **Moderation Sync:** Optional shared ban lists across trusted instances
700
+
6. **Content Mirroring:** Cache federated posts locally for performance
701
+
7. **User Migration:** Transfer account between instances
708
+
- [atProto Service Auth Spec](https://atproto.com/specs/service-auth) (hypothetical - check actual docs)
709
+
- Lemmy Federation Architecture
710
+
- Mastodon Federation Implementation
712
+
### Code References
713
+
- `internal/core/posts/service.go` - Post creation service
714
+
- `internal/api/handlers/post/create.go` - Post creation handler
715
+
- `internal/atproto/jetstream/` - Firehose consumers
718
+
- atproto SDK (for service auth)
719
+
- PDS v0.4+ (service auth support)
720
+
- PostgreSQL 14+ (for federation tables)
724
+
## Appendix A: Service Auth Request Example
726
+
**Request to Remote PDS:**
728
+
POST https://covesinstance.com/xrpc/com.atproto.server.getServiceAuth
729
+
Authorization: Bearer {coves-social-instance-jwt}
730
+
Content-Type: application/json
733
+
"aud": "did:plc:community123",
735
+
"lxm": "social.coves.community.post.create"
742
+
Content-Type: application/json
745
+
"token": "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9..."
749
+
**Using Token to Create Post:**
751
+
POST https://covesinstance.com/xrpc/com.atproto.repo.createRecord
752
+
Authorization: Bearer {service-auth-token}
753
+
Content-Type: application/json
756
+
"repo": "did:plc:community123",
757
+
"collection": "social.coves.community.post",
759
+
"$type": "social.coves.community.post",
760
+
"community": "did:plc:community123",
761
+
"author": "did:plc:user456",
762
+
"title": "Hello from coves.social!",
763
+
"content": "This is a federated post",
764
+
"createdAt": "2024-11-16T12:00:00Z"
771
+
## Appendix B: Error Handling Matrix
773
+
| Error Condition | HTTP Status | Error Code | User Message | Retry Strategy |
774
+
|----------------|-------------|------------|--------------|----------------|
775
+
| Instance not trusted | 403 | `UntrustedInstance` | "This community doesn't accept posts from your instance" | No retry |
776
+
| User banned | 403 | `Banned` | "You are banned from this community" | No retry |
777
+
| Rate limit exceeded | 429 | `RateLimited` | "Too many posts. Try again in X minutes" | Exponential backoff |
778
+
| PDS unreachable | 503 | `ServiceUnavailable` | "Community temporarily unavailable" | Retry 3x with backoff |
779
+
| Invalid token | 401 | `InvalidToken` | "Session expired. Please try again" | Refresh token & retry |
780
+
| Community not found | 404 | `CommunityNotFound` | "Community not found" | No retry |
781
+
| Service auth failed | 500 | `FederationFailed` | "Unable to connect. Try again later" | Retry 2x |