commit ad399db5f5faa862f60313aaaf45a352d9e7db5c · cduck.me/statusphere-example-app-fork

+14 -8

README.md

···

       1
       -
       # AT Protocol Express App

     

       2
        
       

     

       3
       -
       A demo application covering:

     

       4
       -
        - public firehose ingestion

     

       5
       -
        - identity and login with OAuth

     

       6
       -
        - writing to the network

     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       0
        
       
     

       7
        
       

     

       8
        
       ## Getting Started

     

       9
       -
       ### Development

     

       10
        
       ```sh

     

       11
       -
       pnpm i

     

       0
        
       
     

       12
        
       cp .env.template .env

     

       13
       -
       pnpm run dev

     

       0
        
       
     

       14
        
       # Navigate to http://localhost:8080

     

       15
        
       ```

···

       1
       +
       # AT Protocol "Statusphere" Example App

     

       2
        
       

     

       3
       +
       An example application covering:

     

       4
       +
       

     

       5
       +
       - Signin via OAuth

     

       6
       +
       - Fetch information about users (profiles)

     

       7
       +
       - Listen to the network firehose for new data

     

       8
       +
       - Publish data on the user's account using a custom schema

     

       9
       +
       

     

       10
       +
       See https://atproto.com/guides/applications for a guide through the codebase.

     

       11
        
       

     

       12
        
       ## Getting Started

     

       13
       +
       

     

       14
        
       ```sh

     

       15
       +
       git clone https://github.com/bluesky-social/statusphere-example-app.git

     

       16
       +
       cd statusphere-example-app

     

       17
        
       cp .env.template .env

     

       18
       +
       npm install

     

       19
       +
       npm run dev

     

       20
        
       # Navigate to http://localhost:8080

     

       21
        
       ```

-644

TUTORIAL.md

···

       1
       -
       # Tutorial

     

       2
       -
       

     

       3
       -
       In this guide, we're going to build a simple multi-user app that publishes your current "status" as an emoji.

     

       4
       -
       

     

       5
       -
       ![A screenshot of our example application](./docs/app-screenshot.png)

     

       6
       -
       

     

       7
       -
       At various points we will cover how to:

     

       8
       -
       

     

       9
       -
       - Signin via OAuth

     

       10
       -
       - Fetch information about users (profiles)

     

       11
       -
       - Listen to the network firehose for new data

     

       12
       -
       - Publish data on the user's account using a custom schema

     

       13
       -
       

     

       14
       -
       We're going to keep this light so you can quickly wrap your head around ATProto. There will be links with more information about each step.

     

       15
       -
       

     

       16
       -
       ## Where are we going?

     

       17
       -
       

     

       18
       -
       Data in the Atmosphere is stored on users' personal repos. It's almost like each user has their own website. Our goal is to aggregate data from the users into our SQLite DB. 

     

       19
       -
       

     

       20
       -
       Think of our app like a Google. If Google's job was to say which emoji each website had under `/status.json`, then it would show something like:

     

       21
       -
       

     

       22
       -
       - `nytimes.com` is feeling 📰 according to `https://nytimes.com/status.json`

     

       23
       -
       - `bsky.app` is feeling 🦋 according to `https://bsky.app/status.json`

     

       24
       -
       - `reddit.com` is feeling 🤓 according to `https://reddit.com/status.json`

     

       25
       -
       

     

       26
       -
       The Atmosphere works the same way, except we're going to check `at://` instead of `https://`. Each user has a data repo under an `at://` URL. We'll crawl all the `at://`s in the Atmosphere for all the  "status.json" records and aggregate them into our SQLite database.

     

       27
       -
       

     

       28
       -
       > `at://` is the URL scheme of the AT Protocol. Under the hood it uses common tech like HTTP and DNS, but it adds all of the features we'll be using in this tutorial.

     

       29
       -
       

     

       30
       -
       ## Step 1. Starting with our ExpressJS app

     

       31
       -
       

     

       32
       -
       Start by cloning the repo and installing packages.

     

       33
       -
       

     

       34
       -
       ```bash

     

       35
       -
       git clone TODO

     

       36
       -
       cd TODO

     

       37
       -
       npm i

     

       38
       -
       npm run dev # you can leave this running and it will auto-reload

     

       39
       -
       ```

     

       40
       -
       

     

       41
       -
       Our repo is a regular Web app. We're rendering our HTML server-side like it's 1999. We also have a SQLite database that we're managing with [Kysley](https://kysely.dev/).

     

       42
       -
       

     

       43
       -
       Our starting stack:

     

       44
       -
       

     

       45
       -
       - Typescript

     

       46
       -
       - NodeJS web server ([express](https://expressjs.com/))

     

       47
       -
       - SQLite database ([Kysley](https://kysely.dev/))

     

       48
       -
       - Server-side rendering ([uhtml](https://www.npmjs.com/package/uhtml))

     

       49
       -
       

     

       50
       -
       With each step we'll explain how our Web app taps into the Atmosphere. Refer to the codebase for more detailed code &mdash; again, this tutorial is going to keep it light and quick to digest.

     

       51
       -
       

     

       52
       -
       ## Step 2. Signing in with OAuth

     

       53
       -
       

     

       54
       -
       When somebody logs into our app, they'll give us read & write access to their personal `at://` repo. We'll use that to write the `status.json` record.

     

       55
       -
       

     

       56
       -
       We're going to accomplish this using OAuth ([spec](https://github.com/bluesky-social/proposals/tree/main/0004-oauth)). Most of the OAuth flows are going to be handled for us using the [@atproto/oauth-client-node](https://github.com/bluesky-social/atproto/tree/main/packages/oauth/oauth-client-node) library. This is the arrangement we're aiming toward:

     

       57
       -
       

     

       58
       -
       ![A diagram of the OAuth elements](./docs/diagram-oauth.png)

     

       59
       -
       

     

       60
       -
       When the user logs in, the OAuth client will create a new session with their repo server and give us read/write access along with basic user info.

     

       61
       -
       

     

       62
       -
       ![A screenshot of the login UI](./docs/app-login.png)

     

       63
       -
       

     

       64
       -
       Our login page just asks the user for their "handle," which is the domain name associated with their account. For [Bluesky](https://bsky.app) users, these tend to look like `alice.bsky.social`, but they can be any kind of domain (eg `alice.com`).

     

       65
       -
       

     

       66
       -
       ```html

     

       67
       -
       <!-- src/pages/login.ts -->

     

       68
       -
       <form action="/login" method="post" class="login-form">

     

       69
       -
         <input

     

       70
       -
           type="text"

     

       71
       -
           name="handle"

     

       72
       -
           placeholder="Enter your handle (eg alice.bsky.social)"

     

       73
       -
           required

     

       74
       -
         />

     

       75
       -
         <button type="submit">Log in</button>

     

       76
       -
       </form>

     

       77
       -
       ```

     

       78
       -
       

     

       79
       -
       When they submit the form, we tell our OAuth client to initiate the authorization flow and then redirect the user to their server to complete the process.

     

       80
       -
       

     

       81
       -
       ```typescript

     

       82
       -
       /** src/routes.ts **/

     

       83
       -
       // Login handler

     

       84
       -
       router.post(

     

       85
       -
         '/login',

     

       86
       -
         handler(async (req, res) => {

     

       87
       -
           // Initiate the OAuth flow

     

       88
       -
           const url = await oauthClient.authorize(handle)

     

       89
       -
           return res.redirect(url.toString())

     

       90
       -
         })

     

       91
       -
       )

     

       92
       -
       ```

     

       93
       -
       

     

       94
       -
       This is the same kind of SSO flow that Google or GitHub uses. The user will be asked for their password, then asked to confirm the session with your application.

     

       95
       -
       

     

       96
       -
       When that finishes, they'll be sent back to `/oauth/callback` on our Web app. The OAuth client stores the access tokens for the server, and then we attach their account's [DID](https://atproto.com/specs/did) to their cookie-session.

     

       97
       -
       

     

       98
       -
       ```typescript

     

       99
       -
       /** src/routes.ts **/

     

       100
       -
       // OAuth callback to complete session creation

     

       101
       -
       router.get(

     

       102
       -
         '/oauth/callback',

     

       103
       -
         handler(async (req, res) => {

     

       104
       -
           // Store the credentials

     

       105
       -
           const { agent } = await oauthClient.callback(params)

     

       106
       -
       

     

       107
       -
           // Attach the account DID to our user via a cookie

     

       108
       -
           const session = await getIronSession(req, res)

     

       109
       -
           session.did = agent.accountDid

     

       110
       -
           await session.save()

     

       111
       -
       

     

       112
       -
           // Send them back to the app

     

       113
       -
           return res.redirect('/')

     

       114
       -
         })

     

       115
       -
       )

     

       116
       -
       ```

     

       117
       -
       

     

       118
       -
       With that, we're in business! We now have a session with the user's `at://` repo server and can use that to access their data.

     

       119
       -
       

     

       120
       -
       ## Step 3. Fetching the user's profile

     

       121
       -
       

     

       122
       -
       Why don't we learn something about our user? In [Bluesky](https://bsky.app), users publish a "profile" record which looks like this:

     

       123
       -
       

     

       124
       -
       ```typescript

     

       125
       -
       interface ProfileRecord {

     

       126
       -
         displayName?: string // a human friendly name

     

       127
       -
         description?: string // a short bio

     

       128
       -
         avatar?: BlobRef     // small profile picture

     

       129
       -
         banner?: BlobRef     // banner image to put on profiles

     

       130
       -
         createdAt?: string   // declared time this profile data was added

     

       131
       -
         // ...

     

       132
       -
       }

     

       133
       -
       ```

     

       134
       -
       

     

       135
       -
       You can examine this record directly using [atproto-browser.vercel.app](https://atproto-browser.vercel.app). For instance, [this is the profile record for @bsky.app](https://atproto-browser.vercel.app/at?u=at://did:plc:z72i7hdynmk6r22z27h6tvur/app.bsky.actor.profile/self).

     

       136
       -
       

     

       137
       -
       We're going to use the [Agent](https://github.com/bluesky-social/atproto/tree/main/packages/api) associated with the user's OAuth session to fetch this record.

     

       138
       -
       

     

       139
       -
       ```typescript

     

       140
       -
       await agent.getRecord({

     

       141
       -
         repo: agent.accountDid,               // The user

     

       142
       -
         collection: 'app.bsky.actor.profile', // The collection

     

       143
       -
         rkey: 'self',                         // The record key

     

       144
       -
       })

     

       145
       -
       ```

     

       146
       -
       

     

       147
       -
       When asking for a record, we provide three pieces of information.

     

       148
       -
       

     

       149
       -
       - **repo** The [DID](https://atproto.com/specs/did) which identifies the user,

     

       150
       -
       - **collection** The collection name, and

     

       151
       -
       - **rkey** The record key

     

       152
       -
       

     

       153
       -
       We'll explain the collection name shortly. Record keys are strings with [some restrictions](https://atproto.com/specs/record-key#record-key-syntax) and a couple of common patterns. The `"self"` pattern is used when a collection is expected to only contain one record which describes the user.

     

       154
       -
       

     

       155
       -
       Let's update our homepage to fetch this profile record:

     

       156
       -
       

     

       157
       -
       ```typescript

     

       158
       -
       /** src/routes.ts **/

     

       159
       -
       // Homepage

     

       160
       -
       router.get(

     

       161
       -
         '/',

     

       162
       -
         handler(async (req, res) => {

     

       163
       -
           // If the user is signed in, get an agent which communicates with their server

     

       164
       -
           const agent = await getSessionAgent(req, res, ctx)

     

       165
       -
       

     

       166
       -
           if (!agent) {

     

       167
       -
             // Serve the logged-out view

     

       168
       -
             return res.type('html').send(page(home()))

     

       169
       -
           }

     

       170
       -
       

     

       171
       -
           // Fetch additional information about the logged-in user

     

       172
       -
           const { data: profileRecord } = await agent.getRecord({

     

       173
       -
             repo: agent.accountDid,               // our user's repo

     

       174
       -
             collection: 'app.bsky.actor.profile', // the bluesky profile record type

     

       175
       -
             rkey: 'self',                         // the record's key

     

       176
       -
           })

     

       177
       -
       

     

       178
       -
           // Serve the logged-in view

     

       179
       -
           return res

     

       180
       -
             .type('html')

     

       181
       -
             .send(page(home({ profile: profileRecord.value || {} })))

     

       182
       -
         })

     

       183
       -
       )

     

       184
       -
       ```

     

       185
       -
       

     

       186
       -
       With that data, we can give a nice personalized welcome banner for our user:

     

       187
       -
       

     

       188
       -
       ![A screenshot of the banner image](./docs/app-banner.png)

     

       189
       -
       

     

       190
       -
       ```html

     

       191
       -
       <!-- pages/home.ts -->

     

       192
       -
       <div class="card">

     

       193
       -
         ${profile

     

       194
       -
           ? html`<form action="/logout" method="post" class="session-form">

     

       195
       -
               <div>

     

       196
       -
                 Hi, <strong>${profile.displayName || 'friend'}</strong>.

     

       197
       -
                 What's your status today?

     

       198
       -
               </div>

     

       199
       -
               <div>

     

       200
       -
                 <button type="submit">Log out</button>

     

       201
       -
               </div>

     

       202
       -
             </form>`

     

       203
       -
           : html`<div class="session-form">

     

       204
       -
               <div><a href="/login">Log in</a> to set your status!</div>

     

       205
       -
               <div>

     

       206
       -
                 <a href="/login" class="button">Log in</a>

     

       207
       -
               </div>

     

       208
       -
             </div>`}

     

       209
       -
       </div>

     

       210
       -
       ```

     

       211
       -
       

     

       212
       -
       ## Step 4. Reading & writing records

     

       213
       -
       

     

       214
       -
       You can think of the user repositories as collections of JSON records:

     

       215
       -
       

     

       216
       -
       ![A diagram of a repository](./docs/diagram-repo.png)

     

       217
       -
       

     

       218
       -
       Let's look again at how we read the "profile" record:

     

       219
       -
       

     

       220
       -
       ```typescript

     

       221
       -
       await agent.getRecord({

     

       222
       -
         repo: agent.accountDid,               // The user

     

       223
       -
         collection: 'app.bsky.actor.profile', // The collection

     

       224
       -
         rkey: 'self',                         // The record key

     

       225
       -
       })

     

       226
       -
       ```

     

       227
       -
       

     

       228
       -
       We write records using a similar API. Since our goal is to write "status" records, let's look at how that will happen:

     

       229
       -
       

     

       230
       -
       ```typescript

     

       231
       -
       // Generate a time-based key for our record

     

       232
       -
       const rkey = TID.nextStr()

     

       233
       -
       

     

       234
       -
       // Write the 

     

       235
       -
       await agent.putRecord({

     

       236
       -
         repo: agent.accountDid,              // The user

     

       237
       -
         collection: 'com.example.status',    // The collection

     

       238
       -
         rkey,                                // The record key

     

       239
       -
         record: {                            // The record value

     

       240
       -
           status: "👍",

     

       241
       -
           createdAt: new Date().toISOString()

     

       242
       -
         }

     

       243
       -
       })

     

       244
       -
       ```

     

       245
       -
       

     

       246
       -
       Our `POST /status` route is going to use this API to publish the user's status to their repo.

     

       247
       -
       

     

       248
       -
       ```typescript

     

       249
       -
       /** src/routes.ts **/

     

       250
       -
       // "Set status" handler

     

       251
       -
       router.post(

     

       252
       -
         '/status',

     

       253
       -
         handler(async (req, res) => {

     

       254
       -
           // If the user is signed in, get an agent which communicates with their server

     

       255
       -
           const agent = await getSessionAgent(req, res, ctx)

     

       256
       -
           if (!agent) {

     

       257
       -
             return res.status(401).type('html').send('<h1>Error: Session required</h1>')

     

       258
       -
           }

     

       259
       -
       

     

       260
       -
           // Construct their status record

     

       261
       -
           const record = {

     

       262
       -
             $type: 'com.example.status',

     

       263
       -
             status: req.body?.status,

     

       264
       -
             createdAt: new Date().toISOString(),

     

       265
       -
           }

     

       266
       -
       

     

       267
       -
           try {

     

       268
       -
             // Write the status record to the user's repository

     

       269
       -
             await agent.putRecord({

     

       270
       -
               repo: agent.accountDid,

     

       271
       -
               collection: 'com.example.status',

     

       272
       -
               rkey: TID.nextStr(),

     

       273
       -
               record,

     

       274
       -
             })

     

       275
       -
           } catch (err) {

     

       276
       -
             logger.warn({ err }, 'failed to write record')

     

       277
       -
             return res.status(500).type('html').send('<h1>Error: Failed to write record</h1>')

     

       278
       -
           }

     

       279
       -
       

     

       280
       -
           res.status(200).json({})

     

       281
       -
         })

     

       282
       -
       )

     

       283
       -
       ```

     

       284
       -
       

     

       285
       -
       Now in our homepage we can list out the status buttons:

     

       286
       -
       

     

       287
       -
       ```html

     

       288
       -
       <!-- src/pages/home.ts -->

     

       289
       -
       <form action="/status" method="post" class="status-options">

     

       290
       -
         ${STATUS_OPTIONS.map(status => html`

     

       291
       -
           <button class="status-option" name="status" value="${status}">

     

       292
       -
             ${status}

     

       293
       -
           </button>

     

       294
       -
         `)}

     

       295
       -
       </form>

     

       296
       -
       ```

     

       297
       -
       

     

       298
       -
       And here we are!

     

       299
       -
       

     

       300
       -
       ![A screenshot of the app's status options](./docs/app-status-options.png)

     

       301
       -
       

     

       302
       -
       ## Step 5. Creating a custom "status" schema

     

       303
       -
       

     

       304
       -
       Repo collections are typed, meaning that they have a defined schema. The `app.bsky.actor.profile` type definition [can be found here](https://github.com/bluesky-social/atproto/blob/main/lexicons/app/bsky/actor/profile.json).

     

       305
       -
       

     

       306
       -
       Anybody can create a new schema using the [Lexicon](https://atproto.com/specs/lexicon) language, which is very similar to [JSON-Schema](http://json-schema.org/). The schemas use [reverse-DNS IDs](https://atproto.com/specs/nsid) which indicate ownership, but for this demo app we're going to use `com.example` which is safe for non-production software.

     

       307
       -
       

     

       308
       -
       > ### Why create a schema?

     

       309
       -
       >

     

       310
       -
       > Schemas help other applications understand the data your app is creating. By publishing your schemas, you make it easier for other application authors to publish data in a format your app will recognize and handle.

     

       311
       -
       

     

       312
       -
       Let's create our schema in the `/lexicons` folder of our codebase. You can [read more about how to define schemas here](https://atproto.com/guides/lexicon).

     

       313
       -
       

     

       314
       -
       ```json

     

       315
       -
       /** lexicons/status.json **/

     

       316
       -
       {

     

       317
       -
         "lexicon": 1,

     

       318
       -
         "id": "com.example.status",

     

       319
       -
         "defs": {

     

       320
       -
           "main": {

     

       321
       -
             "type": "record",

     

       322
       -
             "key": "tid",

     

       323
       -
             "record": {

     

       324
       -
               "type": "object",

     

       325
       -
               "required": ["status", "createdAt"],

     

       326
       -
               "properties": {

     

       327
       -
                 "status": {

     

       328
       -
                   "type": "string",

     

       329
       -
                   "minLength": 1,

     

       330
       -
                   "maxGraphemes": 1,

     

       331
       -
                   "maxLength": 32

     

       332
       -
                 },

     

       333
       -
                 "createdAt": {

     

       334
       -
                   "type": "string",

     

       335
       -
                   "format": "datetime"

     

       336
       -
                 }

     

       337
       -
               }

     

       338
       -
             }

     

       339
       -
           }

     

       340
       -
         }

     

       341
       -
       }

     

       342
       -
       ```

     

       343
       -
       

     

       344
       -
       Now let's run some code-generation using our schema:

     

       345
       -
       

     

       346
       -
       ```bash

     

       347
       -
       ./node_modules/.bin/lex gen-server ./src/lexicon ./lexicons/*

     

       348
       -
       ```

     

       349
       -
       

     

       350
       -
       This will produce Typescript interfaces as well as runtime validation functions that we can use in our app. Here's what that generated code looks like:

     

       351
       -
       

     

       352
       -
       ```typescript

     

       353
       -
       /** src/lexicon/types/com/example/status.ts **/

     

       354
       -
       export interface Record {

     

       355
       -
         status: string

     

       356
       -
         createdAt: string

     

       357
       -
         [k: string]: unknown

     

       358
       -
       }

     

       359
       -
       

     

       360
       -
       export function isRecord(v: unknown): v is Record {

     

       361
       -
         return (

     

       362
       -
           isObj(v) &&

     

       363
       -
           hasProp(v, '$type') &&

     

       364
       -
           (v.$type === 'com.example.status#main' || v.$type === 'com.example.status')

     

       365
       -
         )

     

       366
       -
       }

     

       367
       -
       

     

       368
       -
       export function validateRecord(v: unknown): ValidationResult {

     

       369
       -
         return lexicons.validate('com.example.status#main', v)

     

       370
       -
       }

     

       371
       -
       ```

     

       372
       -
       

     

       373
       -
       Let's use that code to improve the `POST /status` route:

     

       374
       -
       

     

       375
       -
       ```typescript

     

       376
       -
       /** src/routes.ts **/

     

       377
       -
       import * as Status from '#/lexicon/types/com/example/status'

     

       378
       -
       // ...

     

       379
       -
       // "Set status" handler

     

       380
       -
       router.post(

     

       381
       -
         '/status',

     

       382
       -
         handler(async (req, res) => {

     

       383
       -
           // ...

     

       384
       -
       

     

       385
       -
           // Construct & validate their status record

     

       386
       -
           const record = {

     

       387
       -
             $type: 'com.example.status',

     

       388
       -
             status: req.body?.status,

     

       389
       -
             createdAt: new Date().toISOString(),

     

       390
       -
           }

     

       391
       -
           if (!Status.validateRecord(record).success) {

     

       392
       -
             return res.status(400).json({ error: 'Invalid status' })

     

       393
       -
           }

     

       394
       -
       

     

       395
       -
           // ...

     

       396
       -
         })

     

       397
       -
       )

     

       398
       -
       ```

     

       399
       -
       

     

       400
       -
       ## Step 6. Listening to the firehose

     

       401
       -
       

     

       402
       -
       So far, we have:

     

       403
       -
       

     

       404
       -
       - Logged in via OAuth

     

       405
       -
       - Created a custom schema

     

       406
       -
       - Read & written records for the logged in user

     

       407
       -
       

     

       408
       -
       Now we want to fetch the status records from other users.

     

       409
       -
       

     

       410
       -
       Remember how we referred to our app as being like a Google, crawling around the repos to get their records? One advantage we have in the AT Protocol is that each repo publishes an event log of their updates.

     

       411
       -
       

     

       412
       -
       ![A diagram of the event stream](./docs/diagram-event-stream.png)

     

       413
       -
       

     

       414
       -
       Using a [Relay service](https://docs.bsky.app/docs/advanced-guides/federation-architecture#relay) we can listen to an aggregated firehose of these events across all users in the network. In our case what we're looking for are valid `com.example.status` records.

     

       415
       -
       

     

       416
       -
       

     

       417
       -
       ```typescript

     

       418
       -
       /** src/firehose.ts **/

     

       419
       -
       import * as Status from '#/lexicon/types/com/example/status'

     

       420
       -
       // ...

     

       421
       -
       const firehose = new Firehose({})

     

       422
       -
       

     

       423
       -
       for await (const evt of firehose.run()) {

     

       424
       -
         // Watch for write events

     

       425
       -
         if (evt.event === 'create' || evt.event === 'update') {

     

       426
       -
           const record = evt.record

     

       427
       -
       

     

       428
       -
           // If the write is a valid status update

     

       429
       -
           if (

     

       430
       -
             evt.collection === 'com.example.status' &&

     

       431
       -
             Status.isRecord(record) &&

     

       432
       -
             Status.validateRecord(record).success

     

       433
       -
           ) {

     

       434
       -
             // Store the status

     

       435
       -
             // TODO

     

       436
       -
           }

     

       437
       -
         }

     

       438
       -
       }

     

       439
       -
       ```

     

       440
       -
       

     

       441
       -
       Let's create a SQLite table to store these statuses:

     

       442
       -
       

     

       443
       -
       ```typescript

     

       444
       -
       /** src/db.ts **/

     

       445
       -
       // Create our statuses table

     

       446
       -
       await db.schema

     

       447
       -
         .createTable('status')

     

       448
       -
         .addColumn('uri', 'varchar', (col) => col.primaryKey())

     

       449
       -
         .addColumn('authorDid', 'varchar', (col) => col.notNull())

     

       450
       -
         .addColumn('status', 'varchar', (col) => col.notNull())

     

       451
       -
         .addColumn('createdAt', 'varchar', (col) => col.notNull())

     

       452
       -
         .addColumn('indexedAt', 'varchar', (col) => col.notNull())

     

       453
       -
         .execute()

     

       454
       -
       ```

     

       455
       -
       

     

       456
       -
       Now we can write these statuses into our database as they arrive from the firehose:

     

       457
       -
       

     

       458
       -
       ```typescript

     

       459
       -
       /** src/firehose.ts **/

     

       460
       -
       // If the write is a valid status update

     

       461
       -
       if (

     

       462
       -
         evt.collection === 'com.example.status' &&

     

       463
       -
         Status.isRecord(record) &&

     

       464
       -
         Status.validateRecord(record).success

     

       465
       -
       ) {

     

       466
       -
         // Store the status in our SQLite

     

       467
       -
         await db

     

       468
       -
           .insertInto('status')

     

       469
       -
           .values({

     

       470
       -
             uri: evt.uri.toString(),

     

       471
       -
             authorDid: evt.author,

     

       472
       -
             status: record.status,

     

       473
       -
             createdAt: record.createdAt,

     

       474
       -
             indexedAt: new Date().toISOString(),

     

       475
       -
           })

     

       476
       -
           .onConflict((oc) =>

     

       477
       -
             oc.column('uri').doUpdateSet({

     

       478
       -
               status: record.status,

     

       479
       -
               indexedAt: new Date().toISOString(),

     

       480
       -
             })

     

       481
       -
           )

     

       482
       -
           .execute()

     

       483
       -
       }

     

       484
       -
       ```

     

       485
       -
       

     

       486
       -
       You can almost think of information flowing in a loop:

     

       487
       -
       

     

       488
       -
       ![A diagram of the flow of information](./docs/diagram-info-flow.png)

     

       489
       -
       

     

       490
       -
       Applications write to the repo. The write events are then emitted on the firehose where they're caught by the apps and ingested into their databases.

     

       491
       -
       

     

       492
       -
       Why sync from the event log like this? Because there are other apps in the network that will write the records we're interested in. By subscribing to the event log, we ensure that we catch all the data we're interested in &mdash; including data published by other apps!

     

       493
       -
       

     

       494
       -
       ## Step 7. Listing the latest statuses

     

       495
       -
       

     

       496
       -
       Now that we have statuses populating our SQLite, we can produce a timeline of status updates by users. We also use a [DID](https://atproto.com/specs/did)-to-handle resolver so we can show a nice username with the statuses:

     

       497
       -
       

     

       498
       -
       ```typescript

     

       499
       -
       /** src/routes.ts **/

     

       500
       -
       // Homepage

     

       501
       -
       router.get(

     

       502
       -
         '/',

     

       503
       -
         handler(async (req, res) => {

     

       504
       -
           // ...

     

       505
       -
       

     

       506
       -
           // Fetch data stored in our SQLite

     

       507
       -
           const statuses = await db

     

       508
       -
             .selectFrom('status')

     

       509
       -
             .selectAll()

     

       510
       -
             .orderBy('indexedAt', 'desc')

     

       511
       -
             .limit(10)

     

       512
       -
             .execute()

     

       513
       -
       

     

       514
       -
           // Map user DIDs to their domain-name handles

     

       515
       -
           const didHandleMap = await resolver.resolveDidsToHandles(

     

       516
       -
             statuses.map((s) => s.authorDid)

     

       517
       -
           )

     

       518
       -
       

     

       519
       -
           // ...

     

       520
       -
         })

     

       521
       -
       )

     

       522
       -
       ```

     

       523
       -
       

     

       524
       -
       Our HTML can now list these status records:

     

       525
       -
       

     

       526
       -
       ```html

     

       527
       -
       <!-- src/pages/home.ts -->

     

       528
       -
       ${statuses.map((status, i) => {

     

       529
       -
         const handle = didHandleMap[status.authorDid] || status.authorDid

     

       530
       -
         return html`

     

       531
       -
           <div class="status-line">

     

       532
       -
             <div>

     

       533
       -
               <div class="status">${status.status}</div>

     

       534
       -
             </div>

     

       535
       -
             <div class="desc">

     

       536
       -
               <a class="author" href="https://bsky.app/profile/${handle}">@${handle}</a>

     

       537
       -
               was feeling ${status.status} on ${status.indexedAt}.

     

       538
       -
             </div>

     

       539
       -
           </div>

     

       540
       -
         `

     

       541
       -
       })}

     

       542
       -
       ```

     

       543
       -
       

     

       544
       -
       ![A screenshot of the app status timeline](./docs/app-status-history.png)

     

       545
       -
       

     

       546
       -
       ## Step 8. Optimistic updates

     

       547
       -
       

     

       548
       -
       As a final optimization, let's introduce "optimistic updates."

     

       549
       -
       

     

       550
       -
       Remember the information flow loop with the repo write and the event log?

     

       551
       -
       

     

       552
       -
       ![A diagram of the flow of information](./docs/diagram-info-flow.png)

     

       553
       -
       

     

       554
       -
       Since we're updating our users' repos locally, we can short-circuit that flow to our own database:

     

       555
       -
       

     

       556
       -
       ![A diagram illustrating optimistic updates](./docs/diagram-optimistic-update.png)

     

       557
       -
       

     

       558
       -
       This is an important optimization to make, because it ensures that the user sees their own changes while using your app. When the event eventually arrives from the firehose, we just discard it since we already have it saved locally.

     

       559
       -
       

     

       560
       -
       To do this, we just update `POST /status` to include an additional write to our SQLite DB:

     

       561
       -
       

     

       562
       -
       ```typescript

     

       563
       -
       /** src/routes.ts **/

     

       564
       -
       // "Set status" handler

     

       565
       -
       router.post(

     

       566
       -
         '/status',

     

       567
       -
         handler(async (req, res) => {

     

       568
       -
           // ...

     

       569
       -
       

     

       570
       -
           let uri

     

       571
       -
           try {

     

       572
       -
             // Write the status record to the user's repository

     

       573
       -
             const res = await agent.putRecord({

     

       574
       -
               repo: agent.accountDid,

     

       575
       -
               collection: 'com.example.status',

     

       576
       -
               rkey: TID.nextStr(),

     

       577
       -
               record,

     

       578
       -
             })

     

       579
       -
             uri = res.uri

     

       580
       -
           } catch (err) {

     

       581
       -
             logger.warn({ err }, 'failed to write record')

     

       582
       -
             return res.status(500).json({ error: 'Failed to write record' })

     

       583
       -
           }

     

       584
       -
       

     

       585
       -
           try {

     

       586
       -
             // Optimistically update our SQLite <-- HERE!

     

       587
       -
             await db

     

       588
       -
               .insertInto('status')

     

       589
       -
               .values({

     

       590
       -
                 uri,

     

       591
       -
                 authorDid: agent.accountDid,

     

       592
       -
                 status: record.status,

     

       593
       -
                 createdAt: record.createdAt,

     

       594
       -
                 indexedAt: new Date().toISOString(),

     

       595
       -
               })

     

       596
       -
               .execute()

     

       597
       -
           } catch (err) {

     

       598
       -
             logger.warn(

     

       599
       -
               { err },

     

       600
       -
               'failed to update computed view; ignoring as it should be caught by the firehose'

     

       601
       -
             )

     

       602
       -
           }

     

       603
       -
       

     

       604
       -
           res.status(200).json({})

     

       605
       -
         })

     

       606
       -
       )

     

       607
       -
       ```

     

       608
       -
       

     

       609
       -
       You'll notice this code looks almost exactly like what we're doing in `firehose.ts`.

     

       610
       -
       

     

       611
       -
       ## Thinking in AT Proto

     

       612
       -
       

     

       613
       -
       In this tutorial we've covered the key steps to building an atproto app. Data is published in its canonical form on users' `at://` repos and then aggregated into apps' databases to produce views of the network.

     

       614
       -
       

     

       615
       -
       When building your app, think in these four key steps:

     

       616
       -
       

     

       617
       -
       - Design the [Lexicon](#) schemas for the records you'll publish into the Atmosphere.

     

       618
       -
       - Create a database for aggregating the records into useful views.

     

       619
       -
       - Build your application to write the records on your users' repos.

     

       620
       -
       - Listen to the firehose to aggregate data across the network.

     

       621
       -
       

     

       622
       -
       Remember this flow of information throughout:

     

       623
       -
       

     

       624
       -
       ![A diagram of the flow of information](./docs/diagram-info-flow.png)

     

       625
       -
       

     

       626
       -
       This is how every app in the Atmosphere works, including the [Bluesky social app](https://bsky.app).

     

       627
       -
       

     

       628
       -
       ## Next steps

     

       629
       -
       

     

       630
       -
       If you want to practice what you've learned, here are some additional challenges you could try:

     

       631
       -
       

     

       632
       -
       - Sync the profile records of all users so that you can show their display names instead of their handles.

     

       633
       -
       - Count the number of each status used and display the total counts.

     

       634
       -
       - Fetch the authed user's `app.bsky.graph.follow` follows and show statuses from them.

     

       635
       -
       - Create a different kind of schema, like a way to post links to websites and rate them 1 through 4 stars.

     

       636
       -
       

     

       637
       -
       You can find more information here:

     

       638
       -
       

     

       639
       -
       |Resources|-|

     

       640
       -
       |-|-|

     

       641
       -
       |[＠ ATProto docs](https://atproto.com)|Learn more about the AT Protocol.|

     

       642
       -
       |[🦋 Bluesky API docs](https://docs.bsky.app/)|See how Bluesky works as an ATProto app.|

     

       643
       -
       |[📦 ATProto monorepo](https://github.com/bluesky-social/atproto)|See the source code first-hand.|

     

       644
       -
       |[💬 ATProto discussions board](https://github.com/bluesky-social/atproto/discussions)|Ask any questions you have!|

docs/app-banner.png

This is a binary file and will not be displayed.

docs/app-login.png

This is a binary file and will not be displayed.

docs/app-screenshot.png

This is a binary file and will not be displayed.

docs/app-status-history.png

This is a binary file and will not be displayed.

docs/app-status-options.png

This is a binary file and will not be displayed.

docs/diagram-event-stream.png

This is a binary file and will not be displayed.

docs/diagram-info-flow.png

This is a binary file and will not be displayed.

docs/diagram-oauth.png

This is a binary file and will not be displayed.

docs/diagram-optimistic-update.png

This is a binary file and will not be displayed.

docs/diagram-repo.png

This is a binary file and will not be displayed.