commit e4ebeab649bd083a9d11acdfad2a5f1f2d78365a · willdot.net/at-shorter-url

+5 -5

README.md

···

3
3

### What is it?

4
4

It's a URL shorten service that uses ATProto so that the users creating the short links will always own the links they are creating and gives them the option to host the service themselves so that they can be sure that the links will always be available.

5
5

6
6
-
Try it out [https://short.willdot.xyz]

7
7
-

8
6

### Inspiration

9
7

This project was inspired by a conversation with someone about the classic engineering interview question about creating a URL shortning service. One of the downfalls of this is that you're relying on the service to always be there so that your links don't die. I think Google announced that they were shutting down their service which caused uproar because it meant that existing links embeded in websites and other places would cease to work. Thankfully they announced that they would allow existing links to work, just not allow new ones to be created. But that scare of links no longer working, made me think about how the ATProto architecture is perfect for allowing users to own their links and be sure that they will always be available without depending on another service.

10
8

11
9

### How it works

12
10

13
13
-
When a user logs in with their ATProto account, they will be able to create a new short URL. This action creates a record in their PDS containing the URL to redirect to and in the future could contain other peices of metadata. The key bit here is that the record in their PDS is created with an RKey, which in this application is a TID ((timestamp identifier)[https://atproto.com/specs/tid]) which is unique and that key becomes the "short URL" ID. Combine that with the host that created the ID and you get a short URL. The users DID, key and URL are then stored in a local database to the service.

11
11
+
When a user logs in with their ATProto account, they will be able to create a new short URL. This action creates a record in their PDS containing the URL to redirect to and in the future could contain other peices of metadata. The key bit here is that the record in their PDS is created with an RKey, which in this application is a TID ([timestamp identifier](https://atproto.com/specs/tid)) which is unique and that key becomes the "short URL" ID. Combine that with the host that created the ID and you get a short URL. The users DID, key and URL are then stored in a local database to the service.

14
12

15
13

As long as the service is running at that host, the short URL will be available to be redirected. When the short URL is hit, the service will take the key part of the URL, look it up in the database to find the real URL and then redirect to that URL.

16
14

···

20
18

21
19

### Self hosting

22
20

23
23
-
Another neat part of this architecture is that if a user wants to self host this, they can and the links they create will only work for their service. For example.

21
21
+
Another neat part of this architecture is that it allows the user to self host the service. Traditional short URL services turn out to be a lot of hassle. Read this really interesting [thread on Bluesky](https://bsky.app/profile/gbl08ma.com/post/3m2pft7a3dc23) about it from the creator of [tny.im](tny.im). Turns out hosting one for others to use isn't a great move these days, so by having one that you can host yourself and only you can use, could be a really handy tool. You create the short URLs and pass them out to people to use.

22
22
+

23
23
+
The way the service works, is that whe you start it up, you configure it to be for your DID (ATProto identifier) so that only you can log into the service to create short URLs. Then when the Jetstream consumer runs, it only listens to events for your DID which means you only get your short URL records in the local database.

24
24

25
25
-
If I create a short URL `https://my-short-service/a/abcd` on my service and then someone creates `https://a-different-short-service/a/1234` those links will only work with the domain that created them. I wouldn't be able to take the key `abcd` and use it with the `https://a-different-short-service` domain because that isn't the domain that created the link. This means that even though every service is consuming all of the short URL ATProto records, there will never be a case where someone could hijack an ID to redirect to somewhere else without the user that created and owning the link from doing so (because updating a record in a PDS requires the users auth).

25
25
+
Remember!!! All of the links you create are stored in your PDS which is public. So anyone that wants to see what links you've created and go to them, will be able to do that.

+5

auth_handlers.go

···

       25
       25
        
       			return

     

       26
       26
        
       		}

     

       27
       27
        
       

     

       28
       28
       +
       		if did.String() != s.usersDID {

     

       29
       29
       +
       			http.Error(w, "not authorized", http.StatusUnauthorized)

     

       30
       30
       +
       			return

     

       31
       31
       +
       		}

     

       32
       32
       +
       

     

       28
       33
        
       		next(w, r)

     

       29
       34
        
       	}

     

       30
       35
        
       }

+10 -4

cmd/atshorter/main.go

···

       51
       51
        
       		return

     

       52
       52
        
       	}

     

       53
       53
        
       

     

       54
       54
       +
       	usersDID := os.Getenv("DID")

     

       55
       55
       +
       	if usersDID == "" {

     

       56
       56
       +
       		slog.Error("DID env not set")

     

       57
       57
       +
       		return

     

       58
       58
       +
       	}

     

       59
       59
       +
       

     

       54
       60
        
       	dbFilename := path.Join(dbMountPath, "database.db")

     

       55
       61
        
       	db, err := database.New(dbFilename)

     

       56
       62
        
       	if err != nil {

     
···

       94
       100
        
       		},

     

       95
       101
        
       	}

     

       96
       102
        
       

     

       97
       97
       -
       	server, err := atshorter.NewServer(host, port, db, oauthClient, httpClient)

     

       103
       103
       +
       	server, err := atshorter.NewServer(host, port, db, oauthClient, httpClient, usersDID)

     

       98
       104
        
       	if err != nil {

     

       99
       105
        
       		slog.Error("create new server", "error", err)

     

       100
       106
        
       		return

     
···

       112
       118
        
       		_ = server.Stop(context.Background())

     

       113
       119
        
       	}()

     

       114
       120
        
       

     

       115
       115
       -
       	go consumeLoop(ctx, db)

     

       121
       121
       +
       	go consumeLoop(ctx, db, usersDID)

     

       116
       122
        
       

     

       117
       123
        
       	server.Run()

     

       118
       124
        
       }

     

       119
       125
        
       

     

       120
       120
       -
       func consumeLoop(ctx context.Context, db *database.DB) {

     

       126
       126
       +
       func consumeLoop(ctx context.Context, db *database.DB, did string) {

     

       121
       127
        
       	jsServerAddr := os.Getenv("JS_SERVER_ADDR")

     

       122
       128
        
       	if jsServerAddr == "" {

     

       123
       129
        
       		jsServerAddr = defaultServerAddr

     

       124
       130
        
       	}

     

       125
       131
        
       

     

       126
       126
       -
       	consumer := atshorter.NewConsumer(jsServerAddr, slog.Default(), db)

     

       132
       132
       +
       	consumer := atshorter.NewConsumer(jsServerAddr, slog.Default(), db, did)

     

       127
       133
        
       

     

       128
       134
        
       	err := retry.Do(func() error {

     

       129
       135
        
       		err := consumer.Consume(ctx)

+2 -2

consumer.go

···

       19
       19
        
       	logger  *slog.Logger

     

       20
       20
        
       }

     

       21
       21
        
       

     

       22
       22
       -
       func NewConsumer(jsAddr string, logger *slog.Logger, store HandlerStore) *consumer {

     

       22
       22
       +
       func NewConsumer(jsAddr string, logger *slog.Logger, store HandlerStore, did string) *consumer {

     

       23
       23
        
       	cfg := client.DefaultClientConfig()

     

       24
       24
        
       	if jsAddr != "" {

     

       25
       25
        
       		cfg.WebsocketURL = jsAddr

     
···

       27
       27
        
       	cfg.WantedCollections = []string{

     

       28
       28
        
       		"com.atshorter.shorturl",

     

       29
       29
        
       	}

     

       30
       30
       -
       	cfg.WantedDids = []string{} // TODO: possibly when self hosting, limit this to just a select few?

     

       30
       30
       +
       	cfg.WantedDids = []string{did}

     

       31
       31
        
       

     

       32
       32
        
       	return &consumer{

     

       33
       33
        
       		cfg:    cfg,

+1

example.env

···

       4
       4
        
       DATABASE_PATH="./"

     

       5
       5
        
       JS_SERVER_ADDR="set to a different Jetstream instance"

     

       6
       6
        
       PORT="3002"

     

       7
       7
       +
       DID="Enter your account DID here"

+3 -1

server.go

···

       29
       29
        
       

     

       30
       30
        
       type Server struct {

     

       31
       31
        
       	host         string

     

       32
       32
       +
       	usersDID     string

     

       32
       33
        
       	httpserver   *http.Server

     

       33
       34
        
       	sessionStore *sessions.CookieStore

     

       34
       35
        
       	templates    []*template.Template

     
···

       44
       45
        
       //go:embed html

     

       45
       46
        
       var htmlFolder embed.FS

     

       46
       47
        
       

     

       47
       47
       -
       func NewServer(host string, port string, store Store, oauthClient *oauth.ClientApp, httpClient *http.Client) (*Server, error) {

     

       48
       48
       +
       func NewServer(host string, port string, store Store, oauthClient *oauth.ClientApp, httpClient *http.Client, usersDID string) (*Server, error) {

     

       48
       49
        
       	sessionStore := sessions.NewCookieStore([]byte(os.Getenv("SESSION_KEY")))

     

       49
       50
        
       

     

       50
       51
        
       	homeTemplate, err := template.ParseFS(htmlFolder, "html/home.html")

     
···

       64
       65
        
       

     

       65
       66
        
       	srv := &Server{

     

       66
       67
        
       		host:         host,

     

       68
       68
       +
       		usersDID:     usersDID,

     

       67
       69
        
       		oauthClient:  oauthClient,

     

       68
       70
        
       		sessionStore: sessionStore,

     

       69
       71
        
       		templates:    templates,