a cache for slack profile pictures and emojis
dunkirk.sh
1<h1 align="center">
2 <img src="https://raw.githubusercontent.com/taciturnaxolotl/cachet/master/.github/images/cachet.webp" width="200" alt="Logo"/><br/>
3 <img src="https://raw.githubusercontent.com/taciturnaxolotl/carriage/master/.github/images/transparent.png" height="45" width="0px"/>
4 Cachet
5 <img src="https://raw.githubusercontent.com/taciturnaxolotl/carriage/master/.github/images/transparent.png" height="30" width="0px"/>
6</h1>
7
8<p align="center">
9 <i><b>noun</b> - A mark or quality, as of distinction, individuality, or authenticity.</i>
10</p>
11
12<p align="center">
13 <img src="https://raw.githubusercontent.com/taciturnaxolotl/carriage/master/.github/images/line-break-thin.svg" />
14</p>
15
16<p align="center">
17 <img src="https://raw.githubusercontent.com/taciturnaxolotl/cachet/master/.github/images/out.gif" />
18</p>
19
20<p align="center">
21 <img src="https://raw.githubusercontent.com/taciturnaxolotl/carriage/master/.github/images/line-break-thin.svg" />
22</p>
23
24## What's this?
25
26Cachet is a cache / proxy for profile pictures and emojis on the hackclub slack! I made it because calling the slack api every time you want a profile image or emoji is expensive and annoying. Now you can just call the cachet api and get a link to the image or emoji you want! Best of all we are just linking to slack's cdn so it doesn't cost me much of anything (besides db space) to run!
27
28## How do I use it?
29
30Well first the question is how do I host it lol.
31
32### Hosting
33
34I'm hosting on nest so I just setup a systemd service file that runs `bun run index.ts` in the root dir of this repo. Then I setup caddy to reverse proxy `cachet.dunkirk.sh` to the app.
35
36Your `.env` file should look like this:
37
38```bash
39SLACK_TOKEN=xoxb-123456789012-123456789012-123456789012-123456789012
40SLACK_SIGNING_SECRET=12345678901234567890123456789012
41NODE_ENV=production
42SENTRY_DSN="https://xxxxx@xxxx.ingest.us.sentry.io/123456" # Optional
43DATABASE_PATH=/path/to/db.sqlite # Optional
44PORT=3000 # Optional
45```
46
47The slack app can be created from the [`manifest.yaml`](./manifest.yaml) in this repo. It just needs the `emoji:read` and `users:read` scopes.
48
49I included a service file in this repo that you can use to run the app. Just copy it to `~/.config/systemd/` and then run `systemctl --user enable cachet` and `systemctl --user start cachet` to start the app.
50
51```bash
52cp cachet.service ~/.config/systemd/user/
53mkdir data
54systemctl --user enable cachet
55systemctl --user start cachet
56```
57
58Now grab a free port from nest (`nest get_port`) and then link your domain to your nest user (`nest caddy add cachet.dunkirk.sh`) (don't for get to make a CNAME on the domain pointing to `kierank.hackclub.app`) and then after editing in a `Caddyfile` entry like the following you should be good to go! (Don't forget to restart caddy: `systemctl restart --user caddy`)
59
60```caddy
61http://cachet.dunkirk.sh {
62 bind unix/.cachet.dunkirk.sh.webserver.sock|777
63 reverse_proxy :38453
64}
65```
66
67### Usage
68
69The api is pretty simple. You can get a profile picture by calling `GET /profile/:id` where `:id` is the slack user id. You can get an emoji by calling `GET /emoji/:name` where `:name` is the name of the emoji. You can also get a list of all emojis by calling `GET /emojis`.
70
71Additionally, you can manually purge a specific user's cache with `POST /users/:user/purge` (requires authentication with a bearer token).
72
73There are also complete swagger docs available at [`/swagger`](https://cachet.dunkirk.sh/swagger)! They are dynamically generated from the code so they should always be up to date! (The types force me to keep them up to date ^_^)
74
75
76
77## How does it work?
78
79The app is honestly super simple. It's pretty much just a cache layer on top of the slack api. When you request a profile picture or emoji it first checks the cache. If the image is in the cache it returns the link to the image. If the image is not in the cache it calls the slack api to get the link to image and then stores that in the cache before returning the image link to you!
80
81There were a few interesting hurdles that made this a bit more confusing though. The first was that slack returns the `emoji.list` endpoint with not just regular emojis but also aliased emojis. The aliased emojis doesn't seem that hard at first untill you realize that someone could alias stock slack emojis. That means that we don't have a url to the image and to make it worse slack doesn't have an offically documented way to get the full list of stock emojis. Thankfully an amazing user ([@impressiver](https://github.com/impressiver)) put this all into a handy [gist](https://gist.github.com/impressiver/87b5b9682d935efba8936898fbfe1919) for everyone to use! It was last updated on 2020-12-22 so it's a bit out of date but slack doesn't seem to be changing their emojis too often so it should be fine for now.
82
83```json
84{
85 "ok": true,
86 "emoji": {
87 "hackhaj": "https://emoji.slack-edge.com/T0266FRGM/hackshark/0bf4771247471a48.png",
88 "hackhaj": "alias:hackshark"
89 "face-grinning": "alias:grinning"
90 }
91}
92
93{
94 "grinning": "https://a.slack-edge.com/production-standard-emoji-assets/14.0/google-medium/1f601.png"
95}
96```
97
98The second challenge (technically its not a challenge; more of a side project) was building a custom cache solution based on `Bun:sqlite`. It ended up being far easier than I thought it was going to be and I'm quite happy with how it turned out! It's fully typed which makes it awesome to use and blazing fast due to the native Bun implementation of sqlite. Using it is also dead simple. Just create a new instance of the cache with a db path, a ttl, and a fetch function for the emojis and you're good to go! Inserting and getting data is also super simple and the cache is fully typed!
99
100```typescript
101const cache = new SlackCache(
102 process.env.DATABASE_PATH ?? "./data/cachet.db",
103 24,
104 async () => {
105 console.log("Fetching emojis from Slack");
106 },
107);
108
109await cache.insertUser("U062UG485EE", "https://avatars.slack-edge.com/2024-11-30/8105375749571_53898493372773a01a1f_original.jpg", null);
110await cache.insertEmoji("hackshark", "https://emoji.slack-edge.com/T0266FRGM/hackshark/0bf4771247471a48.png");
111
112const emoji = await cache.getEmoji("hackshark");
113const user = await cache.getUser("U062UG485EE");
114
115// You can also purge the cache for a specific user
116const purgeResult = await cache.purgeUserCache("U062UG485EE");
117console.log(`Cache purged: ${purgeResult}`); // true if user was in cache and purged
118```
119
120The final bit was at this point a bit of a ridiculous one. I didn't like how heavyweight the `bolt` or `slack-edge` packages were so I rolled my own slack api wrapper. It's again fully typed and designed to be as lightweight as possible.
121
122```typescript
123const slack = new Slack(process.env.SLACK_TOKEN, process.env.SLACK_SIGNING_SECRET);
124
125const user = await slack.getUser("U062UG485EE");
126const emojis = await slack.getEmoji();
127
128// Manually purge a specific user's cache using the API endpoint
129const response = await fetch("https://cachet.example.com/users/U062UG485EE/purge", {
130 method: "POST",
131 headers: {
132 "Authorization": `Bearer ${process.env.BEARER_TOKEN}`
133 }
134});
135const result = await response.json();
136// { message: "User cache purged", userId: "U062UG485EE", success: true }
137```
138
139## Development
140
141### Migrations
142
143The app includes a migration system to handle database schema and data changes between versions. Migrations are automatically run when the app starts.
144
145Previous versions are tracked in a `migrations` table in the database, which records each applied migration with its version number and timestamp.
146
147To create a new migration:
148
149```typescript
150// src/migrations/myNewMigration.ts
151import { Database } from "bun:sqlite";
152import { Migration } from "./types";
153
154export const myNewMigration: Migration = {
155 version: "0.3.2", // Should match package.json version
156 description: "What this migration does",
157
158 async up(db: Database): Promise<void> {
159 // Migration code here
160 db.run(`ALTER TABLE my_table ADD COLUMN new_column TEXT`);
161 }
162};
163
164// Then add to src/migrations/index.ts
165import { myNewMigration } from "./myNewMigration";
166
167export const migrations: Migration[] = [
168 endpointGroupingMigration,
169 myNewMigration,
170 // Add new migrations here
171];
172```
173
174Remember to update the version in `package.json` when adding new migrations.
175
176<p align="center">
177 <img src="https://raw.githubusercontent.com/taciturnaxolotl/carriage/master/.github/images/line-break.svg" />
178</p>
179
180<p align="center">
181 © 2024-present <a href="https://github.com/taciturnaxolotl">Kieran Klukas</a>
182</p>
183
184<p align="center">
185 <a href="https://github.com/taciturnaxolotl/carriage/blob/master/LICENSE.md"><img src="https://img.shields.io/static/v1.svg?style=for-the-badge&label=License&message=AGPL 3.0&logoColor=d9e0ee&colorA=363a4f&colorB=b7bdf8"/></a>
186</p>