commits · ptr.pet/nsid-tracker · tangled

ptr.pet / nsid-tracker

tracks lexicons and how many times they appeared on the jetstream

commits

Author

Commit

Message

Date

ptr.pet

b63273b4

refactor(server): increase block size even more, reenable compression

4mo ago

ptr.pet

bf2179a4

refactor(server): dont use any compression for trees

4mo ago

ptr.pet

a58727c0

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

4mo ago

ptr.pet

a10342d6

feat(server): more threads for fjall workers when migrating

4mo ago

ptr.pet

e7a40253

feat(server): re-enable jemalloc for non-msvc targets

4mo ago

ptr.pet

3a678143

refactor(server): use more workers on migrate db

4mo ago

ptr.pet

f864854c

feat(server): use unlimited journaling and write buffer size on compact cmd as well

4mo ago

ptr.pet

55d6b407

refactor(server): maximum journaling size while migrating, no manual persist

4mo ago

ptr.pet

84b0f5ef

fix(server): use deser to access item data because zero-copy is not guaranteed

4mo ago

ptr.pet

48460bf4

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

4mo ago

ptr.pet

11924417

feat(server): make db configurable, improve migration perf

4mo ago

ptr.pet

4c4a5aa3

feat(server): actually make the block inserts start in parallel

4mo ago

ptr.pet

fddb212c

add more logging for syncing and migrating

4mo ago

ptr.pet

aebca039

chore: add fjall data stuff to gitignore

4mo ago

ptr.pet

d0e611bc

fix(server): encode_block takes correct amount of items, improve rate limiter usage

4mo ago

ptr.pet

7503870a

fix(server): sync remainder if all is passed to sync

4mo ago

ptr.pet

29914c94

feat(server): sort on major compact

4mo ago

ptr.pet

bf7842d0

feat(server): implement compaction :3

4mo ago

ptr.pet

2a7d6005

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

4mo ago

ptr.pet

e2dd9f2b

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

4mo ago

ptr.pet

86f3fa23

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

4mo ago

ptr.pet

52fd5331

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

4mo ago

ptr.pet

a820ae13

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

4mo ago

ptr.pet

925dbc95

refactor(server): use either instead of boxed iter

4mo ago

ptr.pet

7cd979b3

refactor(server): improve record events by splitting it into ingest_events

4mo ago

ptr.pet

13e1dd85

build(nix): add cmake

4mo ago

ptr.pet

7920317e

feat(server): track more bench info in compact

4mo ago

ptr.pet

7d08ae63

fix(server): remove broken imports

4mo ago

ptr.pet

1dd2ef8c

refactor(server): use quanta for last activity on lexicon handles aswell

4mo ago

ptr.pet

6b78fe2d

refactor(server): implement own rate counter that should be faster, use quanta on it

4mo ago

ptr.pet

32529fe8

refactor(server): use relaxed ordering for keeping track of lexicon buf len / last activity / block size since we don't really need any strict ordering

4mo ago

ptr.pet

3456ca30

feat(server): use snmalloc

4mo ago

ptr.pet

5e769184

refactor(server): run sync jobs in threadpool

4mo ago

ptr.pet

84d25308

refactor(server): delete old db impl, rename migrate to compact

4mo ago

ptr.pet

48892f4c

feat(server): run all trees migrations in parallel

4mo ago

ptr.pet

b7a5f07a

feat(server): allow setting a max block size

4mo ago

ptr.pet

4e1f68a4

fix(server): actually sync on migrate lol

4mo ago

ptr.pet

ebeab2a7

fix(server): tracking_since funcs, also fix old db get_hits

4mo ago

ptr.pet

e28fd9c4

feat(server): error on unknwn cmd

4mo ago

ptr.pet

d7265904

feat(server): implement delta-of-deltas encoding and varint and chunking

4mo ago

ptr.pet

f73de8d3

fix(client): make the stats card numbers clip less

4mo ago

ptr.pet

d275783d

feat(client): show tracking since

4mo ago

ptr.pet

999365ad

feat(server): implement fetching tracking since

4mo ago

ptr.pet

383ee9e6

fix(client): fix refresh rate resetting from 2 to realtime this time

4mo ago

ptr.pet

ffd32aa0

feat(client): put bsky toggle right after filter

4mo ago

ptr.pet

90da2f14

build(nix): update modules hash

4mo ago

ptr.pet

310ffddf

docs: update readme

4mo ago

ptr.pet

90220382

fix(client): reset refresh rate to real-time only if user hasnt changed refresh rate

4mo ago

ptr.pet

e020b36f

feat(server): implement fetching hits

4mo ago

ptr.pet

e8dfb4a1

feat(client): implement sorting and setting refresh rate

4mo ago

ptr.pet

b4c008ad

feat(client): improve header, change ws status badge to indicate what its doing better

4mo ago

ptr.pet

f065b614

feat(client): make stat cards numbers animated, rename text

4mo ago

ptr.pet

a13ddcde

feat(client): improve styling a lot to work better on mobile and different screen sizes

4mo ago

ptr.pet

c806b451

refactor(server): use existing Events type, create LatencyMillis for better latency formatting

4mo ago

ptr.pet

9c8c6ece

feat(client): make stat cards more viewable on mobile

4mo ago

ptr.pet

90e4de63

build(nix): add hash for axum-tws

4mo ago

ptr.pet

c2a22b18

feat(server): increase stream send rate

4mo ago

ptr.pet

28ac2ab1

feat(server): fix request tracing

4mo ago

ptr.pet

0c23c8d7

feat: track and send events per second, send 10 WS messages maximum per second, buffering in between

4mo ago

ptr.pet

0e76b961

refactor(server): drop atproto-jetstream, use tokio-websockets directly, improve ctrl_c and jetstream connection retries

4mo ago

refactor(server): increase block size even more, reenable compression

b63273b4

ptr.pet

4mo

refactor(server): dont use any compression for trees

bf2179a4

ptr.pet

4mo

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

a58727c0

ptr.pet

4mo

feat(server): more threads for fjall workers when migrating

a10342d6

ptr.pet

4mo

feat(server): re-enable jemalloc for non-msvc targets

e7a40253

ptr.pet

4mo

refactor(server): use more workers on migrate db

3a678143

ptr.pet

4mo

feat(server): use unlimited journaling and write buffer size on compact cmd as well

f864854c

ptr.pet

4mo

refactor(server): maximum journaling size while migrating, no manual persist

55d6b407

ptr.pet

4mo

fix(server): use deser to access item data because zero-copy is not guaranteed

84b0f5ef

ptr.pet

4mo

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

48460bf4

ptr.pet

4mo

feat(server): make db configurable, improve migration perf

11924417

ptr.pet

4mo

feat(server): actually make the block inserts start in parallel

4c4a5aa3

ptr.pet

4mo

add more logging for syncing and migrating

fddb212c

ptr.pet

4mo

chore: add fjall data stuff to gitignore

aebca039

ptr.pet

4mo

fix(server): encode_block takes correct amount of items, improve rate limiter usage

d0e611bc

ptr.pet

4mo

fix(server): sync remainder if all is passed to sync

7503870a

ptr.pet

4mo

feat(server): sort on major compact

29914c94

ptr.pet

4mo

feat(server): implement compaction :3

bf7842d0

ptr.pet

4mo

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

2a7d6005

ptr.pet

4mo

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

e2dd9f2b

ptr.pet

4mo

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

86f3fa23

ptr.pet

4mo

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

52fd5331

ptr.pet

4mo

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

a820ae13

ptr.pet

4mo

refactor(server): use either instead of boxed iter

925dbc95

ptr.pet

4mo

refactor(server): improve record events by splitting it into ingest_events

7cd979b3

ptr.pet

4mo

build(nix): add cmake

13e1dd85

ptr.pet

4mo

feat(server): track more bench info in compact

7920317e

ptr.pet

4mo

fix(server): remove broken imports

7d08ae63

ptr.pet

4mo

refactor(server): use quanta for last activity on lexicon handles aswell

1dd2ef8c

ptr.pet

4mo

refactor(server): implement own rate counter that should be faster, use quanta on it

6b78fe2d

ptr.pet

4mo

refactor(server): use relaxed ordering for keeping track of lexicon buf len / last activity / block size since we don't really need any strict ordering

32529fe8

ptr.pet

4mo

feat(server): use snmalloc

3456ca30

ptr.pet

4mo

refactor(server): run sync jobs in threadpool

5e769184

ptr.pet

4mo

refactor(server): delete old db impl, rename migrate to compact

84d25308

ptr.pet

4mo

feat(server): run all trees migrations in parallel

48892f4c

ptr.pet

4mo

feat(server): allow setting a max block size

b7a5f07a

ptr.pet

4mo

fix(server): actually sync on migrate lol

4e1f68a4

ptr.pet

4mo

fix(server): tracking_since funcs, also fix old db get_hits

ebeab2a7

ptr.pet

4mo

feat(server): error on unknwn cmd

e28fd9c4

ptr.pet

4mo

feat(server): implement delta-of-deltas encoding and varint and chunking

d7265904

ptr.pet

4mo

fix(client): make the stats card numbers clip less

f73de8d3

ptr.pet

4mo

feat(client): show tracking since

d275783d

ptr.pet

4mo

feat(server): implement fetching tracking since

999365ad

ptr.pet

4mo

fix(client): fix refresh rate resetting from 2 to realtime this time

383ee9e6

ptr.pet

4mo

feat(client): put bsky toggle right after filter

ffd32aa0

ptr.pet

4mo

build(nix): update modules hash

90da2f14

ptr.pet

4mo

docs: update readme

310ffddf

ptr.pet

4mo

fix(client): reset refresh rate to real-time only if user hasnt changed refresh rate

90220382

ptr.pet

4mo

feat(server): implement fetching hits

e020b36f

ptr.pet

4mo

feat(client): implement sorting and setting refresh rate

e8dfb4a1

ptr.pet

4mo

feat(client): improve header, change ws status badge to indicate what its doing better

b4c008ad

ptr.pet

4mo

feat(client): make stat cards numbers animated, rename text

f065b614

ptr.pet

4mo

feat(client): improve styling a lot to work better on mobile and different screen sizes

a13ddcde

ptr.pet

4mo

refactor(server): use existing Events type, create LatencyMillis for better latency formatting

c806b451

ptr.pet

4mo

feat(client): make stat cards more viewable on mobile

9c8c6ece

ptr.pet

4mo

build(nix): add hash for axum-tws

90e4de63

ptr.pet

4mo

feat(server): increase stream send rate

c2a22b18

ptr.pet

4mo

feat(server): fix request tracing

28ac2ab1

ptr.pet

4mo

feat: track and send events per second, send 10 WS messages maximum per second, buffering in between

0c23c8d7

ptr.pet

4mo

refactor(server): drop atproto-jetstream, use tokio-websockets directly, improve ctrl_c and jetstream connection retries

0e76b961

ptr.pet

4mo

next