commits · ptr.pet/nsid-tracker · tangled

ptr.pet / nsid-tracker

tracks lexicons and how many times they appeared on the jetstream

commits

Author

Commit

Message

Date

ptr.pet

d808716d

fix(server): only push for sync if we have anything to sync

4mo ago

ptr.pet

8bbc0246

fix(client): check if date is 'invalid' which means its old nanoseconds since too lazy to fix it in db

4mo ago

ptr.pet

86a2eabe

fix(client): timestamps are seconds but actually

4mo ago

ptr.pet

2d367d44

fix(client): timestamps are seconds now so show that properly, also disable numberflow its bad

4mo ago

ptr.pet

9036e3a9

feat(server): improve the logging of sync and compaction

4mo ago

ptr.pet

ce8f1ca0

refactor(server): make max_last_activity Duration

4mo ago

ptr.pet

cbc9e661

Revert "feat(server): train a zstd dict"

4mo ago

ptr.pet

9b006d33

Revert "feat(server): zstd compression"

4mo ago

ptr.pet

852d8aea

refactor(server): only print eps if its more than 0, eg. doing something

4mo ago

ptr.pet

4174678b

feat(server): zstd compression

4mo ago

ptr.pet

6001b7f8

feat(server): train a zstd dict

4mo ago

ptr.pet

b63273b4

refactor(server): increase block size even more, reenable compression

4mo ago

ptr.pet

bf2179a4

refactor(server): dont use any compression for trees

4mo ago

ptr.pet

a58727c0

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

4mo ago

ptr.pet

a10342d6

feat(server): more threads for fjall workers when migrating

4mo ago

ptr.pet

e7a40253

feat(server): re-enable jemalloc for non-msvc targets

4mo ago

ptr.pet

3a678143

refactor(server): use more workers on migrate db

4mo ago

ptr.pet

f864854c

feat(server): use unlimited journaling and write buffer size on compact cmd as well

4mo ago

ptr.pet

55d6b407

refactor(server): maximum journaling size while migrating, no manual persist

4mo ago

ptr.pet

84b0f5ef

fix(server): use deser to access item data because zero-copy is not guaranteed

4mo ago

ptr.pet

48460bf4

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

4mo ago

ptr.pet

11924417

feat(server): make db configurable, improve migration perf

4mo ago

ptr.pet

4c4a5aa3

feat(server): actually make the block inserts start in parallel

4mo ago

ptr.pet

fddb212c

add more logging for syncing and migrating

4mo ago

ptr.pet

aebca039

chore: add fjall data stuff to gitignore

4mo ago

ptr.pet

d0e611bc

fix(server): encode_block takes correct amount of items, improve rate limiter usage

4mo ago

ptr.pet

7503870a

fix(server): sync remainder if all is passed to sync

4mo ago

ptr.pet

29914c94

feat(server): sort on major compact

4mo ago

ptr.pet

bf7842d0

feat(server): implement compaction :3

4mo ago

ptr.pet

2a7d6005

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

4mo ago

ptr.pet

e2dd9f2b

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

4mo ago

ptr.pet

86f3fa23

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

4mo ago

ptr.pet

52fd5331

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

4mo ago

ptr.pet

a820ae13

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

4mo ago

ptr.pet

925dbc95

refactor(server): use either instead of boxed iter

4mo ago

ptr.pet

7cd979b3

refactor(server): improve record events by splitting it into ingest_events

4mo ago

ptr.pet

13e1dd85

build(nix): add cmake

4mo ago

ptr.pet

7920317e

feat(server): track more bench info in compact

4mo ago

ptr.pet

7d08ae63

fix(server): remove broken imports

4mo ago

ptr.pet

1dd2ef8c

refactor(server): use quanta for last activity on lexicon handles aswell

4mo ago

ptr.pet

6b78fe2d

refactor(server): implement own rate counter that should be faster, use quanta on it

4mo ago

ptr.pet

32529fe8

refactor(server): use relaxed ordering for keeping track of lexicon buf len / last activity / block size since we don't really need any strict ordering

4mo ago

ptr.pet

3456ca30

feat(server): use snmalloc

4mo ago

ptr.pet

5e769184

refactor(server): run sync jobs in threadpool

4mo ago

ptr.pet

84d25308

refactor(server): delete old db impl, rename migrate to compact

4mo ago

ptr.pet

48892f4c

feat(server): run all trees migrations in parallel

4mo ago

ptr.pet

b7a5f07a

feat(server): allow setting a max block size

4mo ago

ptr.pet

4e1f68a4

fix(server): actually sync on migrate lol

4mo ago

ptr.pet

ebeab2a7

fix(server): tracking_since funcs, also fix old db get_hits

4mo ago

ptr.pet

e28fd9c4

feat(server): error on unknwn cmd

4mo ago

ptr.pet

d7265904

feat(server): implement delta-of-deltas encoding and varint and chunking

4mo ago

ptr.pet

f73de8d3

fix(client): make the stats card numbers clip less

4mo ago

ptr.pet

d275783d

feat(client): show tracking since

4mo ago

ptr.pet

999365ad

feat(server): implement fetching tracking since

4mo ago

ptr.pet

383ee9e6

fix(client): fix refresh rate resetting from 2 to realtime this time

4mo ago

ptr.pet

ffd32aa0

feat(client): put bsky toggle right after filter

4mo ago

ptr.pet

90da2f14

build(nix): update modules hash

4mo ago

ptr.pet

310ffddf

docs: update readme

4mo ago

ptr.pet

90220382

fix(client): reset refresh rate to real-time only if user hasnt changed refresh rate

4mo ago

ptr.pet

e020b36f

feat(server): implement fetching hits

4mo ago

fix(server): only push for sync if we have anything to sync

d808716d

ptr.pet

4mo

fix(client): check if date is 'invalid' which means its old nanoseconds since too lazy to fix it in db

8bbc0246

ptr.pet

4mo

fix(client): timestamps are seconds but actually

86a2eabe

ptr.pet

4mo

fix(client): timestamps are seconds now so show that properly, also disable numberflow its bad

2d367d44

ptr.pet

4mo

feat(server): improve the logging of sync and compaction

9036e3a9

ptr.pet

4mo

refactor(server): make max_last_activity Duration

ce8f1ca0

ptr.pet

4mo

Revert "feat(server): train a zstd dict"

cbc9e661

ptr.pet

4mo

Revert "feat(server): zstd compression"

9b006d33

ptr.pet

4mo

refactor(server): only print eps if its more than 0, eg. doing something

852d8aea

ptr.pet

4mo

feat(server): zstd compression

4174678b

ptr.pet

4mo

feat(server): train a zstd dict

6001b7f8

ptr.pet

4mo

refactor(server): increase block size even more, reenable compression

b63273b4

ptr.pet

4mo

refactor(server): dont use any compression for trees

bf2179a4

ptr.pet

4mo

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

a58727c0

ptr.pet

4mo

feat(server): more threads for fjall workers when migrating

a10342d6

ptr.pet

4mo

feat(server): re-enable jemalloc for non-msvc targets

e7a40253

ptr.pet

4mo

refactor(server): use more workers on migrate db

3a678143

ptr.pet

4mo

feat(server): use unlimited journaling and write buffer size on compact cmd as well

f864854c

ptr.pet

4mo

refactor(server): maximum journaling size while migrating, no manual persist

55d6b407

ptr.pet

4mo

fix(server): use deser to access item data because zero-copy is not guaranteed

84b0f5ef

ptr.pet

4mo

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

48460bf4

ptr.pet

4mo

feat(server): make db configurable, improve migration perf

11924417

ptr.pet

4mo

feat(server): actually make the block inserts start in parallel

4c4a5aa3

ptr.pet

4mo

add more logging for syncing and migrating

fddb212c

ptr.pet

4mo

chore: add fjall data stuff to gitignore

aebca039

ptr.pet

4mo

fix(server): encode_block takes correct amount of items, improve rate limiter usage

d0e611bc

ptr.pet

4mo

fix(server): sync remainder if all is passed to sync

7503870a

ptr.pet

4mo

feat(server): sort on major compact

29914c94

ptr.pet

4mo

feat(server): implement compaction :3

bf7842d0

ptr.pet

4mo

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

2a7d6005

ptr.pet

4mo

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

e2dd9f2b

ptr.pet

4mo

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

86f3fa23

ptr.pet

4mo

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

52fd5331

ptr.pet

4mo

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

a820ae13

ptr.pet

4mo

refactor(server): use either instead of boxed iter

925dbc95

ptr.pet

4mo

refactor(server): improve record events by splitting it into ingest_events

7cd979b3

ptr.pet

4mo

build(nix): add cmake

13e1dd85

ptr.pet

4mo

feat(server): track more bench info in compact

7920317e

ptr.pet

4mo

fix(server): remove broken imports

7d08ae63

ptr.pet

4mo

refactor(server): use quanta for last activity on lexicon handles aswell

1dd2ef8c

ptr.pet

4mo

refactor(server): implement own rate counter that should be faster, use quanta on it

6b78fe2d

ptr.pet

4mo

refactor(server): use relaxed ordering for keeping track of lexicon buf len / last activity / block size since we don't really need any strict ordering

32529fe8

ptr.pet

4mo

feat(server): use snmalloc

3456ca30

ptr.pet

4mo

refactor(server): run sync jobs in threadpool

5e769184

ptr.pet

4mo

refactor(server): delete old db impl, rename migrate to compact

84d25308

ptr.pet

4mo

feat(server): run all trees migrations in parallel

48892f4c

ptr.pet

4mo

feat(server): allow setting a max block size

b7a5f07a

ptr.pet

4mo

fix(server): actually sync on migrate lol

4e1f68a4

ptr.pet

4mo

fix(server): tracking_since funcs, also fix old db get_hits

ebeab2a7

ptr.pet

4mo

feat(server): error on unknwn cmd

e28fd9c4

ptr.pet

4mo

feat(server): implement delta-of-deltas encoding and varint and chunking

d7265904

ptr.pet

4mo

fix(client): make the stats card numbers clip less

f73de8d3

ptr.pet

4mo

feat(client): show tracking since

d275783d

ptr.pet

4mo

feat(server): implement fetching tracking since

999365ad

ptr.pet

4mo

fix(client): fix refresh rate resetting from 2 to realtime this time

383ee9e6

ptr.pet

4mo

feat(client): put bsky toggle right after filter

ffd32aa0

ptr.pet

4mo

build(nix): update modules hash

90da2f14

ptr.pet

4mo

docs: update readme

310ffddf

ptr.pet

4mo

fix(client): reset refresh rate to real-time only if user hasnt changed refresh rate

90220382

ptr.pet

4mo

feat(server): implement fetching hits

e020b36f

ptr.pet

4mo

next