commits · ptr.pet/nsid-tracker · tangled

ptr.pet / nsid-tracker

tracks lexicons and how many times they appeared on the jetstream

commits

Author

Commit

Message

Date

ptr.pet

f434daf7

feat(server): use snmalloc again on everything

4mo ago

ptr.pet

c194514c

fix(server): fix ordering on get_hits again sob

4mo ago

ptr.pet

89f0c71b

fix(server): check for max items before adding to current item count in get_hits

4mo ago

ptr.pet

8a875861

refactor(server): change compaction to run from now to period ago

4mo ago

ptr.pet

febe611b

fix(server): fix get_hits ordering bleh

4mo ago

ptr.pet

926fd1ab

fix(server): fix compact ordering???

4mo ago

ptr.pet

ff50dac5

feat(server): add cmd to print all hits

4mo ago

ptr.pet

f4563532

fix(server): implement max items on get_hits so we dont read every block if requested

4mo ago

ptr.pet

dc9df83a

feat(server): add blocks returned trace on get_hits

4mo ago

ptr.pet

5cc9c68c

feat(server): convert microsecond timestamps to seconds

4mo ago

ptr.pet

861c4a13

feat(server): also log when blocks are skipped and when not

4mo ago

ptr.pet

d0e3c929

Revert "feat(server): decrease block size more"

4mo ago

ptr.pet

72dcca0f

feat(server): also log block size in get hits

4mo ago

ptr.pet

13e6ce8d

feat(server): time logging to get_hits

4mo ago

ptr.pet

f5434e9b

feat(server): decrease block size more

4mo ago

ptr.pet

e77f32a4

feat(server): disable write buffer always

4mo ago

ptr.pet

109a2f6c

refactor(server): dont run major compaction on tres

4mo ago

ptr.pet

862edeee

refactor(server): decrease max block size

4mo ago

ptr.pet

89a0ea69

refactor(server): decrease block size for partitions and increase cache

4mo ago

ptr.pet

8d7c71b8

fix(server): reverse the ordering of the sort in compact

4mo ago

ptr.pet

d808716d

fix(server): only push for sync if we have anything to sync

4mo ago

ptr.pet

8bbc0246

fix(client): check if date is 'invalid' which means its old nanoseconds since too lazy to fix it in db

4mo ago

ptr.pet

86a2eabe

fix(client): timestamps are seconds but actually

4mo ago

ptr.pet

2d367d44

fix(client): timestamps are seconds now so show that properly, also disable numberflow its bad

4mo ago

ptr.pet

9036e3a9

feat(server): improve the logging of sync and compaction

4mo ago

ptr.pet

ce8f1ca0

refactor(server): make max_last_activity Duration

4mo ago

ptr.pet

cbc9e661

Revert "feat(server): train a zstd dict"

4mo ago

ptr.pet

9b006d33

Revert "feat(server): zstd compression"

4mo ago

ptr.pet

852d8aea

refactor(server): only print eps if its more than 0, eg. doing something

4mo ago

ptr.pet

4174678b

feat(server): zstd compression

4mo ago

ptr.pet

6001b7f8

feat(server): train a zstd dict

4mo ago

ptr.pet

b63273b4

refactor(server): increase block size even more, reenable compression

4mo ago

ptr.pet

bf2179a4

refactor(server): dont use any compression for trees

4mo ago

ptr.pet

a58727c0

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

4mo ago

ptr.pet

a10342d6

feat(server): more threads for fjall workers when migrating

4mo ago

ptr.pet

e7a40253

feat(server): re-enable jemalloc for non-msvc targets

4mo ago

ptr.pet

3a678143

refactor(server): use more workers on migrate db

4mo ago

ptr.pet

f864854c

feat(server): use unlimited journaling and write buffer size on compact cmd as well

4mo ago

ptr.pet

55d6b407

refactor(server): maximum journaling size while migrating, no manual persist

4mo ago

ptr.pet

84b0f5ef

fix(server): use deser to access item data because zero-copy is not guaranteed

4mo ago

ptr.pet

48460bf4

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

4mo ago

ptr.pet

11924417

feat(server): make db configurable, improve migration perf

4mo ago

ptr.pet

4c4a5aa3

feat(server): actually make the block inserts start in parallel

4mo ago

ptr.pet

fddb212c

add more logging for syncing and migrating

4mo ago

ptr.pet

aebca039

chore: add fjall data stuff to gitignore

4mo ago

ptr.pet

d0e611bc

fix(server): encode_block takes correct amount of items, improve rate limiter usage

4mo ago

ptr.pet

7503870a

fix(server): sync remainder if all is passed to sync

4mo ago

ptr.pet

29914c94

feat(server): sort on major compact

4mo ago

ptr.pet

bf7842d0

feat(server): implement compaction :3

4mo ago

ptr.pet

2a7d6005

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

4mo ago

ptr.pet

e2dd9f2b

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

4mo ago

ptr.pet

86f3fa23

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

4mo ago

ptr.pet

52fd5331

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

4mo ago

ptr.pet

a820ae13

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

4mo ago

ptr.pet

925dbc95

refactor(server): use either instead of boxed iter

4mo ago

ptr.pet

7cd979b3

refactor(server): improve record events by splitting it into ingest_events

4mo ago

ptr.pet

13e1dd85

build(nix): add cmake

4mo ago

ptr.pet

7920317e

feat(server): track more bench info in compact

4mo ago

ptr.pet

7d08ae63

fix(server): remove broken imports

4mo ago

ptr.pet

1dd2ef8c

refactor(server): use quanta for last activity on lexicon handles aswell

4mo ago

feat(server): use snmalloc again on everything

f434daf7

ptr.pet

4mo

fix(server): fix ordering on get_hits again sob

c194514c

ptr.pet

4mo

fix(server): check for max items before adding to current item count in get_hits

89f0c71b

ptr.pet

4mo

refactor(server): change compaction to run from now to period ago

8a875861

ptr.pet

4mo

fix(server): fix get_hits ordering bleh

febe611b

ptr.pet

4mo

fix(server): fix compact ordering???

926fd1ab

ptr.pet

4mo

feat(server): add cmd to print all hits

ff50dac5

ptr.pet

4mo

fix(server): implement max items on get_hits so we dont read every block if requested

f4563532

ptr.pet

4mo

feat(server): add blocks returned trace on get_hits

dc9df83a

ptr.pet

4mo

feat(server): convert microsecond timestamps to seconds

5cc9c68c

ptr.pet

4mo

feat(server): also log when blocks are skipped and when not

861c4a13

ptr.pet

4mo

Revert "feat(server): decrease block size more"

d0e3c929

ptr.pet

4mo

feat(server): also log block size in get hits

72dcca0f

ptr.pet

4mo

feat(server): time logging to get_hits

13e6ce8d

ptr.pet

4mo

feat(server): decrease block size more

f5434e9b

ptr.pet

4mo

feat(server): disable write buffer always

e77f32a4

ptr.pet

4mo

refactor(server): dont run major compaction on tres

109a2f6c

ptr.pet

4mo

refactor(server): decrease max block size

862edeee

ptr.pet

4mo

refactor(server): decrease block size for partitions and increase cache

89a0ea69

ptr.pet

4mo

fix(server): reverse the ordering of the sort in compact

8d7c71b8

ptr.pet

4mo

fix(server): only push for sync if we have anything to sync

d808716d

ptr.pet

4mo

fix(client): check if date is 'invalid' which means its old nanoseconds since too lazy to fix it in db

8bbc0246

ptr.pet

4mo

fix(client): timestamps are seconds but actually

86a2eabe

ptr.pet

4mo

fix(client): timestamps are seconds now so show that properly, also disable numberflow its bad

2d367d44

ptr.pet

4mo

feat(server): improve the logging of sync and compaction

9036e3a9

ptr.pet

4mo

refactor(server): make max_last_activity Duration

ce8f1ca0

ptr.pet

4mo

Revert "feat(server): train a zstd dict"

cbc9e661

ptr.pet

4mo

Revert "feat(server): zstd compression"

9b006d33

ptr.pet

4mo

refactor(server): only print eps if its more than 0, eg. doing something

852d8aea

ptr.pet

4mo

feat(server): zstd compression

4174678b

ptr.pet

4mo

feat(server): train a zstd dict

6001b7f8

ptr.pet

4mo

refactor(server): increase block size even more, reenable compression

b63273b4

ptr.pet

4mo

refactor(server): dont use any compression for trees

bf2179a4

ptr.pet

4mo

refactor(server): use vec instead of writeablebyteview for value to reduce size of values

a58727c0

ptr.pet

4mo

feat(server): more threads for fjall workers when migrating

a10342d6

ptr.pet

4mo

feat(server): re-enable jemalloc for non-msvc targets

e7a40253

ptr.pet

4mo

refactor(server): use more workers on migrate db

3a678143

ptr.pet

4mo

feat(server): use unlimited journaling and write buffer size on compact cmd as well

f864854c

ptr.pet

4mo

refactor(server): maximum journaling size while migrating, no manual persist

55d6b407

ptr.pet

4mo

fix(server): use deser to access item data because zero-copy is not guaranteed

84b0f5ef

ptr.pet

4mo

refactor(server): make encoding items actually par (first collect all items and then par iter the encoding)

48460bf4

ptr.pet

4mo

feat(server): make db configurable, improve migration perf

11924417

ptr.pet

4mo

feat(server): actually make the block inserts start in parallel

4c4a5aa3

ptr.pet

4mo

add more logging for syncing and migrating

fddb212c

ptr.pet

4mo

chore: add fjall data stuff to gitignore

aebca039

ptr.pet

4mo

fix(server): encode_block takes correct amount of items, improve rate limiter usage

d0e611bc

ptr.pet

4mo

fix(server): sync remainder if all is passed to sync

7503870a

ptr.pet

4mo

feat(server): sort on major compact

29914c94

ptr.pet

4mo

feat(server): implement compaction :3

bf7842d0

ptr.pet

4mo

refactor(server): use rclite and parking_lot, replace the scc::Queue with a Arc<Mutex<Vec<_>>>

2a7d6005

ptr.pet

4mo

fix(server): chunk block encodes and make sure each chunk is processed sequentially, but per nsid is processed in parallel

e2dd9f2b

ptr.pet

4mo

fix(server): handle byteview with empty bytes at the end by letting the amount of elements be specified in the blocks

86f3fa23

ptr.pet

4mo

refactor(server): use rayon to compute the blocks, and then exec inserting them in thread pool

52fd5331

ptr.pet

4mo

refactor(server): implement WriteableByteView so we don't have to copy from a Vec while constructing fjall::Slice

a820ae13

ptr.pet

4mo

refactor(server): use either instead of boxed iter

925dbc95

ptr.pet

4mo

refactor(server): improve record events by splitting it into ingest_events

7cd979b3

ptr.pet

4mo

build(nix): add cmake

13e1dd85

ptr.pet

4mo

feat(server): track more bench info in compact

7920317e

ptr.pet

4mo

fix(server): remove broken imports

7d08ae63

ptr.pet

4mo

refactor(server): use quanta for last activity on lexicon handles aswell

1dd2ef8c

ptr.pet

4mo

next