The bmannconsulting.com website
bmann.ca
1---
2aliases:
3 - Community Search Engines
4tags:
5 - search
6 - localfirst
7---
8**Community Search Engines** are an approach to build highly relevant search indexes and collaborative content spaces for communities.
9
10The interface is similar to agent chat interfaces -- chatting and refining to get what you're looking for -- as well as personal notes / bookmark collections[^tft] -- browsing and interacting with "your stuff" to find what you've stored before.
11
12[^tft]: I'm not using tools for thought or second brain labels here. For those that have adopted such a system, its content would be a major input to such a system.
13
14I'm using the label search engine as a term that I think will resonate with the mass market: you go there to find stuff.
15## What are we trying to fix?
16
17Google has broke the social contract of search. Even without [[AI slop]] being placed as the first "result", the results returned lead to top ranking pages that themselves are low quality SEO garbage.
18
19I can't easily find results that are useful or relevant to me any more.
20
21OK, I'll ask my friends for what they think! But, on corporate social platforms such as Twitter, Instagram, or Threads, the algorithm is centrally controlled and can't be scoped to your friends.
22
23You're not sure that your request will be seen by your connections, and your connections won't necessarily see your post.
24
25## Discovery and Distribution
26
27This covers two aspects:
28* discovery -- browsing and finding what you're looking for, ideally with a tune-able, transparent algorithm
29* distribution -- being able to share "your stuff" in a way that it's possible for others to in turn see it and discover it
30
31For both, we think that [[Open Social Protocols]] offer a way forward. We also think distribution is up stream of discovery aka [[DREAM]].
32
33If your information isn't seen because it isn't indexed or shared, or is algorithmically black holed or suppressed, it can't be discovered.
34## Emerging Tools
35
36[[DXOS]] has their [[DXOS Composer]] suite. Local first content, LLM models, and composition of a number of different tools, from Discord API connections for indexing, to audio / video transcription.
37
38The [[Patchwork]] system by [[Ink & Switch]] stores data local-first, cached in your browser. It relies on [[Automerge]] for sync. It doesn't have much in the way of a search index at all, and until [[Keyhive]] is integrated, doesn't have a permissions system. It can do multiplayer collaboration by sharing links to content.
39
40[[Groundmist]] is a proof of concept that uses [[Ink & Switch]]'s tiny essay editor with local-first, private data (a precursor to Patchwork, it also uses [[Automerge]]) and combines it with [[ATProtocol]] for public publishing as well as structured data via the AT Protocol Lexicon system.
41
42[[Tonk]] has just released the [Tonk CLI and TonkbookLM](https://tonk.substack.com/p/tonk-the-explainer). Uses [[Automerge]]. Specifically mentions [[Obsidian]] as a source of local data.
43
44## How do we build community search engines?
45
46I envision a [[Personal Notes & Publishing Stack]] that enables people and communities to collaborate, pool information, and add context and relevance.
47
48## Further Reading
49
50[[Dark Forest and Cozy Web]]: communities are moving to cozy web spaces, and/or have need for members only content. The open internet has degraded search incentivized by click throughs to show ads, and commercial social platforms
51
52[[Brazilianization of the Internet]]: forming communities, which can be considered a kind of commons, trending towards [[Cozy Web]] spaces. Does this cause a kind of elitism, and/or further deteriorate public spaces?
53
54[[Erin Kissane]]'s [[Against the Dark Forest]] specifically argues that we need to protect the publis social internet and not all retreat into [[Cozy Web]] corners. The entire article is dense and amazing and I could quote every paragraph.
55
56[[How Algolia uses Electron to improve internal productivity]]: I’m still inspired by this description of Algolia implementing company wide search across internal tools, with desktop integration. This should be a tool for organizations of all kinds.
57
58## Previously
59
60I rewrote the page that was here previously to refactor it extensively.
61
62I applied to [[SoP 2024 Application]] with some ideas around the original formulation of using events and digital communities as a way to bootstrap the search engine, without tackling the [[Personal Notes & Publishing Stack]].
63
64Part of community is "flow" ([[Stock and Flow]]) and [[Proto Apps]] is how I think social graphs / communities / flows of information form.