proposal: webhook support for git events #97

+1 I like having a stream of my git events sent to my Discord so

1. I can see any activity on my projects without leaving my main comms app
2. Other people in my Discord can see my activity on my projects and know I haven't run away lmao

+1 Enabling CI capabilities would also make me more comfortable moving things to tangled

Aren't Tangled events published as ATProto events? There could be a service that would listen on ATProto stream and then apply required actions whenever matching event is encountered.

All the events for pulls, pull_comments, issues, and issue_comments are, but not the core git events like pushing commits to a branch or pushing a tag, which would be the most useful events for a CI server.

Ahh, ok. TIL

Indeed it would be handy to have actions/web hooks on these as well.

would a webhook event include entire repo owner, repository, commit (if applicable) and user (the user that caused the webhook to fire, i.e.: by pushing to the repository) information, like github? or would it just reference their ids and we'd have to fetch all those items seperatly.

I think it makes most sense to put more data into ATProto…I don’t know the limitations entirely but at a minimum publishing a record that a push occurred seems better than a bespoke webhook API

And secondarily, I wonder what could be if commits existed on-protocol as well

There's a lot of value in something like this but I also think there's definitely value in thinking a bit more about how to properly do this and especially in how to do this in a way that meshes nicely with atproto and the atmosphere as a whole and avoids being too bespoke. I think it will be hard to directly hook up events like this to record creation since the relationship between data and identities in git and in atproto are fundamentally different (git has a one repo to many users model with a somewhat loser coupling between identity and data where atproto has a one user to one repo model with very strong coupling between identity and data) on top of that what we're talking about here is events not persistent data. Publishing a record on push makes a one off event into a persistent data point (a one off event that isnt even a persistent data point in the pure git context). It can be argued that it might make sense to publish persistent git data to an atproto repo (like putting git commits on protocol) however that runs up against the issue of conflicting data models (regarding everything from what identity should hold the commit records to who should create those records to how does whoever/whatever that creates the records gain the authorisation to create them etc) so if we do go that route there needs to be some serious serious thinking done about what exactly we'd want out of that and what compromises we need to make.

For getting this to mesh nicely with the atmosphere and be less of a bespoke API one thing that immediately comes to mind is using the language of atproto: XRPC and lexicons. XRPC includes an event stream endpoint type which is an obvious solution for how to build out an event API like this. This has the added benefits of proper schema definitions for the API from the get go, lexicon based code generation from said schema and that XRPC code already exists in the tangled code base and integrates well with the atmosphere as a whole. The view here is that knots are a part of the atproto network. They are a "supporting service" in the atmosphere that have a similar place protocol wise as feed generators and the like do (not quite the same since feed generators are still ultimately dealing with repo records but I digress). Using XRPC like this to define the API of off protocol services that interact with the network is decently standard in the ecosystem (feed generators, bsky DMs, etc).

I do very much agree that the knot is the logical service to provide git events to the network but I don't think that the appview pushing network events to the knot for it to push to consumers is a good idea architecturally. Consumers should consume the firehose in some form for that info. Although that is very noisy to do straight off the network wide firehose so it would probably be good for the tangled appview to provide a filtered firehose view for this purpose. One where clients can request a specific scope ("everything happening around this repo", "everything this person does", whatever). So that is one event stream from the knot publishing git events and one from the appview publishing network events. This also better takes into account the possibility of multiple independent appviews and would allow this network event stream to be spun off the appview into its own service at some point if wanted.

@nelind.dk While in a perfect world I'd agree, but hopefully there's going to be many users here on Tangled who have no interest in building for ATProto (aiming for a wider scope of devs who just happen to have an ATProto account). The Jetstream/Firehose will be a new concept to them, and they'll be very confused why they can't just find a thing that pushes out events they can capture — especially from a thing where it's easy to lag behind unless optimized properly — and likely be put off by Tangled.

@ducky.ws yea thats true. thats a major downside of going with XRPC for something like this. perhaps doing a hybrid model where there is also a more conventional API is available for those who dont want to deal with XRPC-isms. at that point we might as well start thinking about the value of having multiple APIs though

Note that are two previously existing issues for the same idea though with less discussion: https://tangled.sh/@tangled.sh/core/issues/47, https://tangled.sh/@tangled.sh/core/issues/94.