My agentic slop goes here. Not intended for anyone else!
anil.recoil.org
1# Zulip Library Architecture
2
3## Overview
4
5The Zulip OCaml library follows a clean, layered architecture that separates protocol types, encoding concerns, and HTTP communication.
6
7## Architecture Layers
8
9```
10┌─────────────────────────────────────┐
11│ API Modules (Messages, Channels) │ ← High-level operations
12├─────────────────────────────────────┤
13│ Protocol Types (Message, Channel) │ ← Business logic types with Jsont codecs
14├─────────────────────────────────────┤
15│ Encode Module │ ← JSON/Form encoding utilities
16├─────────────────────────────────────┤
17│ Client Module │ ← HTTP request/response handling
18├─────────────────────────────────────┤
19│ Requests Library (EIO-based) │ ← Low-level HTTP
20└─────────────────────────────────────┘
21```
22
23## Key Design Principles
24
25### 1. **Protocol Types with Jsont Codecs**
26
27Each Zulip API type (Message, Channel, User, etc.) has:
28- A clean OCaml record type
29- A `jsont` codec that defines bidirectional JSON conversion
30- Accessor functions
31- Pretty printer
32
33Example from `channel.ml`:
34```ocaml
35type t = {
36 name : string;
37 description : string;
38 invite_only : bool;
39 history_public_to_subscribers : bool;
40}
41
42let jsont =
43 Jsont.Object.map ~kind:"Channel" make
44 |> Jsont.Object.mem "name" Jsont.string ~enc:name
45 |> Jsont.Object.mem "description" Jsont.string ~enc:description
46 |> Jsont.Object.mem "invite_only" Jsont.bool ~enc:invite_only
47 |> Jsont.Object.mem "history_public_to_subscribers" Jsont.bool ~enc:history_public_to_subscribers
48 |> Jsont.Object.finish
49```
50
51### 2. **Encode Module: Separation of Encoding Concerns**
52
53The `Encode` module provides clean utilities for converting between OCaml types and wire formats:
54
55```ocaml
56(** Convert using a jsont codec *)
57val to_json_string : 'a Jsont.t -> 'a -> string
58val to_form_urlencoded : 'a Jsont.t -> 'a -> string
59val from_json : 'a Jsont.t -> Jsont.json -> ('a, string) result
60```
61
62This eliminates the need for:
63- ❌ Manual JSON tree walking
64- ❌ Round-trip encode→decode conversions
65- ❌ Per-type encoding functions
66
67### 3. **Request/Response Types with Codecs**
68
69API operations define request/response types locally with their codecs:
70
71```ocaml
72(* In channels.ml *)
73module Subscribe_request = struct
74 type t = { subscriptions : string list }
75
76 let codec =
77 Jsont.Object.(
78 map ~kind:"SubscribeRequest" (fun subscriptions -> { subscriptions })
79 |> mem "subscriptions" (Jsont.list Jsont.string) ~enc:(fun r -> r.subscriptions)
80 |> finish
81 )
82end
83
84let subscribe client ~channels =
85 let req = Subscribe_request.{ subscriptions = channels } in
86 let body = Encode.to_form_urlencoded Subscribe_request.codec req in
87 let content_type = "application/x-www-form-urlencoded" in
88 match Client.request client ~method_:`POST ~path:"/api/v1/users/me/subscriptions"
89 ~body ~content_type () with
90 | Ok _json -> Ok ()
91 | Error err -> Error err
92```
93
94### 4. **Type-Safe Decoding**
95
96Response parsing uses codecs directly instead of manual pattern matching:
97
98```ocaml
99(* OLD - manual JSON walking *)
100match json with
101| Jsont.Object (fields, _) ->
102 let assoc = List.map (fun ((k, _), v) -> (k, v)) fields in
103 (match List.assoc_opt "streams" assoc with
104 | Some (Jsont.Array (channel_list, _)) -> ...
105
106(* NEW - type-safe codec *)
107let response_codec =
108 Jsont.Object.(
109 map ~kind:"StreamsResponse" (fun streams -> streams)
110 |> mem "streams" (Jsont.list Channel.jsont) ~enc:(fun x -> x)
111 |> finish
112 )
113in
114match Encode.from_json response_codec json with
115| Ok channels -> Ok channels
116| Error msg -> Error (...)
117```
118
119## Benefits
120
121### ✅ Type Safety
122- Jsont codecs ensure correct JSON structure
123- Compilation errors catch schema mismatches
124- No runtime type confusion
125
126### ✅ Maintainability
127- Protocol changes only require updating codecs
128- No manual JSON manipulation scattered through code
129- Clear separation of concerns
130
131### ✅ Reusability
132- Codecs can be composed and reused
133- Encode module works for any jsont-encoded type
134- Request/response types are self-documenting
135
136### ✅ Testability
137- Easy to test encoding/decoding in isolation
138- Mock responses can be type-checked
139- Round-trip property testing possible
140
141## Migration Pattern
142
143When adding new API endpoints:
144
1451. **Define the protocol type with codec**:
146 ```ocaml
147 type my_request = { field1: string; field2: int }
148
149 let my_request_codec =
150 Jsont.Object.(
151 map ~kind:"MyRequest" (fun field1 field2 -> { field1; field2 })
152 |> mem "field1" Jsont.string ~enc:(fun r -> r.field1)
153 |> mem "field2" Jsont.int ~enc:(fun r -> r.field2)
154 |> finish
155 )
156 ```
157
1582. **Encode using Encode module**:
159 ```ocaml
160 let body = Encode.to_form_urlencoded my_request_codec req in
161 (* or *)
162 let json = Encode.to_json_string my_request_codec req in
163 ```
164
1653. **Decode responses with codec**:
166 ```ocaml
167 match Client.request client ~method_:`POST ~path:"/api/..." ~body () with
168 | Ok json ->
169 (match Encode.from_json response_codec json with
170 | Ok data -> Ok data
171 | Error msg -> Error ...)
172 ```
173
174## Comparison with Old Approach
175
176### Old (Manual JSON Manipulation):
177```ocaml
178let send client message =
179 let json = Message.to_json message in (* Round-trip conversion *)
180 let params = match json with
181 | Jsont.Object (fields, _) -> (* Manual pattern matching *)
182 List.fold_left (fun acc ((key, _), value) ->
183 let str_value = match value with (* More pattern matching *)
184 | Jsont.String (s, _) -> s
185 | Jsont.Bool (true, _) -> "true"
186 | _ -> ""
187 in
188 (key, str_value) :: acc
189 ) [] fields
190 | _ -> [] in
191 (* ... *)
192```
193
194### New (Codec-Based):
195```ocaml
196let send client message =
197 let body = Message.to_form_urlencoded message in (* Clean encoding *)
198 let content_type = "application/x-www-form-urlencoded" in
199 match Client.request client ~method_:`POST ~path:"/api/v1/messages"
200 ~body ~content_type () with
201 | Ok response -> Message_response.of_json response
202 | Error err -> Error err
203```
204
205## Future Enhancements
206
207- **Validation**: Add validation layers on top of codecs
208- **Versioning**: Support multiple API versions with codec variants
209- **Documentation**: Generate API docs from codec definitions
210- **Testing**: Property-based testing with codec round-trips
211- **Code Generation**: Consider generating codecs from OpenAPI specs
212
213## References
214
215- Jsont library: https://erratique.ch/software/jsont
216- Zulip REST API: https://zulip.com/api/rest
217- Original design doc: `CLAUDE.md`