My agentic slop goes here. Not intended for anyone else!
anil.recoil.org
1I would like to build high quality OCaml bindings to the Zulip REST API,
2documented at https://zulip.com/api/rest. As another reference, the Python
3`zulip` library from pip is well maintained.
4
5My target is to use the OCaml EIO direct-style library, with an idiomatic as
6possible API that implements it. For JSON parsing, using the jsonm library is
7right. For HTTPS, use cohttp-eio with the tls-eio library. You have access to
8an OCaml LSP via MCP which provides type hints and other language server
9features after you complete a `dune build`.
10
11# OCaml Zulip Library Design
12
13Based on analysis of:
14- Zulip REST API documentation: https://zulip.com/api/rest
15- Python zulip library: https://github.com/zulip/python-zulip-api
16- Zulip error handling: https://zulip.com/api/rest-error-handling
17- Zulip send message API: https://zulip.com/api/send-message
18
19## Overview
20The library follows OCaml best practices with abstract types (`type t`) per module, comprehensive constructors/accessors, and proper pretty printers. Each core concept gets its own module with a clean interface.
21
22## Module Structure
23
24### Authentication (`Zulip.Auth`)
25```ocaml
26type t (* abstract *)
27
28val create : server_url:string -> email:string -> api_key:string -> t
29val from_zuliprc : ?path:string -> unit -> (t, Error.t) result
30val server_url : t -> string
31val email : t -> string
32val to_basic_auth_header : t -> string
33val pp : Format.formatter -> t -> unit
34```
35
36### Error Handling (`Zulip.Error`)
37```ocaml
38type code =
39 | Invalid_api_key
40 | Request_variable_missing
41 | Bad_request
42 | User_deactivated
43 | Realm_deactivated
44 | Rate_limit_hit
45 | Other of string
46
47type t (* abstract *)
48
49val create : code:code -> msg:string -> ?extra:(string * Jsonm.value) list -> unit -> t
50val code : t -> code
51val message : t -> string
52val extra : t -> (string * Jsonm.value) list
53val pp : Format.formatter -> t -> unit
54val of_json : Jsonm.value -> t option
55```
56
57### Message Types (`Zulip.Message_type`)
58```ocaml
59type t = [ `Direct | `Channel ]
60
61val to_string : t -> string
62val of_string : string -> t option
63val pp : Format.formatter -> t -> unit
64```
65
66### Message (`Zulip.Message`)
67```ocaml
68type t (* abstract *)
69
70val create :
71 type_:Message_type.t ->
72 to_:string list ->
73 content:string ->
74 ?topic:string ->
75 ?queue_id:string ->
76 ?local_id:string ->
77 ?read_by_sender:bool ->
78 unit -> t
79
80val type_ : t -> Message_type.t
81val to_ : t -> string list
82val content : t -> string
83val topic : t -> string option
84val queue_id : t -> string option
85val local_id : t -> string option
86val read_by_sender : t -> bool
87val to_json : t -> Jsonm.value
88val pp : Format.formatter -> t -> unit
89```
90
91### Message Response (`Zulip.Message_response`)
92```ocaml
93type t (* abstract *)
94
95val id : t -> int
96val automatic_new_visibility_policy : t -> string option
97val of_json : Jsonm.value -> (t, Error.t) result
98val pp : Format.formatter -> t -> unit
99```
100
101### Client (`Zulip.Client`)
102```ocaml
103type t (* abstract *)
104
105val create : #Eio.Env.t -> Auth.t -> t
106val with_client : #Eio.Env.t -> Auth.t -> (t -> 'a) -> 'a
107
108val request :
109 t ->
110 method_:[`GET | `POST | `PUT | `DELETE | `PATCH] ->
111 path:string ->
112 ?params:(string * string) list ->
113 ?body:string ->
114 unit ->
115 (Jsonm.value, Error.t) result
116```
117
118### Messages (`Zulip.Messages`)
119```ocaml
120val send : Client.t -> Message.t -> (Message_response.t, Error.t) result
121val edit : Client.t -> message_id:int -> ?content:string -> ?topic:string -> unit -> (unit, Error.t) result
122val delete : Client.t -> message_id:int -> (unit, Error.t) result
123val get : Client.t -> message_id:int -> (Jsonm.value, Error.t) result
124val get_messages :
125 Client.t ->
126 ?anchor:string ->
127 ?num_before:int ->
128 ?num_after:int ->
129 ?narrow:string list ->
130 unit ->
131 (Jsonm.value, Error.t) result
132```
133
134### Channel (`Zulip.Channel`)
135```ocaml
136type t (* abstract *)
137
138val create :
139 name:string ->
140 description:string ->
141 ?invite_only:bool ->
142 ?history_public_to_subscribers:bool ->
143 unit -> t
144
145val name : t -> string
146val description : t -> string
147val invite_only : t -> bool
148val history_public_to_subscribers : t -> bool
149val to_json : t -> Jsonm.value
150val of_json : Jsonm.value -> (t, Error.t) result
151val pp : Format.formatter -> t -> unit
152```
153
154### Channels (`Zulip.Channels`)
155```ocaml
156val create_channel : Client.t -> Channel.t -> (unit, Error.t) result
157val delete : Client.t -> name:string -> (unit, Error.t) result
158val list : Client.t -> (Channel.t list, Error.t) result
159val subscribe : Client.t -> channels:string list -> (unit, Error.t) result
160val unsubscribe : Client.t -> channels:string list -> (unit, Error.t) result
161```
162
163### User (`Zulip.User`)
164```ocaml
165type t (* abstract *)
166
167val create :
168 email:string ->
169 full_name:string ->
170 ?is_active:bool ->
171 ?is_admin:bool ->
172 ?is_bot:bool ->
173 unit -> t
174
175val email : t -> string
176val full_name : t -> string
177val is_active : t -> bool
178val is_admin : t -> bool
179val is_bot : t -> bool
180val to_json : t -> Jsonm.value
181val of_json : Jsonm.value -> (t, Error.t) result
182val pp : Format.formatter -> t -> unit
183```
184
185### Users (`Zulip.Users`)
186```ocaml
187val list : Client.t -> (User.t list, Error.t) result
188val get : Client.t -> email:string -> (User.t, Error.t) result
189val create_user : Client.t -> email:string -> full_name:string -> (unit, Error.t) result
190val deactivate : Client.t -> email:string -> (unit, Error.t) result
191```
192
193### Event Type (`Zulip.Event_type`)
194```ocaml
195type t =
196 | Message
197 | Subscription
198 | User_activity
199 | Other of string
200
201val to_string : t -> string
202val of_string : string -> t
203val pp : Format.formatter -> t -> unit
204```
205
206### Event (`Zulip.Event`)
207```ocaml
208type t (* abstract *)
209
210val id : t -> int
211val type_ : t -> Event_type.t
212val data : t -> Jsonm.value
213val of_json : Jsonm.value -> (t, Error.t) result
214val pp : Format.formatter -> t -> unit
215```
216
217### Event Queue (`Zulip.Event_queue`)
218```ocaml
219type t (* abstract *)
220
221val register :
222 Client.t ->
223 ?event_types:Event_type.t list ->
224 unit ->
225 (t, Error.t) result
226
227val id : t -> string
228val get_events : t -> Client.t -> ?last_event_id:int -> unit -> (Event.t list, Error.t) result
229val delete : t -> Client.t -> (unit, Error.t) result
230val pp : Format.formatter -> t -> unit
231```
232
233## EIO Bot Framework Extension
234
235Based on analysis of the Python bot framework at:
236- https://github.com/zulip/python-zulip-api/blob/main/zulip_bots/zulip_bots/lib.py
237- https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/server.py
238
239### Bot Handler (`Zulip.Bot`)
240```ocaml
241module Storage : sig
242 type t (* abstract *)
243
244 val create : Client.t -> t
245 val get : t -> key:string -> string option
246 val put : t -> key:string -> value:string -> unit
247 val contains : t -> key:string -> bool
248end
249
250module Identity : sig
251 type t (* abstract *)
252
253 val full_name : t -> string
254 val email : t -> string
255 val mention_name : t -> string
256end
257
258type handler = {
259 handle_message :
260 client:Client.t ->
261 message:Jsonm.value ->
262 response:(Message.t -> unit) ->
263 unit;
264
265 usage : unit -> string;
266 description : unit -> string;
267}
268
269type t (* abstract *)
270
271val create :
272 Client.t ->
273 handler:handler ->
274 ?storage:Storage.t ->
275 unit -> t
276
277val identity : t -> Identity.t
278val storage : t -> Storage.t
279val handle_message : t -> Jsonm.value -> unit
280val send_reply : t -> original_message:Jsonm.value -> content:string -> unit
281val send_message : t -> Message.t -> unit
282```
283
284### Bot Server (`Zulip.Bot_server`)
285```ocaml
286module Config : sig
287 type bot_config = {
288 email : string;
289 api_key : string;
290 token : string; (* webhook token *)
291 server_url : string;
292 module_name : string;
293 }
294
295 type t (* abstract *)
296
297 val create : bot_configs:bot_config list -> ?host:string -> ?port:int -> unit -> t
298 val from_file : string -> (t, Error.t) result
299 val from_env : string -> (t, Error.t) result
300 val host : t -> string
301 val port : t -> int
302 val bot_configs : t -> bot_config list
303end
304
305type t (* abstract *)
306
307val create : #Eio.Env.t -> Config.t -> (t, Error.t) result
308
309val run : t -> unit
310(* Starts the server using EIO structured concurrency *)
311
312val with_server : #Eio.Env.t -> Config.t -> (t -> 'a) -> ('a, Error.t) result
313(* Resource-safe server management *)
314```
315
316### Bot Registry (`Zulip.Bot_registry`)
317```ocaml
318type bot_module = {
319 name : string;
320 handler : Bot.handler;
321 create_instance : Client.t -> Bot.t;
322}
323
324type t (* abstract *)
325
326val create : unit -> t
327val register : t -> bot_module -> unit
328val get_handler : t -> email:string -> Bot.t option
329val list_bots : t -> string list
330
331(* Dynamic module loading *)
332val load_from_file : string -> (bot_module, Error.t) result
333val load_from_directory : string -> (bot_module list, Error.t) result
334```
335
336### Webhook Handler (`Zulip.Webhook`)
337```ocaml
338type webhook_event = {
339 bot_email : string;
340 token : string;
341 message : Jsonm.value;
342 trigger : [`Direct_message | `Mention];
343}
344
345type response = {
346 content : string option;
347 message_type : Message_type.t option;
348 to_ : string list option;
349 topic : string option;
350}
351
352val parse_webhook : string -> (webhook_event, Error.t) result
353val handle_webhook : Bot_registry.t -> webhook_event -> (response option, Error.t) result
354```
355
356### Structured Concurrency Design
357
358The EIO-based server uses structured concurrency to manage multiple bots safely:
359
360```ocaml
361(* Example server implementation using EIO *)
362let run_server env config =
363 let registry = Bot_registry.create () in
364
365 (* Load and register all configured bots concurrently *)
366 Eio.Switch.run @@ fun sw ->
367
368 (* Start each bot in its own fiber *)
369 List.iter (fun bot_config ->
370 Eio.Fiber.fork ~sw (fun () ->
371 let auth = Auth.create
372 ~server_url:bot_config.server_url
373 ~email:bot_config.email
374 ~api_key:bot_config.api_key in
375
376 Client.with_client env auth @@ fun client ->
377
378 (* Load bot module *)
379 match Bot_registry.load_from_file bot_config.module_name with
380 | Ok bot_module ->
381 let bot = bot_module.create_instance client in
382 Bot_registry.register registry bot_module;
383
384 (* Keep bot alive and handle events *)
385 Event_loop.run client bot
386 | Error e ->
387 Printf.eprintf "Failed to load bot %s: %s\n"
388 bot_config.email (Error.message e)
389 )
390 ) (Config.bot_configs config);
391
392 (* Start HTTP server for webhooks *)
393 let server_addr = `Tcp (Eio.Net.Ipaddr.V4.loopback, Config.port config) in
394 Eio.Net.run_server env#net server_addr ~on_error:raise @@ fun flow _addr ->
395
396 (* Handle each webhook request concurrently *)
397 Eio.Switch.run @@ fun req_sw ->
398 Eio.Fiber.fork ~sw:req_sw (fun () ->
399 handle_http_request registry flow
400 )
401```
402
403### Event Loop (`Zulip.Event_loop`)
404```ocaml
405type t (* abstract *)
406
407val create : Client.t -> Bot.t -> t
408
409val run : Client.t -> Bot.t -> unit
410(* Runs the event loop using real-time events API *)
411
412val run_webhook_mode : Client.t -> Bot.t -> unit
413(* Runs in webhook mode, waiting for HTTP callbacks *)
414
415(* For advanced use cases *)
416val with_event_loop :
417 Client.t ->
418 Bot.t ->
419 (Event_queue.t -> unit) ->
420 unit
421```
422
423## Key EIO Advantages
424
4251. **Structured Concurrency**: Each bot runs in its own fiber with proper cleanup
4262. **Resource Safety**: Automatic cleanup of connections, event queues, and HTTP servers
4273. **Backpressure**: Natural flow control through EIO's cooperative scheduling
4284. **Error Isolation**: Bot failures don't crash the entire server
4295. **Graceful Shutdown**: Structured teardown of all resources
430
431## Design Principles
432
4331. **Abstract Types**: Each major concept has its own module with abstract `type t`
4342. **Constructors**: Clear `create` functions with optional parameters
4353. **Accessors**: All fields accessible via dedicated functions
4364. **Pretty Printing**: Every type has a `pp` function for debugging
4375. **JSON Conversion**: Bidirectional JSON conversion where appropriate
4386. **Error Handling**: Consistent `(_, Error.t) result` return types
439
440## Authentication Strategy
441
442- Support zuliprc files and direct credential passing
443- Abstract `Auth.t` prevents credential leakage
444- HTTP Basic Auth with proper encoding
445
446## EIO Integration
447
448- All operations use EIO's direct-style async
449- Resource-safe client management with `with_client`
450- Proper cleanup of connections and event queues
451
452## Example Usage
453
454### Simple Message Sending
455```ocaml
456let () =
457 Eio_main.run @@ fun env ->
458 let auth = Zulip.Auth.create
459 ~server_url:"https://example.zulipchat.com"
460 ~email:"bot@example.com"
461 ~api_key:"your-api-key" in
462
463 Zulip.Client.with_client env auth @@ fun client ->
464
465 let message = Zulip.Message.create
466 ~type_:`Channel
467 ~to_:["general"]
468 ~content:"Hello from OCaml!"
469 ~topic:"Bots"
470 () in
471
472 match Zulip.Messages.send client message with
473 | Ok response ->
474 Printf.printf "Message sent with ID: %d\n"
475 (Zulip.Message_response.id response)
476 | Error error ->
477 Printf.printf "Error: %s\n"
478 (Zulip.Error.message error)
479```
480
481### Simple Bot
482```ocaml
483let echo_handler = Zulip.Bot.{
484 handle_message = (fun ~client ~message ~response ->
485 let content = extract_content message in
486 let echo_msg = Message.create
487 ~type_:`Direct
488 ~to_:[sender_email message]
489 ~content:("Echo: " ^ content) () in
490 response echo_msg
491 );
492 usage = (fun () -> "Echo bot - repeats your message");
493 description = (fun () -> "A simple echo bot");
494}
495
496let () =
497 Eio_main.run @@ fun env ->
498 let auth = Auth.from_zuliprc () |> Result.get_ok in
499
500 Client.with_client env auth @@ fun client ->
501 let bot = Bot.create client ~handler:echo_handler () in
502 Event_loop.run client bot
503```
504
505### Multi-Bot Server
506```ocaml
507let () =
508 Eio_main.run @@ fun env ->
509 let config = Bot_server.Config.from_file "bots.conf" |> Result.get_ok in
510
511 Bot_server.with_server env config @@ fun server ->
512 Bot_server.run server
513```
514
515## Package Dependencies
516
517- `eio` - Effects-based I/O
518- `cohttp-eio` - HTTP client with EIO support
519- `tls-eio` - TLS support for HTTPS
520- `jsonm` - Streaming JSON codec
521- `uri` - URI parsing and manipulation
522- `base64` - Base64 encoding for authentication
523
524# Architecture Analysis: zulip_bot vs zulip_botserver
525
526## Library Separation
527
528### `zulip_bot` - Individual Bot Framework
529**Purpose**: Library for building and running a single bot instance
530
531**Key Components**:
532- `Bot_handler` - Interface for bot logic with EIO environment access
533- `Bot_runner` - Manages lifecycle of one bot (real-time events or webhook mode)
534- `Bot_config` - Configuration for a single bot
535- `Bot_storage` - Simple in-memory storage for bot state
536
537**Usage Pattern**:
538```ocaml
539(* Run a single bot directly *)
540let my_bot = Bot_handler.create (module My_echo_bot) ~config ~storage ~identity in
541let runner = Bot_runner.create ~client ~handler:my_bot in
542Bot_runner.run_realtime runner (* Bot connects to Zulip events API directly *)
543```
544
545### `zulip_botserver` - Multi-Bot Server Infrastructure
546**Purpose**: HTTP server that manages multiple bots via webhooks
547
548**Key Components**:
549- `Bot_server` - HTTP server receiving webhook events from Zulip
550- `Bot_registry` - Manages multiple bot instances
551- `Server_config` - Configuration for multiple bots + server settings
552- `Webhook_handler` - Parses incoming webhook requests and routes to appropriate bots
553
554**Usage Pattern**:
555```ocaml
556(* Run a server hosting multiple bots *)
557let registry = Bot_registry.create () in
558Bot_registry.register registry echo_bot_module;
559Bot_registry.register registry weather_bot_module;
560
561let server = Bot_server.create ~env ~config ~registry in
562Bot_server.run server (* HTTP server waits for webhook calls *)
563```
564
565## EIO Environment Requirements
566
567### Why Bot Handlers Need Direct EIO Access
568
569Bot handlers require direct access to the EIO environment for legitimate I/O operations beyond HTTP requests to Zulip:
570
5711. **Network Operations**: Custom HTTP requests, API calls to external services
5722. **File System Operations**: Reading configuration files, CSV dictionaries, logs
5733. **Resource Management**: Proper cleanup via structured concurrency
574
575### Example: URL Checker Bot
576```ocaml
577module Url_checker_bot : Zulip_bot.Bot_handler.Bot_handler = struct
578 let handle_message ~config ~storage ~identity ~message ~env =
579 match parse_command message with
580 | "!check", url ->
581 (* Direct EIO network access needed *)
582 Eio.Switch.run @@ fun sw ->
583 let client = Cohttp_eio.Client.make ~sw env#net in
584 let response = Cohttp_eio.Client.head ~sw client (Uri.of_string url) in
585 let status = Cohttp.Code.code_of_status response.status in
586 Ok (Response.reply ~content:(format_status_message url status))
587 | _ -> Ok Response.none
588end
589```
590
591### Example: CSV Dictionary Bot
592```ocaml
593module Csv_dict_bot : Zulip_bot.Bot_handler.Bot_handler = struct
594 let handle_message ~config ~storage ~identity ~message ~env =
595 match parse_command message with
596 | "!lookup", term ->
597 (* Direct EIO file system access needed *)
598 let csv_path = Bot_config.get_required config ~key:"csv_file" in
599 let content = Eio.Path.load env#fs (Eio.Path.parse csv_path) in
600 let matches = search_csv_content content term in
601 Ok (Response.reply ~content:(format_matches matches))
602 | _ -> Ok Response.none
603end
604```
605
606## Refined Bot Handler Interface
607
608Based on analysis, the current EIO environment plumbing is **essential** and should be cleaned up:
609
610```ocaml
611(** Clean bot handler interface with direct EIO access *)
612module type Bot_handler = sig
613 val initialize : Bot_config.t -> (unit, Zulip.Error.t) result
614 val usage : unit -> string
615 val description : unit -> string
616
617 (** Handle message with full EIO environment access *)
618 val handle_message :
619 config:Bot_config.t ->
620 storage:Bot_storage.t ->
621 identity:Identity.t ->
622 message:Message_context.t ->
623 env:#Eio.Env.t -> (* Essential for custom I/O *)
624 (Response.t, Zulip.Error.t) result
625end
626
627type t
628
629(** Single creation interface *)
630val create :
631 (module Bot_handler) ->
632 config:Bot_config.t ->
633 storage:Bot_storage.t ->
634 identity:Identity.t ->
635 t
636
637(** Single message handler requiring EIO environment *)
638val handle_message : t -> #Eio.Env.t -> Message_context.t -> (Response.t, Zulip.Error.t) result
639```
640
641## Storage Strategy
642
643Bot storage can be simplified to in-memory key-value storage since it's server-side:
644
645```ocaml
646(* In zulip_bot - storage per bot instance *)
647module Bot_storage = struct
648 type t = (string, string) Hashtbl.t (* Simple in-memory key-value *)
649
650 let create () = Hashtbl.create 16
651 let get t ~key = Hashtbl.find_opt t key
652 let put t ~key ~value = Hashtbl.replace t key value
653 let contains t ~key = Hashtbl.mem t key
654end
655
656(* In zulip_botserver - storage shared across bots *)
657module Server_storage = struct
658 type t = (string * string, string) Hashtbl.t (* (bot_email, key) -> value *)
659
660 let create () = Hashtbl.create 64
661 let get t ~bot_email ~key = Hashtbl.find_opt t (bot_email, key)
662 let put t ~bot_email ~key ~value = Hashtbl.replace t (bot_email, key) value
663end
664```
665
666## Interface Cleanup Recommendations
667
6681. **Remove** the problematic `handle_message` function with mock environment
6692. **Keep** `handle_message_with_env` but rename to `handle_message`
6703. **Use** `#Eio.Env.t` constraint for clean typing
6714. **Document** that bot handlers have full EIO access for custom I/O operations
672
673This design maintains flexibility for real-world bot functionality while providing clean, type-safe interfaces.
674
675## Sources and References
676
677This design is based on comprehensive analysis of:
678
6791. **Zulip REST API Documentation**:
680 - Main API: https://zulip.com/api/rest
681 - Error Handling: https://zulip.com/api/rest-error-handling
682 - Send Message: https://zulip.com/api/send-message
683
6842. **Python Zulip Library**:
685 - Main repository: https://github.com/zulip/python-zulip-api
686 - Bot framework: https://github.com/zulip/python-zulip-api/blob/main/zulip_bots/zulip_bots/lib.py
687 - Bot server: https://github.com/zulip/python-zulip-api/blob/main/zulip_botserver/zulip_botserver/server.py
688
689The design adapts these Python patterns to idiomatic OCaml with abstract types, proper error handling, and EIO's structured concurrency for robust, type-safe Zulip integration.