anil.recoil.org / slop
My agentic slop goes here. Not intended for anyone else!
fork atom
slop / stack / zulip / ARCHITECTURE.md
at main 7.1 kB view raw view code

Zulip Library Architecture#

Overview#

The Zulip OCaml library follows a clean, layered architecture that separates protocol types, encoding concerns, and HTTP communication.

Architecture Layers#

┌─────────────────────────────────────┐
│   API Modules (Messages, Channels)  │  ← High-level operations
├─────────────────────────────────────┤
│   Protocol Types (Message, Channel) │  ← Business logic types with Jsont codecs
├─────────────────────────────────────┤
│   Encode Module                     │  ← JSON/Form encoding utilities
├─────────────────────────────────────┤
│   Client Module                     │  ← HTTP request/response handling
├─────────────────────────────────────┤
│   Requests Library (EIO-based)      │  ← Low-level HTTP
└─────────────────────────────────────┘

Key Design Principles#

1. Protocol Types with Jsont Codecs#

Each Zulip API type (Message, Channel, User, etc.) has:

  • A clean OCaml record type
  • A jsont codec that defines bidirectional JSON conversion
  • Accessor functions
  • Pretty printer

Example from channel.ml:

type t = {
  name : string;
  description : string;
  invite_only : bool;
  history_public_to_subscribers : bool;
}

let jsont =
  Jsont.Object.map ~kind:"Channel" make
  |> Jsont.Object.mem "name" Jsont.string ~enc:name
  |> Jsont.Object.mem "description" Jsont.string ~enc:description
  |> Jsont.Object.mem "invite_only" Jsont.bool ~enc:invite_only
  |> Jsont.Object.mem "history_public_to_subscribers" Jsont.bool ~enc:history_public_to_subscribers
  |> Jsont.Object.finish

2. Encode Module: Separation of Encoding Concerns#

The Encode module provides clean utilities for converting between OCaml types and wire formats:

(** Convert using a jsont codec *)
val to_json_string : 'a Jsont.t -> 'a -> string
val to_form_urlencoded : 'a Jsont.t -> 'a -> string
val from_json : 'a Jsont.t -> Jsont.json -> ('a, string) result

This eliminates the need for:

  • ❌ Manual JSON tree walking
  • ❌ Round-trip encode→decode conversions
  • ❌ Per-type encoding functions

3. Request/Response Types with Codecs#

API operations define request/response types locally with their codecs:

(* In channels.ml *)
module Subscribe_request = struct
  type t = { subscriptions : string list }

  let codec =
    Jsont.Object.(
      map ~kind:"SubscribeRequest" (fun subscriptions -> { subscriptions })
      |> mem "subscriptions" (Jsont.list Jsont.string) ~enc:(fun r -> r.subscriptions)
      |> finish
    )
end

let subscribe client ~channels =
  let req = Subscribe_request.{ subscriptions = channels } in
  let body = Encode.to_form_urlencoded Subscribe_request.codec req in
  let content_type = "application/x-www-form-urlencoded" in
  match Client.request client ~method_:`POST ~path:"/api/v1/users/me/subscriptions"
          ~body ~content_type () with
  | Ok _json -> Ok ()
  | Error err -> Error err

4. Type-Safe Decoding#

Response parsing uses codecs directly instead of manual pattern matching:

(* OLD - manual JSON walking *)
match json with
| Jsont.Object (fields, _) ->
    let assoc = List.map (fun ((k, _), v) -> (k, v)) fields in
    (match List.assoc_opt "streams" assoc with
     | Some (Jsont.Array (channel_list, _)) -> ...

(* NEW - type-safe codec *)
let response_codec =
  Jsont.Object.(
    map ~kind:"StreamsResponse" (fun streams -> streams)
    |> mem "streams" (Jsont.list Channel.jsont) ~enc:(fun x -> x)
    |> finish
  )
in
match Encode.from_json response_codec json with
| Ok channels -> Ok channels
| Error msg -> Error (...)

Benefits#

✅ Type Safety#

  • Jsont codecs ensure correct JSON structure
  • Compilation errors catch schema mismatches
  • No runtime type confusion

✅ Maintainability#

  • Protocol changes only require updating codecs
  • No manual JSON manipulation scattered through code
  • Clear separation of concerns

✅ Reusability#

  • Codecs can be composed and reused
  • Encode module works for any jsont-encoded type
  • Request/response types are self-documenting

✅ Testability#

  • Easy to test encoding/decoding in isolation
  • Mock responses can be type-checked
  • Round-trip property testing possible

Migration Pattern#

When adding new API endpoints:

  1. Define the protocol type with codec:

    type my_request = { field1: string; field2: int }

let my_request_codec =
  Jsont.Object.(
    map ~kind:"MyRequest" (fun field1 field2 -> { field1; field2 })
    |> mem "field1" Jsont.string ~enc:(fun r -> r.field1)
    |> mem "field2" Jsont.int ~enc:(fun r -> r.field2)
    |> finish
  )

  2. Encode using Encode module:

    let body = Encode.to_form_urlencoded my_request_codec req in
(* or *)
let json = Encode.to_json_string my_request_codec req in

  3. Decode responses with codec:

    match Client.request client ~method_:`POST ~path:"/api/..." ~body () with
| Ok json ->
    (match Encode.from_json response_codec json with
     | Ok data -> Ok data
     | Error msg -> Error ...)

Comparison with Old Approach#

Old (Manual JSON Manipulation):#

let send client message =
  let json = Message.to_json message in  (* Round-trip conversion *)
  let params = match json with
    | Jsont.Object (fields, _) ->  (* Manual pattern matching *)
        List.fold_left (fun acc ((key, _), value) ->
          let str_value = match value with  (* More pattern matching *)
            | Jsont.String (s, _) -> s
            | Jsont.Bool (true, _) -> "true"
            | _ -> ""
          in
          (key, str_value) :: acc
        ) [] fields
    | _ -> [] in
  (* ... *)

New (Codec-Based):#

let send client message =
  let body = Message.to_form_urlencoded message in  (* Clean encoding *)
  let content_type = "application/x-www-form-urlencoded" in
  match Client.request client ~method_:`POST ~path:"/api/v1/messages"
          ~body ~content_type () with
  | Ok response -> Message_response.of_json response
  | Error err -> Error err

Future Enhancements#

  • Validation: Add validation layers on top of codecs
  • Versioning: Support multiple API versions with codec variants
  • Documentation: Generate API docs from codec definitions
  • Testing: Property-based testing with codec round-trips
  • Code Generation: Consider generating codecs from OpenAPI specs

References#