anil.recoil.org / slop
My agentic slop goes here. Not intended for anyone else!
fork atom
slop / jmap / DESIGN.md
at main 16 kB view raw view code

JMAP OCaml Implementation Design#

Type System Architecture with GADTs#

Core Design Principles#

  1. Type Safety: Use GADTs to ensure compile-time type safety between method calls and responses
  2. No Generic Names: Each module named after its purpose (Jmap_invocation, Jmap_session, etc.)
  3. Abstract Types: Each module exposes type t with submodules for related functionality
  4. JSON Library: Use only jsonm or ezjsonm (no yojson)
  5. Error Handling: Custom exception types via Jmap_error module

Module Structure#

jmap/
├── jmap-core/           (Core JMAP protocol - RFC 8620)
│   ├── jmap_error.ml    (Exception types and error handling)
│   ├── jmap_id.ml       (Abstract Id type)
│   ├── jmap_primitives.ml (Int, UnsignedInt, Date, UTCDate)
│   ├── jmap_capability.ml (Capability URN handling)
│   ├── jmap_invocation.ml (GADT-based invocation types)
│   ├── jmap_request.ml  (Request type with abstract t)
│   ├── jmap_response.ml (Response type with abstract t)
│   ├── jmap_session.ml  (Session and Account types)
│   ├── jmap_filter.ml   (FilterOperator and FilterCondition)
│   ├── jmap_comparator.ml (Sort comparators)
│   ├── jmap_standard_methods.ml (Standard method types)
│   ├── jmap_push.ml     (Push notification types)
│   ├── jmap_binary.ml   (Binary data operations)
│   └── jmap_parser.ml   (jsonm-based parsers)
│
├── jmap-mail/           (JMAP Mail extension - RFC 8621)
│   ├── jmap_mailbox.ml  (Mailbox type and methods)
│   ├── jmap_thread.ml   (Thread type and methods)
│   ├── jmap_email.ml    (Email type and methods)
│   ├── jmap_identity.ml (Identity type and methods)
│   ├── jmap_email_submission.ml (EmailSubmission type and methods)
│   ├── jmap_vacation_response.ml (VacationResponse type and methods)
│   ├── jmap_search_snippet.ml (SearchSnippet type and methods)
│   └── jmap_mail_parser.ml (Mail-specific parsers)
│
├── jmap-client/         (Client utilities)
│   ├── jmap_client.ml   (HTTP client with abstract t)
│   └── jmap_connection.ml (Connection management)
│
└── test/
    ├── data/            (JSON test files)
    │   ├── core/
    │   │   ├── request_echo.json
    │   │   ├── response_echo.json
    │   │   ├── request_get.json
    │   │   ├── response_get.json
    │   │   ├── error_unknownMethod.json
    │   │   └── ...
    │   └── mail/
    │       ├── mailbox_get_request.json
    │       ├── mailbox_get_response.json
    │       ├── email_get_request.json
    │       └── ...
    └── test_jmap.ml     (Alcotest test suite)

GADT Design for Type-Safe Method Calls#

The core idea is to use GADTs to pair method names with their request/response types:

(* jmap_invocation.ml *)

(* Method witness type - encodes method name and argument/response types *)
type (_, _) method_type =
  | Echo : (echo_args, echo_args) method_type
  | Get : 'a get_request -> ('a get_request, 'a get_response) method_type
  | Changes : 'a changes_request -> ('a changes_request, 'a changes_response) method_type
  | Set : 'a set_request -> ('a set_request, 'a set_response) method_type
  | Copy : 'a copy_request -> ('a copy_request, 'a copy_response) method_type
  | Query : 'a query_request -> ('a query_request, 'a query_response) method_type
  | QueryChanges : 'a query_changes_request -> ('a query_changes_request, 'a query_changes_response) method_type

(* Type-safe invocation *)
type 'resp invocation = {
  method_name : string;
  arguments : 'args;
  call_id : string;
  method_type : ('args, 'resp) method_type;
}

(* Heterogeneous list of invocations *)
type invocation_list =
  | [] : invocation_list
  | (::) : 'resp invocation * invocation_list -> invocation_list

Error Hierarchy#

(* jmap_error.ml *)

(* Error classification *)
type error_level =
  | Request_level    (* HTTP 4xx/5xx errors *)
  | Method_level     (* Method execution errors *)
  | Set_level        (* Object-level errors in /set operations *)

(* Request-level errors (RFC 8620 Section 3.6.1) *)
type request_error =
  | Unknown_capability of string
  | Not_json
  | Not_request
  | Limit of string  (* limit property name *)

(* Method-level errors (RFC 8620 Section 3.6.2) *)
type method_error =
  | Server_unavailable
  | Server_fail of string option
  | Server_partial_fail
  | Unknown_method
  | Invalid_arguments of string option
  | Invalid_result_reference
  | Forbidden
  | Account_not_found
  | Account_not_supported_by_method
  | Account_read_only
  (* Standard method errors *)
  | Request_too_large
  | State_mismatch
  | Cannot_calculate_changes
  | Anchor_not_found
  | Unsupported_sort
  | Unsupported_filter
  | Too_many_changes
  (* /copy specific *)
  | From_account_not_found
  | From_account_not_supported_by_method

(* Set-level errors (RFC 8620 Section 5.3) *)
type set_error =
  | Forbidden
  | Over_quota
  | Too_large
  | Rate_limit
  | Not_found
  | Invalid_patch
  | Will_destroy
  | Invalid_properties of string list option
  | Singleton
  | Already_exists of string option  (* existingId *)
  (* Mail-specific set errors *)
  | Mailbox_has_child
  | Mailbox_has_email
  | Blob_not_found of string list option  (* notFound blob ids *)
  | Too_many_keywords
  | Too_many_mailboxes
  | Invalid_email
  | Too_many_recipients of int option  (* maxRecipients *)
  | No_recipients
  | Invalid_recipients of string list option
  | Forbidden_mail_from
  | Forbidden_from
  | Forbidden_to_send of string option  (* description *)
  | Cannot_unsend

(* Main exception type *)
exception Jmap_error of error_level * string * string option

(* Helper constructors *)
val request_error : request_error -> exn
val method_error : method_error -> exn
val set_error : set_error -> exn

(* Parsing error *)
exception Parse_error of string

Primitive Types#

(* jmap_id.ml *)
module Id : sig
  type t
  val of_string : string -> t
  val to_string : t -> string
  val of_json : Ezjsonm.value -> t
  val to_json : t -> Ezjsonm.value
end

(* jmap_primitives.ml *)
module Int53 : sig
  type t
  val of_int : int -> t
  val to_int : t -> int
  val of_json : Ezjsonm.value -> t
end

module UnsignedInt : sig
  type t
  val of_int : int -> t
  val to_int : t -> int
  val of_json : Ezjsonm.value -> t
end

module Date : sig
  type t
  val of_string : string -> t
  val to_string : t -> string
  val of_json : Ezjsonm.value -> t
end

module UTCDate : sig
  type t
  val of_string : string -> t
  val to_string : t -> string
  val of_json : Ezjsonm.value -> t
end

Core Protocol Types#

(* jmap_request.ml *)
type t = {
  using : Jmap_capability.t list;
  method_calls : Jmap_invocation.invocation_list;
  created_ids : (Jmap_id.t * Jmap_id.t) list option;
}

module Parser : sig
  val of_json : Ezjsonm.value -> t
  val of_string : string -> t
  val of_channel : in_channel -> t
end

(* jmap_response.ml *)
type t = {
  method_responses : Jmap_invocation.response_list;
  created_ids : (Jmap_id.t * Jmap_id.t) list option;
  session_state : string;
}

module Parser : sig
  val of_json : Ezjsonm.value -> t
  val of_string : string -> t
  val of_channel : in_channel -> t
end

Standard Method Types#

(* jmap_standard_methods.ml *)

(* Polymorphic over object type 'a *)

module Get : sig
  type 'a request = {
    account_id : Jmap_id.t;
    ids : Jmap_id.t list option;
    properties : string list option;
  }

  type 'a response = {
    account_id : Jmap_id.t;
    state : string;
    list : 'a list;
    not_found : Jmap_id.t list;
  }
end

module Changes : sig
  type 'a request = {
    account_id : Jmap_id.t;
    since_state : string;
    max_changes : UnsignedInt.t option;
  }

  type 'a response = {
    account_id : Jmap_id.t;
    old_state : string;
    new_state : string;
    has_more_changes : bool;
    created : Jmap_id.t list;
    updated : Jmap_id.t list;
    destroyed : Jmap_id.t list;
  }
end

module Set : sig
  (* PatchObject type *)
  type patch_object = (string * Ezjsonm.value option) list

  type 'a request = {
    account_id : Jmap_id.t;
    if_in_state : string option;
    create : (Jmap_id.t * 'a) list option;
    update : (Jmap_id.t * patch_object) list option;
    destroy : Jmap_id.t list option;
  }

  type 'a response = {
    account_id : Jmap_id.t;
    old_state : string option;
    new_state : string;
    created : (Jmap_id.t * 'a) list option;
    updated : (Jmap_id.t * 'a option) list option;
    destroyed : Jmap_id.t list option;
    not_created : (Jmap_id.t * Jmap_error.set_error_detail) list option;
    not_updated : (Jmap_id.t * Jmap_error.set_error_detail) list option;
    not_destroyed : (Jmap_id.t * Jmap_error.set_error_detail) list option;
  }
end

(* Similar for Copy, Query, QueryChanges *)

Mail-Specific Types#

(* jmap_mailbox.ml *)
type t = {
  id : Jmap_id.t;
  name : string;
  parent_id : Jmap_id.t option;
  role : string option;
  sort_order : UnsignedInt.t;
  total_emails : UnsignedInt.t;
  unread_emails : UnsignedInt.t;
  total_threads : UnsignedInt.t;
  unread_threads : UnsignedInt.t;
  my_rights : Rights.t;
  is_subscribed : bool;
}

module Rights : sig
  type t = {
    may_read_items : bool;
    may_add_items : bool;
    may_remove_items : bool;
    may_set_seen : bool;
    may_set_keywords : bool;
    may_create_child : bool;
    may_rename : bool;
    may_delete : bool;
    may_submit : bool;
  }
end

module Get : sig
  type request = t Jmap_standard_methods.Get.request
  type response = t Jmap_standard_methods.Get.response
end

module Query : sig
  type filter = {
    parent_id : Jmap_id.t option;
    name : string option;
    role : string option;
    has_any_role : bool option;
    is_subscribed : bool option;
  }

  type request = {
    (* Standard query fields *)
    account_id : Jmap_id.t;
    filter : Jmap_filter.t option;
    sort : Jmap_comparator.t list option;
    position : int option;
    anchor : Jmap_id.t option;
    anchor_offset : int option;
    limit : UnsignedInt.t option;
    calculate_total : bool option;
    (* Mailbox-specific *)
    sort_as_tree : bool option;
    filter_as_tree : bool option;
  }

  type response = filter Jmap_standard_methods.Query.response
end

module Parser : sig
  val of_json : Ezjsonm.value -> t
end

(* jmap_email.ml *)
type t = {
  (* Metadata *)
  id : Jmap_id.t;
  blob_id : Jmap_id.t;
  thread_id : Jmap_id.t;
  mailbox_ids : Jmap_id.t list;
  keywords : string list;
  size : UnsignedInt.t;
  received_at : UTCDate.t;

  (* Header fields *)
  message_id : string list option;
  in_reply_to : string list option;
  references : string list option;
  sender : Email_address.t list option;
  from : Email_address.t list option;
  to_ : Email_address.t list option;
  cc : Email_address.t list option;
  bcc : Email_address.t list option;
  reply_to : Email_address.t list option;
  subject : string option;
  sent_at : Date.t option;

  (* Body *)
  body_structure : Body_part.t option;
  body_values : (string * Body_value.t) list option;
  text_body : Body_part.t list option;
  html_body : Body_part.t list option;
  attachments : Body_part.t list option;
  has_attachment : bool;
  preview : string;
}

module Email_address : sig
  type t = {
    name : string option;
    email : string;
  }
end

module Body_part : sig
  type t = {
    part_id : string option;
    blob_id : Jmap_id.t option;
    size : UnsignedInt.t;
    headers : (string * string) list;
    name : string option;
    type_ : string;
    charset : string option;
    disposition : string option;
    cid : string option;
    language : string list option;
    location : string option;
    sub_parts : t list option;
  }
end

module Body_value : sig
  type t = {
    value : string;
    is_encoding_problem : bool;
    is_truncated : bool;
  }
end

(* Similar structure for other mail types *)

Parser Structure#

(* jmap_parser.ml *)

module type PARSER = sig
  type t
  val of_json : Ezjsonm.value -> t
  val of_string : string -> t
  val of_channel : in_channel -> t
end

(* Helper functions for jsonm parsing *)
module Jsonm_helpers : sig
  val decode : Jsonm.decoder -> Ezjsonm.value
  val expect_object : Ezjsonm.value -> (string * Ezjsonm.value) list
  val expect_array : Ezjsonm.value -> Ezjsonm.value list
  val expect_string : Ezjsonm.value -> string
  val expect_int : Ezjsonm.value -> int
  val expect_bool : Ezjsonm.value -> bool
  val find_field : string -> (string * Ezjsonm.value) list -> Ezjsonm.value option
  val require_field : string -> (string * Ezjsonm.value) list -> Ezjsonm.value
end

(* Core parsers *)
val parse_invocation : Ezjsonm.value -> Jmap_invocation.invocation
val parse_request : Ezjsonm.value -> Jmap_request.t
val parse_response : Ezjsonm.value -> Jmap_response.t
val parse_session : Ezjsonm.value -> Jmap_session.t

Test Structure#

(* test/test_jmap.ml *)

let test_echo_request () =
  let json = load_json "test/data/core/request_echo.json" in
  let request = Jmap_parser.parse_request json in
  (* Assertions *)
  ()

let test_mailbox_get () =
  let json = load_json "test/data/mail/mailbox_get_request.json" in
  (* Parse and verify *)
  ()

let () =
  Alcotest.run "JMAP" [
    "core", [
      test_case "Echo request" `Quick test_echo_request;
      test_case "Get request" `Quick test_get_request;
      (* ... *)
    ];
    "mail", [
      test_case "Mailbox/get" `Quick test_mailbox_get;
      test_case "Email/get" `Quick test_email_get;
      (* ... *)
    ];
  ]

Implementation Strategy#

  1. Phase 1: Create project structure, error module, primitive types
  2. Phase 2: Implement core protocol types (Request, Response, Invocation)
  3. Phase 3: Implement standard methods (Get, Changes, Set, etc.)
  4. Phase 4: Implement mail-specific types and methods
  5. Phase 5: Generate comprehensive test JSON files
  6. Phase 6: Implement parsers with jsonm
  7. Phase 7: Write test suite with alcotest
  8. Phase 8: Complete parser implementations and documentation

JSON Test File Coverage#

Core Protocol (test/data/core/)#

  • request_echo.json, response_echo.json
  • request_get.json, response_get.json
  • request_changes.json, response_changes.json
  • request_set_create.json, response_set_create.json
  • request_set_update.json, response_set_update.json
  • request_set_destroy.json, response_set_destroy.json
  • request_copy.json, response_copy.json
  • request_query.json, response_query.json
  • request_query_changes.json, response_query_changes.json
  • error_unknownMethod.json
  • error_invalidArguments.json
  • error_stateMismatch.json
  • session.json
  • push_state_change.json
  • push_subscription.json

Mail Protocol (test/data/mail/)#

  • mailbox_get_request.json, mailbox_get_response.json
  • mailbox_query_request.json, mailbox_query_response.json
  • mailbox_set_request.json, mailbox_set_response.json
  • thread_get_request.json, thread_get_response.json
  • email_get_request.json, email_get_response.json
  • email_get_full_request.json, email_get_full_response.json
  • email_query_request.json, email_query_response.json
  • email_set_request.json, email_set_response.json
  • email_import_request.json, email_import_response.json
  • email_parse_request.json, email_parse_response.json
  • search_snippet_request.json, search_snippet_response.json
  • identity_get_request.json, identity_get_response.json
  • email_submission_get_request.json, email_submission_response.json
  • vacation_response_get_request.json, vacation_response_response.json

Total: ~40-50 JSON test files covering all message types