My agentic slop goes here. Not intended for anyone else!
anil.recoil.org
1# Parser Implementation Guide
2
3This guide will help you complete the JSON parser implementations throughout the JMAP codebase. All type definitions are complete - only the parsing logic needs to be filled in.
4
5## Overview
6
7**Status**: All `of_json` and `to_json` functions have stub implementations that raise "not yet implemented" errors.
8
9**Goal**: Implement these functions using the provided test JSON files as specifications.
10
11**Tools**: Use `Jmap_parser.Helpers` module for common parsing operations.
12
13## Implementation Strategy
14
15### Step 1: Start with Primitives (Easiest)
16
17These are already complete, but review them as examples:
18
19```ocaml
20(* jmap-core/jmap_id.ml - COMPLETE *)
21let of_json json =
22 match json with
23 | `String s -> of_string s
24 | _ -> raise (Jmap_error.Parse_error "Id must be a JSON string")
25
26(* jmap-core/jmap_primitives.ml - COMPLETE *)
27module UnsignedInt = struct
28 let of_json = function
29 | `Float f -> of_int (int_of_float f)
30 | `Int i -> if i >= 0 then of_int i else raise (Parse_error "...")
31 | _ -> raise (Parse_error "Expected number")
32end
33```
34
35### Step 2: Implement Core Parsers
36
37#### 2.1 Comparator (Simple Object)
38
39**File**: `jmap-core/jmap_comparator.ml`
40**Test**: `test/data/core/request_query.json` (sort field)
41
42```ocaml
43let of_json json =
44 let open Jmap_parser.Helpers in
45 let fields = expect_object json in
46 let property = get_string "property" fields in
47 let is_ascending = get_bool_opt "isAscending" fields true in
48 let collation = get_string_opt "collation" fields in
49 { property; is_ascending; collation }
50```
51
52#### 2.2 Filter (Recursive Type)
53
54**File**: `jmap-core/jmap_filter.ml`
55**Test**: `test/data/core/request_query.json` (filter field)
56
57The generic `of_json` function is already complete. You need to implement condition parsers for each type (Mailbox, Email, etc.).
58
59#### 2.3 Session (Complex Nested Object)
60
61**File**: `jmap-core/jmap_session.ml`
62**Test**: `test/data/core/session.json`
63
64```ocaml
65(* Account parser *)
66module Account = struct
67 let of_json json =
68 let open Jmap_parser.Helpers in
69 let fields = expect_object json in
70 {
71 name = get_string "name" fields;
72 is_personal = get_bool "isPersonal" fields;
73 is_read_only = get_bool "isReadOnly" fields;
74 account_capabilities = parse_map (fun v -> v)
75 (require_field "accountCapabilities" fields);
76 }
77end
78
79(* Session parser *)
80let of_json json =
81 let open Jmap_parser.Helpers in
82 let fields = expect_object json in
83 {
84 capabilities = parse_map (fun v -> v) (require_field "capabilities" fields);
85 accounts = parse_map Account.of_json (require_field "accounts" fields);
86 primary_accounts = parse_map expect_string (require_field "primaryAccounts" fields);
87 username = get_string "username" fields;
88 api_url = get_string "apiUrl" fields;
89 download_url = get_string "downloadUrl" fields;
90 upload_url = get_string "uploadUrl" fields;
91 event_source_url = get_string "eventSourceUrl" fields;
92 state = get_string "state" fields;
93 }
94```
95
96#### 2.4 Invocation (3-tuple Array)
97
98**File**: `jmap-core/jmap_invocation.ml`
99**Test**: Any request or response file (methodCalls/methodResponses field)
100
101```ocaml
102let of_json json =
103 let open Jmap_parser.Helpers in
104 match json with
105 | `A [method_name_json; arguments_json; call_id_json] ->
106 let method_name = expect_string method_name_json in
107 let call_id = expect_string call_id_json in
108
109 (* Parse based on method name *)
110 begin match witness_of_method_name method_name with
111 | Packed template ->
112 (* Parse arguments based on witness type *)
113 (* Return properly typed invocation *)
114 (* TODO: Complete this logic *)
115 raise (Parse_error "Invocation parsing not complete")
116 end
117
118 | _ -> raise (Parse_error "Invocation must be 3-element array")
119```
120
121#### 2.5 Request and Response
122
123**File**: `jmap-core/jmap_request.ml` and `jmap_response.ml`
124**Test**: All `test/data/core/request_*.json` and `response_*.json`
125
126```ocaml
127(* Request *)
128let of_json json =
129 let open Jmap_parser.Helpers in
130 let fields = expect_object json in
131 {
132 using = parse_array Jmap_capability.of_json
133 (require_field "using" fields);
134 method_calls = parse_array Jmap_invocation.of_json
135 (require_field "methodCalls" fields);
136 created_ids = match find_field "createdIds" fields with
137 | Some obj -> Some (parse_map Jmap_id.of_json obj)
138 | None -> None;
139 }
140
141(* Response - similar pattern *)
142```
143
144### Step 3: Implement Standard Method Parsers
145
146These follow predictable patterns. Example for Get:
147
148**File**: `jmap-core/jmap_standard_methods.ml`
149**Tests**: `test/data/core/request_get.json`, `response_get.json`
150
151```ocaml
152module Get = struct
153 let request_of_json parse_obj json =
154 let open Jmap_parser.Helpers in
155 let fields = expect_object json in
156 {
157 account_id = Jmap_id.of_json (require_field "accountId" fields);
158 ids = parse_array_opt Jmap_id.of_json (find_field "ids" fields);
159 properties = parse_array_opt expect_string (find_field "properties" fields);
160 }
161
162 let response_of_json parse_obj json =
163 let open Jmap_parser.Helpers in
164 let fields = expect_object json in
165 {
166 account_id = Jmap_id.of_json (require_field "accountId" fields);
167 state = get_string "state" fields;
168 list = parse_array parse_obj (require_field "list" fields);
169 not_found = parse_array Jmap_id.of_json (require_field "notFound" fields);
170 }
171end
172
173(* Repeat for Changes, Set, Copy, Query, QueryChanges *)
174```
175
176### Step 4: Implement Mail Type Parsers
177
178#### 4.1 Mailbox (Simple Mail Type)
179
180**File**: `jmap-mail/jmap_mailbox.ml`
181**Tests**: `test/data/mail/mailbox_get_response.json`
182
183```ocaml
184module Rights = struct
185 let of_json json =
186 let open Jmap_parser.Helpers in
187 let fields = expect_object json in
188 {
189 may_read_items = get_bool "mayReadItems" fields;
190 may_add_items = get_bool "mayAddItems" fields;
191 may_remove_items = get_bool "mayRemoveItems" fields;
192 may_set_seen = get_bool "maySetSeen" fields;
193 may_set_keywords = get_bool "maySetKeywords" fields;
194 may_create_child = get_bool "mayCreateChild" fields;
195 may_rename = get_bool "mayRename" fields;
196 may_delete = get_bool "mayDelete" fields;
197 may_submit = get_bool "maySubmit" fields;
198 }
199end
200
201let of_json json =
202 let open Jmap_parser.Helpers in
203 let fields = expect_object json in
204 {
205 id = Jmap_id.of_json (require_field "id" fields);
206 name = get_string "name" fields;
207 parent_id = Option.map Jmap_id.of_json (find_field "parentId" fields);
208 role = get_string_opt "role" fields;
209 sort_order = Jmap_primitives.UnsignedInt.of_json
210 (require_field "sortOrder" fields);
211 total_emails = Jmap_primitives.UnsignedInt.of_json
212 (require_field "totalEmails" fields);
213 unread_emails = Jmap_primitives.UnsignedInt.of_json
214 (require_field "unreadEmails" fields);
215 total_threads = Jmap_primitives.UnsignedInt.of_json
216 (require_field "totalThreads" fields);
217 unread_threads = Jmap_primitives.UnsignedInt.of_json
218 (require_field "unreadThreads" fields);
219 my_rights = Rights.of_json (require_field "myRights" fields);
220 is_subscribed = get_bool "isSubscribed" fields;
221 }
222```
223
224#### 4.2 Email (Most Complex)
225
226**File**: `jmap-mail/jmap_email.ml`
227**Tests**:
228- `test/data/mail/email_get_response.json` (basic)
229- `test/data/mail/email_get_full_response.json` (with body structure)
230
231Start with submodules:
232
233```ocaml
234module EmailAddress = struct
235 let of_json json =
236 let open Jmap_parser.Helpers in
237 let fields = expect_object json in
238 {
239 name = get_string_opt "name" fields;
240 email = get_string "email" fields;
241 }
242end
243
244module BodyPart = struct
245 (* Recursive parser for MIME structure *)
246 let rec of_json json =
247 let open Jmap_parser.Helpers in
248 let fields = expect_object json in
249 {
250 part_id = get_string_opt "partId" fields;
251 blob_id = Option.map Jmap_id.of_json (find_field "blobId" fields);
252 size = Jmap_primitives.UnsignedInt.of_json (require_field "size" fields);
253 headers = parse_array parse_header (require_field "headers" fields);
254 name = get_string_opt "name" fields;
255 type_ = get_string "type" fields;
256 charset = get_string_opt "charset" fields;
257 disposition = get_string_opt "disposition" fields;
258 cid = get_string_opt "cid" fields;
259 language = parse_array_opt expect_string (find_field "language" fields);
260 location = get_string_opt "location" fields;
261 sub_parts = parse_array_opt of_json (find_field "subParts" fields);
262 }
263
264 and parse_header json =
265 let open Jmap_parser.Helpers in
266 let fields = expect_object json in
267 (get_string "name" fields, get_string "value" fields)
268end
269
270(* Main Email parser *)
271let of_json json =
272 let open Jmap_parser.Helpers in
273 let fields = expect_object json in
274 {
275 (* Parse all 24 fields *)
276 id = Jmap_id.of_json (require_field "id" fields);
277 blob_id = Jmap_id.of_json (require_field "blobId" fields);
278 thread_id = Jmap_id.of_json (require_field "threadId" fields);
279 mailbox_ids = parse_map (fun _ -> true) (require_field "mailboxIds" fields);
280 keywords = parse_map (fun _ -> true) (require_field "keywords" fields);
281 (* ... continue for all fields ... *)
282 from = parse_array_opt EmailAddress.of_json (find_field "from" fields);
283 to_ = parse_array_opt EmailAddress.of_json (find_field "to" fields);
284 body_structure = Option.map BodyPart.of_json (find_field "bodyStructure" fields);
285 (* ... etc ... *)
286 }
287```
288
289### Step 5: Testing Pattern
290
291For each parser you implement:
292
293```ocaml
294(* In test/test_jmap.ml *)
295
296let test_mailbox_parse () =
297 (* Load test JSON *)
298 let json = load_json "test/data/mail/mailbox_get_response.json" in
299
300 (* Parse response *)
301 let response = Jmap_mail.Jmap_mailbox.Get.response_of_json
302 Jmap_mailbox.Parser.of_json json in
303
304 (* Validate *)
305 check int "Mailbox count" 3 (List.length response.list);
306
307 let inbox = List.hd response.list in
308 check string "Inbox name" "Inbox" inbox.name;
309 check (option string) "Inbox role" (Some "inbox") inbox.role;
310 check bool "Can read" true inbox.my_rights.may_read_items;
311
312let () =
313 run "JMAP" [
314 "Mailbox", [
315 test_case "Parse mailbox response" `Quick test_mailbox_parse;
316 ];
317 ]
318```
319
320## Common Patterns
321
322### Optional Fields
323
324```ocaml
325(* Option with default *)
326let is_ascending = get_bool_opt "isAscending" fields true
327
328(* Option without default *)
329let collation = get_string_opt "collation" fields
330
331(* Map with option *)
332parent_id = Option.map Jmap_id.of_json (find_field "parentId" fields)
333```
334
335### Arrays
336
337```ocaml
338(* Required array *)
339ids = parse_array Jmap_id.of_json (require_field "ids" fields)
340
341(* Optional array (null or array) *)
342properties = parse_array_opt expect_string (find_field "properties" fields)
343```
344
345### Maps (JSON Objects)
346
347```ocaml
348(* String -> value *)
349keywords = parse_map (fun v -> true) (require_field "keywords" fields)
350
351(* Id -> value *)
352mailbox_ids = parse_map Jmap_id.of_json (require_field "mailboxIds" fields)
353```
354
355### Recursive Types
356
357```ocaml
358(* Mutually recursive *)
359let rec parse_filter parse_condition json =
360 match json with
361 | `O fields ->
362 match List.assoc_opt "operator" fields with
363 | Some op -> (* FilterOperator *)
364 let conditions = parse_array (parse_filter parse_condition) ... in
365 Operator (op, conditions)
366 | None -> (* FilterCondition *)
367 Condition (parse_condition json)
368 | _ -> raise (Parse_error "...")
369```
370
371## Helper Reference
372
373```ocaml
374(* From Jmap_parser.Helpers *)
375
376(* Type expectations *)
377expect_object : json -> (string * json) list
378expect_array : json -> json list
379expect_string : json -> string
380expect_int : json -> int
381expect_bool : json -> bool
382
383(* Field access *)
384find_field : string -> fields -> json option
385require_field : string -> fields -> json
386
387(* Typed getters *)
388get_string : string -> fields -> string
389get_string_opt : string -> fields -> string option
390get_bool : string -> fields -> bool
391get_bool_opt : string -> fields -> bool -> bool (* with default *)
392get_int : string -> fields -> int
393get_int_opt : string -> fields -> int option
394
395(* Parsers *)
396parse_map : (json -> 'a) -> json -> (string * 'a) list
397parse_array : (json -> 'a) -> json -> 'a list
398parse_array_opt : (json -> 'a) -> json option -> 'a list option
399```
400
401## Order of Implementation
402
403Recommended order (easiest to hardest):
404
4051. ✅ Primitives (already done)
4062. `Jmap_comparator` - Simple object
4073. `Jmap_capability.CoreCapability` - Nested object
4084. `Jmap_session` - Complex nested object with maps
4095. `Jmap_standard_methods.Get.request` - Simple with optionals
4106. `Jmap_standard_methods.Get.response` - With generic list
4117. Other standard methods (Changes, Query, etc.)
4128. `Jmap_invocation` - Array tuple with GADT dispatch
4139. `Jmap_request` and `Jmap_response` - Top-level protocol
41410. `Jmap_mailbox` - Simplest mail type
41511. `Jmap_thread` - Very simple (2 fields)
41612. `Jmap_identity` - Medium complexity
41713. `Jmap_vacation_response` - Singleton pattern
41814. `Jmap_search_snippet` - Search results
41915. `Jmap_email_submission` - With enums and envelope
42016. `Jmap_email` - Most complex (save for last)
421
422## Validation Strategy
423
424For each parser:
425
4261. **Parse test file**: Ensure no exceptions
4272. **Check required fields**: Verify non-optional fields are present
4283. **Validate values**: Check actual values match test file
4294. **Round-trip**: Serialize and parse again, compare
4305. **Error cases**: Try malformed JSON, missing fields
431
432## Serialization (to_json)
433
434After parsing is complete, implement serialization:
435
436```ocaml
437let to_json t =
438 `O [
439 ("id", Jmap_id.to_json t.id);
440 ("name", `String t.name);
441 ("sortOrder", Jmap_primitives.UnsignedInt.to_json t.sort_order);
442 (* ... *)
443 ]
444```
445
446Remove fields that are None:
447
448```ocaml
449let fields = [
450 ("id", Jmap_id.to_json t.id);
451 ("name", `String t.name);
452] in
453let fields = match t.parent_id with
454 | Some pid -> ("parentId", Jmap_id.to_json pid) :: fields
455 | None -> fields
456in
457`O fields
458```
459
460## Common Pitfalls
461
4621. **Case sensitivity**: JSON field names are case-sensitive
463 - Use `"receivedAt"` not `"receivedat"`
464
4652. **Null vs absent**: Distinguish between `null` and field not present
466 ```ocaml
467 | Some `Null -> None (* null *)
468 | Some value -> Some (parse value) (* present *)
469 | None -> None (* absent *)
470 ```
471
4723. **Empty arrays**: `[]` is different from `null`
473 ```ocaml
474 parse_array_opt (* Returns None for null, Some [] for [] *)
475 ```
476
4774. **Number types**: JSON doesn't distinguish int/float
478 ```ocaml
479 | `Float f -> int_of_float f
480 | `Int i -> i
481 ```
482
4835. **Boolean maps**: Many fields are `Id[Boolean]`
484 ```ocaml
485 mailbox_ids = parse_map (fun _ -> true) field
486 ```
487
488## Getting Help
489
4901. **Check test files**: They contain the exact JSON structure
4912. **Look at existing parsers**: Id and primitives are complete
4923. **Use the helpers**: They handle most common cases
4934. **Follow the types**: Type errors will guide you
494
495## Success Criteria
496
497Parser implementation is complete when:
498
499- [ ] All test files parse without errors
500- [ ] All required fields are extracted
501- [ ] Optional fields handled correctly
502- [ ] Round-trip works (parse -> serialize -> parse)
503- [ ] All 50 test files pass
504- [ ] No TODO comments remain in parser code
505
506Good luck! Start simple and build up to the complex types. The type system will guide you.