Summary of OxCaml changes for llms.txt
anil.recoil.org
1# OxCaml Language Model Context
2
3OxCaml is Jane Street's enhanced version of OCaml that adds numerous performance and safety features while maintaining OCaml compatibility. This document provides comprehensive guidance for code generation and porting OCaml code to OxCaml.
4
5## Core OxCaml Philosophy
6
7OxCaml extends OCaml with:
8- **Zero-allocation programming**: Stack allocation, unboxed types, comprehensive optimization
9- **Parallelism with safety**: Data-race-free parallel programming through modes
10- **Fine-grained control**: Modes, kinds, uniqueness for memory and performance optimization
11- **Enhanced expressiveness**: Comprehensions, SIMD, templates for better abstractions
12
13## Quick Reference: Key Extensions
14
15### Stack Allocation (`local` mode)
16- Use `@ local` for values that don't escape scope
17- Enables stack allocation, reducing GC pressure
18- Use `exclave_` to return local values from functions
19- `stack_` forces stack allocation (fails if impossible)
20
21### Unboxed Types (layouts)
22- `float#`, `int32#`, `int64#` for unboxed numeric types
23- Literals: `#3.14` (float#), `#123l` (int32#), `#456L` (int64#)
24- Unboxed records: `#{ field = value }`
25- Unboxed tuples: `#(a, b, c)`
26
27### Parallelism
28- Use `Parallel.fork_join2` for parallel computation
29- Mark data `@ portable` for cross-domain sharing
30- Use `@ contended` for thread-safe mutable data
31- Capsules for complex shared mutable state
32
33### Comprehensions
34- List: `[ expr for pat in seq when cond ]`
35- Array: `[| expr for pat in seq when cond |]`
36- Range iteration: `for i = 1 to n`
37- Parallel iteration: `for x in xs and y in ys`
38
39### SIMD
40- Vector types: `float32x4#`, `int32x4#`, etc.
41- Load from memory: `Float32x4.String.get`
42- Operations: `Float32x4.add`, `Float32x4.mul`
43- Use `[@unboxed]` for C interop
44
45## Mode System Reference
46
47### Locality Axis (scope)
48- `global` (default): May escape scope
49- `local`: Cannot escape scope, enables stack allocation
50
51### Contention Axis (thread safety)
52- `uncontended` (default): Single thread access
53- `shared`: Multiple threads read-only
54- `contended`: Multiple threads with writes
55
56### Portability Axis (thread movement)
57- `nonportable` (default): Cannot move between threads
58- `portable`: May move across thread boundaries
59
60### Uniqueness Axis (aliasing)
61- `aliased` (default): Multiple references
62- `unique`: Single reference only
63
64### Other Axes
65- Linearity: `many` (default) vs `once` (single use)
66- Yielding: `unyielding` (default) vs `yielding` (effects)
67- Visibility: `read_write` (default) vs `read` vs `immutable`
68- Statefulness: `stateful` (default) vs `observing` vs `stateless`
69
70## Kind System Reference
71
72### Layouts
73- `value`: Standard OCaml values
74- `immediate`: Int-like types
75- `float64`, `float32`: Unboxed floats
76- `bits64`, `bits32`: Unboxed integers
77- `word`: Machine word
78- `vec128`, `vec256`: SIMD vectors
79- `any`: Maximum layout
80
81### Kind Abbreviations
82- `immediate`: Values like `int`, `bool`
83- `immutable_data`: Plain immutable data
84- `mutable_data`: Plain mutable data
85- `any_non_null`: Any layout excluding NULL
86
87## Migration Guidelines: OCaml to OxCaml
88
89### 1. Performance Optimization
90
91**Stack Allocation:**
92```ocaml
93(* Before *)
94let process data =
95 let temp = expensive_computation data in
96 extract_result temp
97
98(* After *)
99let process data =
100 let temp @ local = expensive_computation data in
101 extract_result temp
102```
103
104**Unboxed Numeric Code:**
105```ocaml
106(* Before *)
107let distance x1 y1 x2 y2 =
108 sqrt ((x2 -. x1) ** 2.0 +. (y2 -. y1) ** 2.0)
109
110(* After *)
111let distance x1 y1 x2 y2 =
112 Float_u.sqrt (Float_u.add
113 (Float_u.pow (Float_u.sub x2 x1) #2.0)
114 (Float_u.pow (Float_u.sub y2 y1) #2.0))
115```
116
117**Array Processing with SIMD:**
118```ocaml
119(* Before *)
120let add_arrays a b = Array.map2 (+.) a b
121
122(* After *)
123module F32x4 = Ocaml_simd_sse.Float32x4
124let add_vectors v1 v2 = F32x4.add v1 v2
125```
126
127### 2. Parallel Programming
128
129**Basic Parallelism:**
130```ocaml
131(* Before *)
132let map_reduce f reduce_f init list =
133 List.fold_left (fun acc x -> reduce_f acc (f x)) init list
134
135(* After *)
136let parallel_map_reduce par f reduce_f init list =
137 let process chunk =
138 List.fold_left (fun acc x -> reduce_f acc (f x)) init chunk
139 in
140 match list with
141 | [] -> init
142 | _ ->
143 let mid = List.length list / 2 in
144 let left, right = List.split_at mid list in
145 let left_result, right_result =
146 Parallel.fork_join2 par
147 (fun _ -> process left)
148 (fun _ -> process right)
149 in
150 reduce_f left_result right_result
151```
152
153**Shared Mutable State:**
154```ocaml
155(* Before *)
156let shared_counter = ref 0
157let increment () = incr shared_counter
158
159(* After *)
160let shared_counter = Atomic.make 0
161let increment () = Atomic.incr shared_counter
162```
163
164### 3. Loop Transformation
165
166**Comprehensions:**
167```ocaml
168(* Before *)
169let result = ref [] in
170for i = 1 to n do
171 for j = 1 to m do
172 if condition i j then
173 result := (f i j) :: !result
174 done
175done;
176List.rev !result
177
178(* After *)
179[ f i j for i = 1 to n for j = 1 to m when condition i j ]
180```
181
182**Filter + Map:**
183```ocaml
184(* Before *)
185list |> List.filter predicate |> List.map transform
186
187(* After *)
188[ transform x for x in list when predicate x ]
189```
190
191### 4. Memory Management
192
193**Unique Resources:**
194```ocaml
195(* Define safe resource management *)
196type buffer : value
197
198val create : size:int -> buffer @ unique
199val free : buffer @ unique -> unit
200val write : buffer @ unique -> string -> buffer @ unique
201
202(* Use with functional threading *)
203let process_data size data =
204 create ~size
205 |> write data
206 |> write " processed"
207 |> fun buf -> let result = extract buf in free buf; result
208```
209
210**Immutable Arrays:**
211```ocaml
212(* Before *)
213let readonly_data = [| 1; 2; 3; 4 |]
214
215(* After *)
216let readonly_data = [: 1; 2; 3; 4 :]
217```
218
219## Syntax Patterns
220
221### Mode Annotations
222```ocaml
223let func (param @ local) : result @ local = ...
224let value : type @ modes = ...
225let (pattern @ modes) = expression
226function_name @ modes arg1 arg2
227```
228
229### Kind Annotations
230```ocaml
231type ('a : float64) vector
232val process : ('a : immediate) -> unit
233let func (type (a : value) (b : bits32)) (x : a) (y : b) = ...
234```
235
236### Templates (for mode/kind polymorphism)
237```ocaml
238let%template[@mode m = (global, local)] identity
239 : 'a. 'a @ m -> 'a @ m = fun x -> x
240
241module%template[@kind k = (value, float64)] Math = struct
242 let add x y = (Float.add [@kind k]) x y
243end
244```
245
246### Unboxed Type Usage
247```ocaml
248(* Literals *)
249let pi = #3.14159 (* float# *)
250let count = #42l (* int32# *)
251
252(* Records *)
253type point = #{ x : float#; y : float# }
254let origin = #{ x = #0.0; y = #0.0 }
255
256(* Tuples *)
257let coords = #(#1.0, #2.0, #3.0)
258```
259
260## Error Handling Patterns
261
262### Zero Allocation Checking
263```ocaml
264let[@zero_alloc] fast_math x y = x + y * 2
265let[@zero_alloc assume] external_func x = C.some_func x
266```
267
268### Custom Error Messages
269```ocaml
270let process (x : (_ : immediate)[@error_message "requires immediate type"]) = x
271```
272
273## Performance Considerations
274
2751. **Stack vs Heap**: Use `@ local` for temporary values
2762. **Unboxed vs Boxed**: Use unboxed types (`#` suffix) for numeric code
2773. **Parallel vs Sequential**: Use `Parallel.fork_join2` for independent tasks
2784. **SIMD**: Use vector types for data-parallel numeric operations
2795. **Zero Allocation**: Mark performance-critical code with `[@zero_alloc]`
2806. **Immutable Arrays**: Use `[: ... :]` for read-only data
281
282## Common Pitfalls
283
284### Stack Allocation
285- Don't use `@ local` values in tail calls (use `[@nontail]`)
286- Avoid partial application with local parameters
287- Thread unique values through function calls, don't store in loops
288
289### Parallelism
290- Mark shared data as `@ portable contended`
291- Use capsules for complex mutable state
292- Ensure functions are `@ portable` for parallel use
293
294### Unboxed Types
295- Can't be used in tuples or variant constructors (current limitation)
296- Limited container type support
297- No polymorphic operations (comparison, marshaling)
298
299### Templates
300- Use locally abstract types for multiple kinds
301- Prefer mono-attributes for instantiation
302- Test all generated template instances
303
304## Library Usage
305
306### Key Libraries
307- `Parallel`: Parallel computation primitives
308- `Atomic`: Thread-safe atomic operations
309- `Float_u`, `Int32_u`, etc.: Unboxed numeric operations
310- `ocaml_simd`: SIMD vector operations
311- `Capsule`: Safe sharing of mutable state
312
313### Common Imports
314```ocaml
315open Parallel.Std
316module F32x4 = Ocaml_simd_sse.Float32x4
317```
318
319This reference enables effective OxCaml code generation that leverages the language's advanced features while maintaining safety and performance.