Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://www.tunbury.org/2025/03/23/real-time-trains",
3 "title": "Real Time Trains API",
4 "link": "https://www.tunbury.org/2025/03/23/real-time-trains/",
5 "updated": "2025-03-23T00:00:00",
6 "published": "2025-03-23T00:00:00",
7 "summary": "After the Heathrow substation electrical fire, I found myself in Manchester with a long train ride ahead. Checking on Real Time Trains for the schedule I noticed that they had an API. With time to spare, I registered for an account and downloaded the sample code from ocaml-cohttp.",
8 "content": "<p>After the Heathrow substation electrical fire, I found myself in Manchester with a long train ride ahead. Checking on <a href=\"https://www.realtimetrains.co.uk\">Real Time Trains</a> for the schedule I noticed that they had an API. With time to spare, I registered for an account and downloaded the sample code from <a href=\"https://github.com/mirage/ocaml-cohttp\">ocaml-cohttp</a>.</p>\n\n<p>The API account details uses HTTP basic authentication which is added via the HTTP header:</p>\n\n<div><div><pre><code> <span>let</span> <span>headers</span> <span>=</span> <span>Cohttp</span><span>.</span><span>Header</span><span>.</span><span>init</span> <span>()</span> <span>in</span>\n <span>let</span> <span>headers</span> <span>=</span>\n <span>Cohttp</span><span>.</span><span>Header</span><span>.</span><span>add_authorization</span> <span>headers</span> <span>(</span><span>`Basic</span> <span>(</span><span>user</span><span>,</span> <span>password</span><span>))</span>\n</code></pre></div></div>\n\n<p>The response from the API can be converted to JSON using <a href=\"https://github.com/ocaml-community/yojson\">Yojson</a>.</p>\n\n<div><div><pre><code><span>let</span> <span>json</span> <span>=</span>\n <span>Eio</span><span>.</span><span>Buf_read</span><span>.(</span><span>parse_exn</span> <span>take_all</span><span>)</span> <span>body</span> <span>~</span><span>max_size</span><span>:</span><span>max_int</span>\n <span>|></span> <span>Yojson</span><span>.</span><span>Safe</span><span>.</span><span>from_string</span>\n</code></pre></div></div>\n\n<p>The JSON field can be read using the <code>Util</code> functions. For example, <code>Yojson.Basic.Util.member \"services\" json</code> will read the <code>services</code> entry. Elements can be converted to lists with <code>Yojson.Basic.Util.to_list</code>. After a bit of hacking this turned out to be quite tedious to code.</p>\n\n<p>As an alternative, I decided to use <code>ppx_deriving_yojson.runtime</code>. I described the JSON blocks as OCaml types, e.g. <code>station</code> as below.</p>\n\n<div><div><pre><code><span>type</span> <span>station</span> <span>=</span> <span>{</span>\n <span>tiploc</span> <span>:</span> <span>string</span><span>;</span>\n <span>description</span> <span>:</span> <span>string</span><span>;</span>\n <span>workingTime</span> <span>:</span> <span>string</span><span>;</span>\n <span>publicTime</span> <span>:</span> <span>string</span><span>;</span>\n<span>}</span>\n<span>[</span><span>@@</span><span>deriving</span> <span>yojson</span><span>]</span>\n</code></pre></div></div>\n\n<p>The preprocessor automatically generates two functions:<code>station_of_json</code> and <code>station_to_json</code> which handle the conversion.</p>\n\n<p>The only negative on this approach is that RTT doesn’t emit empty JSON fields, so they need to be flagged as possibly missing and a default value provided. For example, <code>realtimeArrivalNextDay</code> is not emitted unless the value is <code>true</code>.</p>\n\n<div><div><pre><code> <span>realtimeArrivalNextDay</span> <span>:</span> <span>(</span><span>bool</span><span>[</span><span>@</span><span>default</span> <span>false</span><span>]);</span>\n</code></pre></div></div>\n\n<p>Now once the JSON has been received we can just convert it to OCaml types very easily:</p>\n\n<div><div><pre><code> <span>match</span> <span>reply_of_yojson</span> <span>json</span> <span>with</span>\n <span>|</span> <span>Ok</span> <span>reply</span> <span>-></span>\n <span>(* Use reply.services *)</span>\n <span>|</span> <span>Error</span> <span>err</span> <span>-></span> <span>Printf</span><span>.</span><span>printf</span> <span>\"Error %s</span><span>\\n</span><span>\"</span> <span>err</span>\n</code></pre></div></div>\n\n<p>My work in progress code is available on <a href=\"https://github.com/mtelvers/ocaml-rtt\">GitHub</a></p>\n\n<div><div><pre><code>dune exec --release -- rtt --user USER --pass PASS --station RTR\nrtt: [DEBUG] received 3923 bytes of body\nrtt: [DEBUG] received 4096 bytes of body\nrtt: [DEBUG] received 4096 bytes of body\nrtt: [DEBUG] received 4096 bytes of body\nrtt: [DEBUG] received 1236 bytes of body\nrtt: [DEBUG] end of inbound body\n2025-03-23 2132 W16178 1C69 1 Ramsgate St Pancras International\n2025-03-23 2132 W25888 9P59 2 Plumstead Rainham (Kent)\n2025-03-23 2136 J00119 1U28 2 London Victoria Ramsgate\n2025-03-23 2144 W25927 9P86 1 Rainham (Kent) Plumstead\n2025-03-23 2157 W16899 1C66 2 St Pancras International Ramsgate\n2025-03-23 2202 W25894 9P61 2 Plumstead Rainham (Kent)\n2025-03-23 2210 J26398 1U80 1 Ramsgate London Victoria\n2025-03-23 2214 W25916 9P70 1 Rainham (Kent) Plumstead\n2025-03-23 2232 W16910 1C73 1 Ramsgate St Pancras International\n2025-03-23 2232 W25900 9P63 2 Plumstead Rainham (Kent)\n2025-03-23 2236 J00121 1U30 2 London Victoria Ramsgate\n2025-03-23 2244 W25277 9A92 1 Rainham (Kent) Dartford\n2025-03-23 2257 W16450 1F70 2 St Pancras International Faversham\n2025-03-23 2302 W25906 9P65 2 Plumstead Rainham (Kent)\n2025-03-23 2314 W25283 9A94 1 Rainham (Kent) Dartford\n2025-03-23 2318 J00155 1U82 1 Ramsgate London Victoria\n2025-03-23 2332 W25912 9P67 2 Plumstead Gillingham (Kent)\n2025-03-23 2336 J00123 1U32 2 London Victoria Ramsgate\n2025-03-23 2344 W25289 9A96 1 Rainham (Kent) Dartford\n2025-03-23 2357 W16475 1F74 2 St Pancras International Faversham\n2025-03-23 0002 W25915 9P69 2 Plumstead Gillingham (Kent)\n2025-03-23 0041 J26381 1Z34 2 London Victoria Faversham\n</code></pre></div></div>",
9 "content_type": "html",
10 "author": {
11 "name": "Mark Elvers",
12 "email": "mark.elvers@tunbury.org",
13 "uri": null
14 },
15 "categories": [
16 "OCaml",
17 "tunbury.org"
18 ],
19 "source": "https://www.tunbury.org/atom.xml"
20}