Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://www.tunbury.org/2025/07/25/build-analysis",
3 "title": "Website Build Analysis with Claude",
4 "link": "https://www.tunbury.org/2025/07/25/build-analysis/",
5 "updated": "2025-07-25T00:00:00",
6 "published": "2025-07-25T00:00:00",
7 "summary": "The Tarides website is built using Docker, and it would be interesting to run a quick analysis over the logs, given that we have over 300 days’ worth. This is one of those things where I’d usually turn to AWK and spend ages fiddling with the script.",
8 "content": "<p>The Tarides website is built using Docker, and it would be interesting to run a quick analysis over the logs, given that we have over 300 days’ worth. This is one of those things where I’d usually turn to AWK and spend ages fiddling with the script.</p>\n\n<p>However, this time I decided to ask Claude. The log files are organised by date e.g. 2024-09-24/HHMMSS-docker-build-HASH.log, where each day directory may contain many logs, as there can be several builds in a day. The HHMMSS is the time the job was created, and HASH is the MD5 hash of the job. The log format is as below, with only the start and end shown.</p>\n\n<div><div><pre><code>2024-09-24 14:45.02: New job: docker build\n...\n2024-09-24 14:55.14: Job succeeded\n</code></pre></div></div>\n\n<p>I would like a graph over time showing the duration each build takes to see if there are any trends.</p>\n\n<p>With a few iterations and very few minutes of effort, Claude had a working script. Beyond my initial description, I added the complexity that I wanted to run it in a Docker container with a bind mount for my logs and to exclude failed jobs and jobs that completed very quickly (likely due to the Docker caching).</p>\n\n<p>Claude’s code is in this <a href=\"https://gist.github.com/mtelvers/8383fb563e171778bfaf412f3119d50c\">gist</a></p>\n\n<p>Here’s the summary output</p>\n\n<div><div><pre><code>==================================================\nBUILD ANALYSIS SUMMARY (FILTERED DATA)\n==================================================\nOriginal builds found: 1676\nBuilds after filtering: 655\nFiltered out: 1021 (60.9%)\nFilter criteria: min_duration >= 100s, exclude_failed = True\n\nDuration Statistics (minutes):\n Mean: 10.16\n Median: 6.92\n Min: 5.53\n Max: 68.87\n Std Dev: 6.00\n\nDate Range:\n First build: 2024-09-24 14:45:50\n Last build: 2025-07-25 09:29:10\n\nAnalysis period: 305 days\nAverage builds per day: 2.1\n\nTop 5 longest builds:\n ✓ 2025-02-05 15:37 - 68.87m - 153726-docker-build-f9426a.log\n ✓ 2025-02-05 15:37 - 62.72m - 153724-docker-build-d227b6.log\n ✓ 2025-02-05 15:37 - 56.03m - 153723-docker-build-65de8e.log\n ✓ 2025-05-07 12:41 - 55.90m - 124115-docker-build-f4091b.log\n ✓ 2025-02-05 15:37 - 42.47m - 153722-docker-build-dafc1d.log\n\nTop 5 shortest builds (above threshold):\n ✓ 2025-01-13 14:26 - 5.53m - 142624-docker-build-fec55f.log\n ✓ 2024-09-25 10:10 - 5.65m - 101005-docker-build-c78655.log\n ✓ 2024-09-26 10:01 - 5.77m - 100119-docker-build-efd190.log\n ✓ 2025-02-07 18:09 - 5.83m - 180951-docker-build-ab19e5.log\n ✓ 2024-09-30 14:03 - 5.85m - 140301-docker-build-4028bb.log\nFiltered data exported to /data/output/build_analysis.csv\nRaw data exported to /data/output/build_analysis_raw.csv\n</code></pre></div></div>\n\n<p>And the graphs</p>\n\n<p><img alt=\"\" src=\"https://www.tunbury.org/images/build_times_timeline.png\"></p>\n\n<p><img alt=\"\" src=\"https://www.tunbury.org/images/daily_performance_trends.png\"></p>",
9 "content_type": "html",
10 "author": {
11 "name": "Mark Elvers",
12 "email": "mark.elvers@tunbury.org",
13 "uri": null
14 },
15 "categories": [
16 "tarides"
17 ],
18 "source": "https://www.tunbury.org/atom.xml"
19}