Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://anil.recoil.org/ideas/parallel-scheduling-with-effects",
3 "title": "Using effect handlers for efficient parallel scheduling",
4 "link": "https://anil.recoil.org/ideas/parallel-scheduling-with-effects",
5 "updated": "2022-01-01T00:00:00",
6 "published": "2022-01-01T00:00:00",
7 "summary": "<h2><a href=\"https://anil.recoil.org/ideas/parallel-scheduling-with-effects\">Using effect handlers for efficient parallel scheduling</a> <span>/ Jan 2022</span></h2><div><p>This is an idea proposed in 2022 as a Cambridge Computer Science Part III or MPhil project, and has been <span>completed</span> by <a href=\"https://github.com/bartoszmodelski\">Bartosz Modelski</a>.</p>\n<p>Modern hardware is so parallel and workloads are so concurrent, that there is\nno single, perfect scheduling strategy across a complex application software\nstack. Therefore, there are significant performance advantages to be gained\nfrom customizing and composing schedulers.</p>\n<p>Multicore parallelism is here to stay, and in contrast with clock frequency\nincreases, schedulers have to be carefully crafted in order to take full\nadvantage of horizontal scaling of the underlying architecture. That\u2019s because\ndesigns need to evolve as synchronization primitives such as locks or atomics\ndo not scale endlessly to many cores, and a naive work stealing scheduler that\nmay have been good enough on 16-thread Intel Xeon in 2012 will fail to utilize\nall 128 threads of a contemporary AMD ThreadRipper in 2022. Modern high-core\narchitectures also feature non-uniform memory and so memory latency patterns\nvary with the topology. Scheduling decisions will benefit from taking mem- ory\nhierarchy into account. Moreover, the non-uniformity also appears also in\nconsumer products such as Apple M1 or Intel Core i7-1280P. These highlight two\nsets of cores in modern architectures: one optimized for performance and\nanother one for efficiency.</p>\n<span>[\u2026<a href=\"https://anil.recoil.org/ideas/parallel-scheduling-with-effects\">483 words</a>]</span></div>",
8 "content": "<h1><a href=\"https://anil.recoil.org/#using-effect-handlers-for-efficient-parallel-scheduling\"></a>Using effect handlers for efficient parallel scheduling</h1>\n<p>This is an idea proposed in 2022 as a Cambridge Computer Science Part III or MPhil project, and has been <span>completed</span> by <a href=\"https://github.com/bartoszmodelski\">Bartosz Modelski</a>.</p>\n<p>Modern hardware is so parallel and workloads are so concurrent, that there is\nno single, perfect scheduling strategy across a complex application software\nstack. Therefore, there are significant performance advantages to be gained\nfrom customizing and composing schedulers.</p>\n<p>Multicore parallelism is here to stay, and in contrast with clock frequency\nincreases, schedulers have to be carefully crafted in order to take full\nadvantage of horizontal scaling of the underlying architecture. That\u2019s because\ndesigns need to evolve as synchronization primitives such as locks or atomics\ndo not scale endlessly to many cores, and a naive work stealing scheduler that\nmay have been good enough on 16-thread Intel Xeon in 2012 will fail to utilize\nall 128 threads of a contemporary AMD ThreadRipper in 2022. Modern high-core\narchitectures also feature non-uniform memory and so memory latency patterns\nvary with the topology. Scheduling decisions will benefit from taking mem- ory\nhierarchy into account. Moreover, the non-uniformity also appears also in\nconsumer products such as Apple M1 or Intel Core i7-1280P. These highlight two\nsets of cores in modern architectures: one optimized for performance and\nanother one for efficiency.</p>\n<p>This project uses the experimental multicore OCaml extension to explore\nconcurrent scheduling on multicore hardware, using library schedulers. Common\nprogramming languages either include threading support, which is tightly\ncoupled with the language itself, or offer no support and, thus,\nlibrary-schedulers cannot offer much beyond simply running scheduled functions\nin some order. OCaml, on the other hand, features fibers and effects. Together,\nthey allow writing a direct style, stack-switching scheduler as a library.\nFurther, OCaml allows composing schedulers -- a much-needed mechanism for\nexecuting diverse workloads with portions having different optimization\ncriteria.</p>\n<h2><a href=\"https://anil.recoil.org/#results\"></a>Results</h2>\n<p>The project was successfully concluded. To validate the hypothesis, it\ndeveloped several practical userspace schedulers and extended them with a\nnumber of work distribution methods. The code was written in OCaml with\nmulticore support, which features a novel effects-based approach to\nmultithreading. Most importantly, it decoupled lightweight threading from the\nruntime and lets user compose schedulers.\nThe evaluation involved several real-world benchmarks executed on up to 120\nthreads of a dual-socket machine with two AMD EPYC 7702 processors.</p>\n<p>The results showed that scaling applications to high core counts is\nnon-trivial, and some classic methods such as work stealing do not provide\noptimal performance. Secondly, different scheduling policies have a profound\nimpact on the throughput and latency of specific benchmarks, which justifies\nthe need to compose schedulers for heterogeneous workloads. Further, a\ncomposition of schedulers in a staged architecture was shown to provide better\ntail latency than its components. Moreover, the performance of the scheduler\ndeveloped in this project was shown to improve over the existing default\nMulticore OCaml scheduler - Domainslib. Finally, the results put in question a\ncommon design of overflow queue present in e.g., Go and Tokio (Rust).</p>\n<p>Read the full <a href=\"https://github.com/bartoszmodelski/ebsl/blob/main/report/report.pdf\">report\nPDF</a>\nonline, and see the <a href=\"https://github.com/bartoszmodelski/ebsl\">notebooks</a>\nassociated with the experiments here.</p>",
9 "content_type": "html",
10 "author": {
11 "name": "Anil Madhavapeddy",
12 "email": "anil@recoil.org",
13 "uri": "https://anil.recoil.org"
14 },
15 "categories": [],
16 "rights": "(c) 1998-2025 Anil Madhavapeddy, all rights reserved",
17 "source": "https://anil.recoil.org/news.xml"
18}