this repo has no description
dekker.one
1# Dynamic tracing in MiniZinc
2
3MiniZinc has preliminary support for dynamic tracing. This allows developers to
4enable inserting custom tracing scripts at certain crucial points during
5execution without any the overhead when tracing is not required.
6
7This tracing is supported through DTrace. DTrace is only available for MacOS
8and BSD. For Linux there is an alternative called
9[SystemTap](https://sourceware.org/systemtap/). SystemTap uses the same
10resources as DTrace, but uses different scripts.
11
12## DTrace
13
14[DTrace](http://dtrace.org/guide/preface.html) provides a library for dynamic
15tracing of events. This document describes the implementation of DTrace within
16the MiniZinc compiler. In particular, it focuses on how to use and extend the DTrace
17implementation within MiniZinc.
18
19
20### Executing DTrace scripts with MiniZinc
21
22You can find various example DTrace scripts in `tests/dtrace/`. To run these
23scripts, you can use a command in the form of `[script] -c "[command]"`. You need
24to use a MiniZinc compiler compiled with the DTrace to use DTrace.
25
26You can also execute a script using the dtrace command. This is necessary if the
27script does not contain a "Shebang". You can do it using the following command:
28`dtrace -s [script] -c "[command]"`.
29
30#### MacOS 10.11+
31
32MacOS El Capitan for the first time included "System Integrity Protection". This
33feature does not allow for all DTrace features to be used. This includes custom
34DTrace providers. To get around this problem, the following steps can be
35followed:
36
371. Boot your Mac into Recovery Mode.
382. Open the Terminal from the Utilities menu.
393. Then use the following commands:
40
41```bash
42
43csrutil clear # restore the default configuration first
44
45csrutil enable --without dtrace # disable dtrace restrictions *only*
46
47```
48
49After completing these steps your Mac should be able to use DTrace after a
50restart. For more information visit the following article:
51[link](http://internals.exposed/blog/dtrace-vs-sip.html).
52
53### Writing scripts for DTrace with MiniZinc
54
55The following paragraph will inform you on the MiniZinc provider and its probes
56for DTrace. We refer to the [DTrace
57documentation](http://dtrace.org/guide/preface.html) for more information on
58the syntax and possibilities.
59
60The MiniZinc provider for DTrace consists of DTrace probes implemented in the
61Runtime. A binary compiled with DTrace enabled will allow the user access to
62the information of the probes. You can find a list of all probes and their
63arguments in
64[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d).
65The following is a toy example DTrace script:
66
67```
68
69pony$target:::gc-start
70{
71 @ = count();
72}
73
74END
75{
76 printa(@);
77}
78
79```
80
81This script increases a counter every time the "GC Start" probe is *fired* and
82prints it result at the end. Note the way in which we access the probe. It
83consists of two parts: the first part is the provider and the second part,
84after `:::` is the name of the probe. The provider is during runtime appended
85by the process ID. In this example we use the `$target` variable to find the
86process ID. Another option is to use `pony*`, but note that this could match
87many processes. The name of the probe are also different from the names in
88[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d).
89In Dtrace scripts you have to replace the `__` in the names by `-`.
90
91To make these scripts executable, like the ones in the examples, we use the
92following "Shebang": `#!/usr/bin/env dtrace -s`. Extra options can be added to
93improve its functionality.
94
95
96# SystemTap
97
98SystemTap is the Linux alternative to DTrace. Although DTrace is available for
99Linux the licensing and philosophical differences have caused Linux developers
100to create an alternative tracing framework that is compatible with the GPLv2,
101thus SystemTap was created. SystemTap and DTrace operate in a similar fashion
102but since then the frameworks have been developed independently. Complete
103SystemTap documentation can be found
104[here](https://sourceware.org/systemtap/documentation.html)
105
106## Requirements
107
108* Linux Kernel with UPROBES or UTRACE.
109
110 If your Linux kernel is version 3.5 or higher, it already includes UPROBES. To
111 verify that the current kernel supports UPROBES natively, run the following
112 command:
113
114 ```
115 # grep CONFIG_UPROBES /boot/config-`uname -r`
116 ```
117
118 If the kernel supports the probes this will be the output:
119 ```
120 CONFIG_UPROBES=y
121 ```
122
123 If the kernel's version is prior to 3.5, SystemTap automatically builds the
124 UPROBES module. However, you also need the UTRACE kernel extensions required
125 by the SystemTap user-space probing to track various user-space events. This
126 can be verified with this command:
127
128 ```
129 # grep CONFIG_UTRACE /boot/config-`uname -r
130 ```
131 If the < 3.5 kernel supports user tracing this will be the output:
132
133 ```
134 CONFIG_UTRACE=y
135 ```
136
137 Note: UTRACE does not exist on kernel versions > 3.5 and has been replaced by
138 UPROBES
139
140* SystemTap > 2.6
141
142 You need the `dtrace` commandline tool to generate header and object files
143 that are needed in the compilation and linking of MiniZinc compiler. This is
144 required on Linux systems. At the time of writing this the version of
145 SystemTap that these probes have been implemented and tested with is 2.6, on
146 Debian 8 Stable. Later versions should work. Earlier versions might not work
147 and must be tested independently. In debian based systems, SystemTap can be
148 installed with apt-get
149
150 ```
151 $sudo apt-get install systemtap systemtap-sdt-dev
152 ```
153
154## Executing SystemTap scripts with MiniZinc
155
156You can find various example SystemTap scripts in `tests/dtrace/`. To run
157these scripts, a sample command would be of the form:
158
159```
160# stap [script path] -c "[command]"
161```
162
163NB: stap must be run as root (or sudo) since it compiles the SystemTap script
164into a loadable kernel module and loads it into the kernel when the stap script
165is running.
166
167You need to use a PonyC compiler compiled with the DTrace support to use SystemTap.
168
169## Writing scripts for SystemTap in MiniZinc
170
171SystemTap and DTrace use the same syntax for creating providers and thus we
172direct to the [DTrace documentation](http://dtrace.org/guide/preface.html) for
173more information on the syntax and possibilities.
174
175The MiniZinc provider for SystemTap consists of DTrace probes implemented in
176the Runtime. A binary compiled with DTrace enabled will allow the user access
177to the information of the probes, which work with SystemTap scripts. You can
178find a list of all probes and their arguments in
179[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d).
180The following is a toy example of a SystemTap script:
181
182```
183global gc_passes
184
185probe process.mark("gc-start")
186{
187 gc_passes <<< $arg1;
188}
189
190probe end
191{
192 printf("Total number of GC passes: %d\n", @count(gc_passes));
193}
194
195```
196
197This simple example will hook into the executable that the -c parameter provides
198and searches for the probe named "gc-start". Once execution of the executable is
199started, the script increases a counter every time the "gc-start" probe is
200accessed and at the end of the run the results of the counter are printed
201
202In SystemTap you can use the DTrace syntax of "gc-start" but you may also call
203them like they are in the
204[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d)
205file, "gc__start".
206
207All available probes in an executable can be listed like this:
208
209```
210# stap -L 'process("<name_and_path_of_executable>").mark("*")'
211```
212
213This is useful to confirm that probes are ready to be hooked in.
214
215## Extending dynamic tracing in MiniZinc
216
217You can extend the dynamic tracing implementation by adding more probes or
218extra information to existing probes. All probes are defined in
219[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d).
220Usually their names of their module and the event that triggers them. To
221install Probes in C use the macros defined in
222`include/minizinc/support/dtrace.h`. To fire a probe in the C code use the
223macro `DTRACEx`; where `x` is the number of arguments of the probe. There is
224also a macro `DTRACE_ENABLED`; its use allows for code to be only executed when
225a probe is enabled.
226
227### Adding a probe
228
229The first step to add a probe is to add its definition into
230[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d).
231We have split the names of the probes into two parts: its category and the
232specific event. The name is split using `__`. We also add a comment to a probe
233explaining when it's fired and the information contained in its arguments.
234
235After adding the probe, we use it in the C code using the macros. Note that the
236split in the name is only a single underscore in the C code. The name of the
237probe should also be capitalised. We can call the probe defined as `gc__start`
238using the statement `DTRACE1(GC_START, scheduler)`. In this statement
239`scheduler` is the data used as the first argument.
240
241Then once the probe has been placed in all the appropriate places, we are ready
242to recompile. Make sure to use the DTrace flag while compiling. Recompiling will
243create the appropriate files using the system installation of DTrace.
244
245### Extending a probe
246
247We can extend a probe by adding an extra argument to the probe. Like adding a
248probe, the probe definition needs to be appended in
249[`include/minizinc/support/dtrace_probes.d`](../../include/minizinc/support/dtrace_probes.d)
250Note that this extra information needs to be available **everywhere** the probe
251is used.
252
253Once you've added the argument, you need to change **all** instances where the
254probe is in use. Note that you have to both add the new argument and change the
255`DTRACE` macro. Then you can recompile the MiniZinc compiler to use your new
256arguments.
257
258*Do not forget that if you change the order of the arguments for any of the
259existing nodes, you also need to change their usage in the example scripts.*