commit 3d73fd93a66f0856c68af571a877a551040cfc00 · hauleth.dev/blog

+19 -13

content/post/log-all-the-things.md

···

       9
       9
        
         "programming",

     

       10
       10
        
         "observability"

     

       11
       11
        
       ]

     

       12
       12
       +
       

     

       13
       13
       +
       [[extra.thanks]]

     

       14
       14
       +
       name = "Kai Wern Choong"

     

       12
       15
        
       +++

     

       13
       16
        
       

     

       14
       17
        
       In Elixir 1.11 landed set of new features that allows for more powerful logging

     

       15
       15
       -
       by utilising Erlang's [`logger`][erl-log] features. Here I will try to descibe

     

       18
       18
       +
       by utilising Erlang's [`logger`][erl-log] features. Here I will try to describe

     

       16
       19
        
       new possibilities and how You can use them to improve your logs.

     

       17
       20
        
       

     

       18
       21
        
       <!-- more -->

     
···

       102
       105
        
       # Nothing got logged as top-level verbositi is still set to `:error`

     

       103
       106
        
       ```

     

       104
       107
        
       

     

       105
       105
       -
       Of course it will not work if you decide to use [compile time purging](https://hexdocs.pm/logger/Logger.html#module-application-configuration).

     

       108
       108
       +
       Of course it will not work if you decide to use [compile time purging][logger-purge]

     

       106
       109
        
       

     

       107
       110
        
       ## Logger handlers {#handlers}

     

       108
       111
        
       

     
···

       144
       147
        
             hierarchy which then can be used for filtering. All events fired using

     

       145
       148
        
             `Logger` macros and functions will have `:elixir` prepended to their

     

       146
       149
        
             domain list.

     

       147
       147
       -
           - `:report_cb` - function that will be used to format `{:report, map() | keyword()}`

     

       148
       148
       -
             messages. This can be either 1-ary function, that takes report and returns

     

       149
       149
       -
             `{:io.format(), [term()]}` leaving truncation and further formatting up to

     

       150
       150
       -
             the main formatter, or 2-ary function that takes report and configuration

     

       151
       151
       -
             map `%{depth: pos_integer() | :unlimited, chars_limit: pos_integer() |

     

       152
       152
       -
             :unlimited, single_line: boolean()}` and returns already formatted

     

       153
       153
       -
             `:unicode.chardata()`. More about it can be found in [separate section](#structured-logging).

     

       150
       150
       +
           - `:report_cb` - function that will be used to format `{:report, map() |

     

       151
       151
       +
             keyword()}` messages. This can be either 1-ary function, that takes report

     

       152
       152
       +
             and returns `{:io.format(), [term()]}` leaving truncation and further

     

       153
       153
       +
             formatting up to the main formatter, or 2-ary function that takes report

     

       154
       154
       +
             and configuration map `%{depth: pos_integer() | :unlimited, chars_limit:

     

       155
       155
       +
             pos_integer() | :unlimited, single_line: boolean()}` and returns already

     

       156
       156
       +
             formatted `:unicode.chardata()`. More about it can be found in [separate

     

       157
       157
       +
             section](#structured-logging).

     

       154
       158
        
       

     

       155
       159
        
       Return value of this function is ignored. If there will be any exception raised

     

       156
       160
        
       when calling this function, then it will be captured and failing handler will be

     
···

       163
       167
        
       can cause quite substantial load on application.

     

       164
       168
        
       

     

       165
       169
        
       To read on other, optional, callbacks that can be defined by Erlang handler, that

     

       166
       166
       -
       will not be covered there, I suggest looking into [Erlang documentation](https://erlang.org/doc/man/logger.html#formatter-callback-functions).

     

       170
       170
       +
       will not be covered there, I suggest looking into [Erlang documentation][formatter_cb].

     

       167
       171
        
       

     

       168
       172
        
       ## Structured logging {#structured-logging}

     

       169
       173
        
       

     
···

       328
       332
        
       

     

       329
       333
        
       The rule of thumb you can follow is:

     

       330
       334
        
       

     

       331
       331
       -
       > If it is thing that you will want to filter on, then it propably should be

     

       335
       335
       +
       > If it is thing that you will want to filter on, then it probably should be

     

       332
       336
        
       > part of the metadata. If you want to aggregate information or just display

     

       333
       337
        
       > them, it should be part of the message report.

     

       334
       338
        
       

     
···

       414
       418
        
       approach you could abuse translators, but that was error prone, as first

     

       415
       419
        
       successful translator was breaking pipeline, so you couldn't just smash one on

     

       416
       420
        
       top and then keep rest working as is. With "new" approach and structured logging

     

       417
       417
       -
       you can just traverse the report and replace all occurences of the unsafe data

     

       421
       421
       +
       you can just traverse the report and replace all occurrences of the unsafe data

     

       418
       422
        
       with anonymised data. For example:

     

       419
       423
        
       

     

       420
       424
        
       ```elixir

     
···

       450
       454
        
       defp replace(other), do: other

     

       451
       455
        
       ```

     

       452
       456
        
       

     

       453
       453
       -
       This snippet will replace all occurences of `:password` or `"password"` with

     

       457
       457
       +
       This snippet will replace all occurrences of `:password` or `"password"` with

     

       454
       458
        
       filtered out value.

     

       455
       459
        
       

     

       456
       460
        
       The disadvantage of such approach - it will make all messages with such fields

     
···

       476
       480
        
       [`LoggerJSON`]: https://github.com/Nebo15/logger_json

     

       477
       481
        
       [`Ink`]: https://hex.pm/packages/ink

     

       478
       482
        
       [logger_filters]: https://erlang.org/doc/man/logger_filters.html

     

       483
       483
       +
       [logger-purge]: https://hexdocs.pm/logger/Logger.html#module-application-configuration

     

       484
       484
       +
       [formatter_cb]: https://erlang.org/doc/man/logger.html#formatter-callback-functions

templates/index.html

···

       42
       42
        
           </nav>

     

       43
       43
        
         </header>

     

       44
       44
        
       {% endblock %}

     

       45
       45
       +
       

     

       46
       46
       +
       {#

     

       47
       47
       +
         vi: ft=jinja

     

       48
       48
       +
       #}

templates/macros/posts.html

···

       52
       52
        
               #<a class="p-category" href="{{get_taxonomy_url(kind="tags", name=tag )}}">{{ tag }}</a>

     

       53
       53
        
           {% endfor -%}

     

       54
       54
        
       {% endmacro tags %}

     

       55
       55
       +
       

     

       56
       56
       +
       {#

     

       57
       57
       +
         vi: ft=jinja

     

       58
       58
       +
       #}

templates/macros/toc.html

···

       9
       9
        
         {%- endfor -%}

     

       10
       10
        
         </ol>

     

       11
       11
        
       {% endmacro toc %}

     

       12
       12
       +
       

     

       13
       13
       +
       {#

     

       14
       14
       +
         vi: ft=jinja

     

       15
       15
       +
       #}

+28 -3

templates/page.html

···

       3
       3
        
       {%- import "macros/toc.html" as toc -%}

     

       4
       4
        
       

     

       5
       5
        
       {%- block title -%}

     

       6
       6
       -
           <title>{{ page.title }} - {{  config.extra.author }}</title>

     

       6
       6
       +
         <title>{{ page.title }} - {{  config.extra.author }}</title>

     

       7
       7
        
       {%- endblock title -%}

     

       8
       8
        
       

     

       9
       9
        
       {%- block main -%}

     
···

       24
       24
        
           <div class="post-content e-content">

     

       25
       25
        
             {{ page.content | safe }}

     

       26
       26
        
           </div>

     

       27
       27
       -
           {% if not page.extra.no_comments %}

     

       27
       27
       +
           {%- if page.extra.thanks -%}

     

       28
       28
       +
           <hr />

     

       29
       29
       +
           <p>

     

       30
       30
       +
             <b>Special thanks</b>:

     

       31
       31
       +
             <ul>

     

       32
       32
       +
               {%- for person in page.extra.thanks -%}

     

       33
       33
       +
               <li class="h-card">

     

       34
       34
       +
                 {%- if person is object -%}

     

       35
       35
       +
                   {%- if person.url -%}

     

       36
       36
       +
                     <a class="u-url p-name" href="{{ person.url }}">{{ person.name }}</a>

     

       37
       37
       +
                   {%- else -%}

     

       38
       38
       +
                     <span class="p-name">{{ person.name }}</span>

     

       39
       39
       +
                   {%- endif -%}

     

       40
       40
       +
                 {%- else -%}

     

       41
       41
       +
                   <span class="p-name">{{ person }}</span>

     

       42
       42
       +
                 {%- endif -%}

     

       43
       43
       +
               </li>

     

       44
       44
       +
               {%- endfor -%}

     

       45
       45
       +
             </ul>

     

       46
       46
       +
           </p>

     

       47
       47
       +
           {%- endif -%}

     

       48
       48
       +
           {%- if not page.extra.no_comments -%}

     

       28
       49
        
           <hr />

     

       29
       50
        
           <div>

     

       30
       51
        
             <p>You can provide feedback via mailing list

     
···

       32
       53
        
                 page.title }}">~hauleth/blog@lists.sr.ht</a>

     

       33
       54
        
                 (<a href="https://lists.sr.ht/~hauleth/blog">archive</a>).</p>

     

       34
       55
        
           </div>

     

       35
       35
       -
           {% endif %}

     

       56
       56
       +
           {%- endif -%}

     

       36
       57
        
         </article>

     

       37
       58
        
       {%- endblock main -%}

     

       59
       59
       +
       

     

       60
       60
       +
       {#

     

       61
       61
       +
         vi: ft=jinja

     

       62
       62
       +
       #}