☰

# Nim Tutorial (Part III)

Theme:

- Manual
- Standard library
- Index
- Compiler docs
- Fusion docs
- devel, stable

Search:
Group by:
Source Edit

Author: | Arne Döring
Version: | 2.2.10

# Introduction

> "With Great Power Comes Great Responsibility." -- Spider Man's Uncle

This document is a tutorial about Nim's macro system. A macro is a function that is executed at compile-time and transforms a Nim syntax tree into a different tree.

Examples of things that can be implemented in macros:

- An assert macro that prints both sides of a comparison operator, if the assertion fails. `myAssert(a == b)` is converted to `if a != b: quit($a " != " $b)`
- A debug macro that prints the value and the name of the symbol. `myDebugEcho(a)` is converted to `echo "a: ", a`
- Symbolic differentiation of an expression. `diff(a*pow(x,3) + b*pow(x,2) + c*x + d, x)` is converted to `3*a*pow(x,2) + 2*b*x + c`

## Macro Arguments

The types of macro arguments have two faces. One face is used for the overload resolution and the other face is used within the macro body. For example, if `macro foo(arg: int)` is called in an expression `foo(x)`, `x` has to be of a type compatible to int, but within the macro's body `arg` has the type `NimNode`, not `int`! Why it is done this way will become obvious later, when we have seen concrete examples.

There are two ways to pass arguments to a macro, an argument can be either `typed` or `untyped`.

## Untyped Arguments

Untyped macro arguments are passed to the macro before they are semantically checked. This means the syntax tree that is passed down to the macro does not need to make sense for Nim yet, the only limitation is that it needs to be parsable. Usually, the macro does not check the argument either but uses it in the transformation's result somehow. The result of a macro expansion is always checked by the compiler, so apart from weird error messages, nothing bad can happen.

The downside for an `untyped` argument is that these do not play well with Nim's overloading resolution.

The upside for untyped arguments is that the syntax tree is quite predictable and less complex compared to its `typed` counterpart.

## Typed Arguments

For typed arguments, the semantic checker runs on the argument and does transformations on it, before it is passed to the macro. Here identifier nodes are resolved as symbols, implicit type conversions are visible in the tree as calls, templates are expanded, and probably most importantly, nodes have type information. Typed arguments can have the type `typed` in the arguments list. But all other types, such as `int`, `float` or `MyObjectType` are typed arguments as well, and they are passed to the macro as a syntax tree.

## Static Arguments

Static arguments are a way to pass values as values and not as syntax tree nodes to a macro. For example for `macro foo(arg: static[int])` in the expression `foo(x)`, `x` needs to be an integer constant, but in the macro body `arg` is just like a normal parameter of type `int`.

    import std/macros

    macro myMacro(arg: static[int]): untyped =
      echo arg # just an int (7), not `NimNode`

    myMacro(1 + 2 * 3)

## Code Blocks as Arguments

It is possible to pass the last argument of a call expression in a separate code block with indentation. For example, the following code example is a valid (but not a recommended) way to call `echo`:

    echo "Hello ":
      let a = "Wor"
      let b = "ld!"
      a & b

For macros this way of calling is very useful; syntax trees of arbitrary complexity can be passed to macros with this notation.

## The Syntax Tree

In order to build a Nim syntax tree one needs to know how Nim source code is represented as a syntax tree, and how such a tree needs to look like so that the Nim compiler will understand it. The nodes of the Nim syntax tree are documented in the macros module. But a more interactive way to explore the Nim syntax tree is with `macros.treeRepr`, it converts a syntax tree into a multi-line string for printing on the console. It can be used to explore how the argument expressions are represented in tree form and for debug printing of generated syntax tree. `dumpTree` is a predefined macro that just prints its argument in a tree representation, but does nothing else. Here is an example of such a tree representation:

    dumpTree:
      var mt: MyType = MyType(a:123.456, b:"abcdef")

    # output:
    #   StmtList
    #     VarSection
    #       IdentDefs
    #         Ident "mt"
    #         Ident "MyType"
    #         ObjConstr
    #           Ident "MyType"
    #           ExprColonExpr
    #             Ident "a"
    #             FloatLit 123.456
    #           ExprColonExpr
    #             Ident "b"
    #             StrLit "abcdef"

## Custom Semantic Checking

The first thing that a macro should do with its arguments is to check if the argument is in the correct form. Not every type of wrong input needs to be caught here, but anything that could cause a crash during macro evaluation should be caught and create a nice error message. `macros.expectKind` and `macros.expectLen` are a good start. If the checks need to be more complex, arbitrary error messages can be created with the `macros.error` proc.

    macro myAssert(arg: untyped): untyped =
      arg.expectKind nnkInfix

## Generating Code

There are two ways to generate the code. Either by creating the syntax tree with expressions that contain a lot of calls to `newTree` and `newLit`, or with `quote do:` expressions. The first option offers the best low-level control for the syntax tree generation, but the second option is much less verbose. If you choose to create the syntax tree with calls to `newTree` and `newLit` the macro `macros.dumpAstGen` can help you with the verbosity.

`quote do:` allows you to write the code that you want to generate literally. Backticks are used to insert code from `NimNode` symbols into the generated expression.

    import std/macros
    macro a(i) = quote do:
      let `i` = 0

    a b
    doAssert b == 0

A custom prefix operator can be defined whenever backticks are needed.

    import std/macros
    macro a(i) = quote("@") do:
      assert @i == 0

    let b = 0
    a b

The injected symbol needs accent quoted when it resolves to a symbol.

    import std/macros
    macro a(i) = quote("@") do:
      let `@i` = 0

    a b
    doAssert b == 0

Make sure to inject only symbols of type `NimNode` into the generated syntax tree. You can use `newLit` to convert arbitrary values into expressions trees of type `NimNode` so that it is safe to inject them into the tree.

    import std/macros

    type
      MyType = object
        a: float
        b: string

    macro myMacro(arg: untyped): untyped =
      var mt: MyType = MyType(a:123.456, b:"abcdef")

      # ...

      let mtLit = newLit(mt)

      result = quote do:
        echo `arg`
        echo `mtLit`

    myMacro("Hallo")

The call to `myMacro` will generate the following code:

    echo "Hallo"
    echo MyType(a: 123.456'f64, b: "abcdef")

## Building Your First Macro

To give a starting point to writing macros we will show now how to implement the `myAssert` macro mentioned earlier. The first thing to do is to build a simple example of the macro usage, and then just print the argument. This way it is possible to get an idea of what a correct argument should look like.

    import std/macros

    macro myAssert(arg: untyped): untyped =
      echo arg.treeRepr

    let a = 1
    let b = 2

    myAssert(a != b)

    Infix
      Ident "!="
      Ident "a"
      Ident "b"

From the output, it is possible to see that the argument is an infix operator (node kind is "Infix"), as well as that the two operands are at index 1 and 2. With this information, the actual macro can be written.

    import std/macros

    macro myAssert(arg: untyped): untyped =
      # all node kind identifiers are prefixed with "nnk"
      arg.expectKind nnkInfix
      arg.expectLen 3
      # operator as string literal
      let op  = newLit(" " & arg[0].repr & " ")
      let lhs = arg[1]
      let rhs = arg[2]

      result = quote do:
        if not `arg`:
          raise newException(AssertionDefect,$`lhs` & `op` & $`rhs`)

    let a = 1
    let b = 2

    myAssert(a != b)
    myAssert(a == b)

This is the code that will be generated. To debug what the macro actually generated, the statement `echo result.repr` can be used, in the last line of the macro. It is also the statement that has been used to get this output.

    if not (a != b):
      raise newException(AssertionDefect, $a & " != " & $b)

## Going further

It is possible to create more complex macros by combining different `NimNode` symbols with `quote do:` expressions. For example, you may use `newStmtList` to build your macro iteratively, and `ident` in cases in which you wish to create an identifier from a string, as shown below.

    import std/macros

    macro createProcedures() =
      result = newStmtList()

      for i in 0..<10:
        let name = ident("myProc" & $i)
        let content = newLit("I am procedure number #" & $i)

        result.add quote do:
          proc `name`() =
            echo `content`

    createProcedures()
    myProc7()

The call to `myProc7` will echo `I am procedure number #7`.

## With Power Comes Responsibility

Macros are very powerful. A piece of good advice is to use them as little as possible, but as much as necessary. Macros can change the semantics of expressions, making the code incomprehensible for anybody who does not know exactly what the macro does with it. So whenever a macro is not necessary and the same logic can be implemented using templates or generics, it is probably better not to use a macro. And when a macro is used for something, the macro should better have a well-written documentation. For all the people who claim to write only perfectly self-explanatory code: when it comes to macros, the implementation is not enough for documentation.

## Limitations

Since macros are evaluated in the compiler in the NimVM, macros share all the limitations of the NimVM. They have to be implemented in pure Nim code. Macros can start external processes on the shell, but they cannot call C functions except those that are built in the compiler.

# More Examples

This tutorial can only cover the basics of the macro system. There are macros out there that could be an inspiration for you of what is possible with it.

## Strformat

In the Nim standard library, the `strformat` library provides a macro that parses a string literal at compile time. Parsing a string in a macro like here is generally not recommended. The parsed AST cannot have type information, and parsing implemented on the VM is generally not very fast. Working on AST nodes is almost always the recommended way. But still `strformat` is a good example for a practical use case for a macro that is slightly more complex than the `assert` macro.

Strformat

## Ast Pattern Matching

Ast Pattern Matching is a macro library to aid in writing complex macros. This can be seen as a good example of how to repurpose the Nim syntax tree with new semantics.

Ast Pattern Matching

## OpenGL Sandbox

This project has a working Nim to GLSL compiler written entirely in macros. It scans recursively through all used function symbols to compile them so that cross library functions can be executed on the GPU.

OpenGL Sandbox