How to generate Lua boilerplate with D

Lua is a scripting language for embedding. It is implemented by and provides an API for C. Then there is the D programming language; A comprehensive language and a great host for Lua via the LuaD package. Even better than C itself as you will see.

For an example, we make a simple function available in Lua code. The function computes the sine. To do that it must convert a Lua argument into a number, apply the sin C-function, and return the result as a Lua value. Here is the necessary code with the C bindings.

static int l_sin (lua_State *L) {
  double d = luaL_checknumber(L, 1);
  double result = sin(d);
  lua_pushnumber(L, result);
  return 1; /* number of results */
}

lua_pushcfunction(l, l_sin);
lua_setglobal(l, "sin");

Now compare this with the D version, where all the boilerplate happens implicitly.

l["sin"] = &sin;

With some compile-time metaprogramming, D can magically generate all the boilerplate. Let us go down the rabbit hole of LuaD, to understand how it works.

Looking behind the Curtain

There is some syntactic sugar with the assignment, but ultimately what happens is LuaTable.set("sin", &sin). You can see the code in context in the LuaD repo.

this.push();
scope(success) lua_pop(this.state, 1);

pushValue(this.state, key);    // "sin"
pushValue(this.state, value);  // &sin
lua_settable(this.state, -3);

If you do not know D, the scope(success) line requires an explanation. This is a scope guard statement. Here it means to call lua_pop at the very end if the function returns successfully (not throwing an exception). This mechanism is useful to keep the push and pop closer together. In C/C++ we would place the lua_pop after the lua_settable line.

The call to lua_settable is equivalent to lua_setglobal only passing the global table explicitly.

The leaves the pushValue function for the magic. Especially, the line with value, because that has to wrap the boilerplate around the sin function.

First, pushValue can handly arbitrary types. How does that work? Look into the code.

void pushValue(T)(lua_State* L, T value)
{
  static if(is(T : LuaObject))
      value.push();
  else static if(is(T == LuaDynamic))
      value.object.push();
  // many more cases ...
  else static if(is(T : const(char)*))
      lua_pushstring(L, value);
  // ...
  else static if(isSomeFunction!T)
      pushFunction(L, value);
  else
      static assert(false, "Unsupported type `" ~ T.stringof ~ "` in stack push operation");
}

It is a template function. In C++ the syntax would be pushValue<T>(lua_State* L, T value). Inside is an if-cascade, which checks the template parameter type T. The static keyword before each if means that this happens at compile time. This static-if mechanism was proposed to C++ multiple times, but so far it was always rejected.

You can see the case for the string (our key with value "sin" in the example) maps directly to the Lua API lua_pushstring.

The real magic happens where T is some function. We are redirected to pushFunction.

pushFunction(T)(lua_State* L, T func) if (isSomeFunction!T)
{
    lua_pushlightuserdata(L, func);
    lua_pushcclosure(L, &functionWrapper!T, 1);
}

This function is simple but uses the more complex functionWrapper template. The template instantiation syntax for D is via !, while in C++ we would write functionWrapper<T> instead.

We see that LuaD is slightly inefficient at this point. It registers a closure instead of a function to pass the function reference into the boilerplate wrapper. Although, that should not change performance significantly.

Note that functionWrapper!T is not called here only instantiated. It is called by Lua it later. The body of the function is similar to the following, but I simplified the real code a little by removing special cases unnecessary for our example.

extern(C) int functionWrapper(T)(lua_State* L)
{
    alias FillableParameterTypeTuple!T Args;
    enum requiredArgs = Args.length;
    int top = lua_gettop(L);
    if(top < requiredArgs)
        argsError(L, top, requiredArgs);

    T func = cast(T)lua_touserdata(L, lua_upvalueindex(1));

    Args args;  // a typed tuple where we put the parameters
    foreach(i, Arg; Args)
        args[i] = getArgument!(T, i)(L, i + 1); // pop from Lua stack
    return callFunction!T(L, func, args);
}

The first part checks that the number of Lua parameters matches the number parameters our function takes according to its type and calls argsError if not. Then we pop the function reference we pushed for the closure. Due to the template parameter we still know the type T of the function reference.

In the foreach loop the parameters are pop'd from the Lua stack. The getArgument call redirects to getValue, which contains a similar static-if cascade as pushValue. It returns a value from the Lua stack with the correct static type. So, at this point half the boilerplate is created.

This leaves callFunction to explain. It looks very much like this:

int callFunction(T)(lua_State* L, T func, ParameterTypeTuple!T args)
{
    alias BindableReturnType!T RetType;
    RetType ret;
    try {
        ret = func(args);   // call &sin
    } catch (Exception e) {
        luaL_error(L, "%s", toStringz(e.toString()));
    }
    return pushReturnValues(L, ret);   // push result to Lua stack
}

You can even see some exception catching, which is not necessary for sin, though.

The pushReturnValues function is a template; Again with a static-if-cascade to handle the different types correctly. The template parameter is infered from the type of the ret argument, which we extracted from the template parameter via BindableReturnType!T.

Now you know how LuaD does its magic. You know how D merges the power of templates and compile-time function evaluation with the convenience of inference and scope guard statements.

Tweet This“It's still magic even if you know how it's done.” ― Terry Pratchett, A Hat Full of Sky

The content of this article will probably be used for my TopConf talk in October.