The Puppet 4x function API

In Puppet 4.0.0 there is a new API for writing Ruby functions that extend the functionality of the Puppet language. This API is available in the 3.7.x versions of Puppet when using --parser future, so you can try out this functionality today.

The new 4x API for functions was created to fix problems and add missing features in the 3x API:

• The function runs as a method on Scope (and has access to too much non-API)
• Undefined arguments are given to the function as empty strings, but as a :undef Symbol if undefined values are given inside collections.
• There is no automatic type checking
• Functions share a flat namespace and you have to ensure you use a unique name
• Functions cannot be private to a module
• Functions are defined in the Puppet::Parser::Functions namespace. Future use of functions is to also use them where no parser is available. The concept of "parser function" is just odd.
• Methods defined in a Function pollute Scope - if you require helper logic it must be in a separate class.
• There are problems with reloading complex functions
• There is a distinction between functions of expression and statement kind, and this distinction is no longer meaningful.
• The specification of arity (number of arguments) used in 3x to describe parameters to a function, is a blunt tool (no typing, no overloading, and it can not express a variable number of arguments that is capped).
• Documentation can not (at least not easily) be retrieved without running the ruby code that defines the function.

The 4x function API solves all of these issues. (With the exception of private functions, which did not make it into 4.0.0, but will be added during the 4x series).

A simple function in the 4x API

The new API has many features, yet, for simple functions, it is very easy to use. Here is a basic example.

Puppet::Functions.create_function(:max) do
  def max(x, y)
    x >= y ? x : y
  end
end

This defines the function max taking two arguments (of Any kind). As you can see, it is slightly different from the 3x function API in that the body of the function is expressed in a defined method.

Also different is that functions are now stored under <moduleroot>/lib/puppet/functions instead of under the terribly confusing <moduleroot>/lib/puppet/parser/functions in 3x which has mislead everyone to talk about "parser functions" - which I guess could mean some kind of function used for parsing. Neither the 3x nor the 4x function plays any role during parsing, and they should be referred to as "functions". So please, no more "parser function" crazy talk...

Automatic Type Checking

In the 4x API there is support for type checking. Here is the same function again, now with type checking:

Puppet::Functions.create_function(:max) do
  dispatch :max do
    param 'Numeric', :a
    param 'Numeric', :b
  end

  def max(x, y)
    x >= y ? x : y
  end
end

As you can see, the max method is identical to the first version. A call to a dispatch method has been added to type the parameters. In addition to typing the parameters, the dispatch call also informs puppet that the call should be dispatched to a particular method (in the example above to :max). If we inside our function want to call the method max_num (instead of the method max) we would change the definition like this:

Puppet::Functions.create_function(:max) do
  dispatch :max_num do
    param 'Numeric', :a
    param 'Numeric', :b
  end

  def max_num(x, y)
    x >= y ? x : y
  end
end

The function is still named max() in the Puppet Language, but internally, when it is called with two Numeric arguments, the call is now dispatched to the max_num method. As you will see in the next section, this is very useful when we want to write functions that have different implementations depending on the types of the arguments given to it when it is called.

When defining a parameter, the type is always given in a string using the Puppet Language Type System notation. This means you can be very detailed in your specification and get type checking with high fidelity.

Multiple Dispatch

The 4x API supports multiple dispatch; so far you have seen two examples. In the first there where no calls to dispatch and the system automatically figured out that the call should be dispatched to a method with the same name as the function.

In the second example we took over dispatching, and declared that a call requires two Numeric arguments.

What if you want to call max with either Numeric, or String arguments? We could certainly type the arguments as Variant[Numeric, String], but we would then also need to write the logic in our method to deal with all of the possible cases. A much simpler approach is to use multiple dispatch. Here is an example - this time for a min function:

Puppet::Functions.create_function(:min) do
  dispatch :min do
    param 'Numeric', :a
    param 'Numeric', :b
  end

  dispatch :min_s do
    param 'String', :s1
    param 'String', :s2
  end

  def min(x,y)
    x <= y ? x : y
  end

  def min_s(x,y)
    cmp = (x.downcase <=> y.downcase)
    cmp <= 0 ? x : y
  end
end

Now the system will look at the types of the given arguments and pick the first matching dispatcher. Thus, in min we know that the arguments are Numeric, and in min_s we know that they are String. Everything is precise, small, clear and easy to read. We also did not have to spend time on dealing with error handling as type checking always takes place in all calls.

Variable Number of Arguments

The 4x API can handle a variable number of arguments. If you do not use a dispatcher the logic introspects the Ruby method declaration and checks the types of the arguments. If we change the max function to return the maximum of a variable number of arguments we can do that like this:

Puppet::Functions.create_function(:max) do
  def max(*args)
    args.reduce {|x, y| x >= y ? x : y }
  end
end

If you want to also type the arguments, or cap the max number of arguments, then this is done in the dispatcher by defining the minimum and maximum argument count with a call to arg_count. In the example below a minimum of 1 argument is specified, and a maximum of :default (which means any number of arguments).

Puppet::Functions.create_function(:max) do
  dispatch :max do
    param 'Numeric', :args
    arg_count 1, :default
  end

  def max(*args)
    args.reduce {|x, y| x >= y ? x : y }
  end
end

Note that the arg_count specifies the min required and max allowed number of arguments given to the function (i.e. it is not just for the last parameter). Also note that the method the call is dispatched to can be defined in any compatible way (i.e. it must handle missing arguments by using default values, or capture variable arguments in an array as in the example below:

Puppet::Functions.create_function(:example) do
  dispatch :example do
    param 'Numeric', :name
    param 'String', :value
    param 'Numeric', :name2
    param 'String', :value2
    arg_count 2, 4
  end

  def example(name, value, *args)
  end
end

Namespaced Functions

In 3x it is not possible to give functions a name-spaced name. They all live in the same name space. This is a problem because one module may override functions in another module. In 4x, the functions can be given a complex name. To do this, the function should be placed in a directory that corresponds to the name space, and it should be named accordingly in the call to create_function.

Here, the function max is placed in the namespace mymodule (which is also the name of the module).

# in <moduleroot>/lib/functions/mymodule/max.rb
Puppet::Functions.create_function(:'mymodule::max') do
  dispatch :max do
    param 'Numeric', :args
    arg_count 1, :default
  end

  def max(*args)
    args.reduce {|x, y| x >= y ? x : y }
  end
end

Note that it is only the name of the function that needs to be given the fully qualified name, in the dispatcher the name of the Ruby method to dispatch to is still used, and it is not a fully qualified name.

You can nest namespaces further if you like.

To call to a fully qualified function from the Puppet Language simply uses the full name - e.g:

mymodule::max(1,2,3,4)

Helper Logic

You can have as many helper methods you like in the function - it is only the methods being dispatched to that are being used by the 4x function API. You are however not allowed to define nested ruby classes, modules, or introduce constants inside the function definition. If you have that much code, you should deliver that elsewhere and call that logic. Note that such external logic is static across all environments.

Documenting the Function

The new Puppet Doc tool (a.k.a Puppet Strings) that will be released with Puppet 4.0.0 can produce documentation from functions written using the 4x function API. In the 3x API functions are documented with a Ruby String that is given in the call to create a function. The 4x API instead processes comments that are associated with the created function. This processing supports a set of YARD tags to make it possible to write documentation of higher quality. Tags for @param, @example, and @since are examples of such tags.

See the Puppet String project at github for examples and more information.

In the Next Post

In the next post I describe how you can pass code blocks to puppet functions and call them from within the function.