Templating with Embedded Puppet Programming Language - EPP

Next up in the long list of new features coming to the Puppet Programming Language i Puppet 3.5.0 is the support for EPP templates. EPP is to the Puppet Programming Language what ERB is to Ruby, and EPP supports the same template tags as ERB.

EPP Syntax

tag	description
<%	Switches to puppet mode
<%=	Switches to puppet expression mode. (Left trimming is not possible)
<%%	Literal <%
%%>	Literal %>
<%-	Trim Left When the opening tag is <%- any whitespace preceding the tag, up to and including a new line is not included in the output.
<%#	Comment A comment not included in the output (up to the next %>, or right trimming -%>). Continues in text mode after having skipped the comment (observes right trimming semantics).
%>	Ends puppet mode
-%>	Ends puppet mode and trims any generated trailing whitespace

In addition to these (ERB compatible) tags, the very first tag can specify a set of parameters that can be given to the template when it is used, much like parameters can be declared in a define. This is done by using this syntax:

<%-| $x, $y, $z='this is a default value' |-%>

That is, the same way as parameters are defined for a lambda expression.

EPP Functions

EPP template text is produced by the two epp functions:

• epp(filename, optional_args_hash) - for evaluating an .epp file
• inline_epp(epp_text_string, optional_args_hash) - for evaluating a string in EPP syntax

Puppet ERB vs. EPP

Basically, existing .erb templates can be changed to .epp and references to variables simply changed to use Puppet's $var notation instead of the Ruby instance variable notation @var. The programming logic in general is different in the Puppet Programming Language, but it is expected that most templates are not complex enough to require Ruby (since Puppet now supports iteration).

The main benefit of using EPP is that that the same syntax and semantics apply as elsewhere in the Puppet manifests, and there is no risk of calling out to Ruby in unsafe ways.

The new functions are also different in that they operate on one file/string. If you want to render multiple files into a single string, you can concatenate them using string interpolation, or the join() function from standard lib.

Visibility of Scoped Variables

A problem with Puppet ERB in general is that the template has full access to the invoking scope which makes it very difficult to write reusable templates (the exact same set of variables must be available in all scopes where the template is being used). This also leads to poor separation of concern - a change to the logic may affect templates that are in use, and templates may cause unintentional side effects.

In EPP the rules are therefore different than in ERB.

• Both inline_epp() and epp() always provide access to the global (i.e. top/node) scope variables.
• inline_epp() provides access to the local scope variables unless an (optional) hash of variable name/value entries is given in which case these are used instead of the local scope variables.
• If a template declares parameters that require a value to be set, these must be given in the name/value hash

While these rules may seem complicated at first, they are quite natural to use in practice. Think of an .epp file as a function that is called. Like all functions it has access to all global variables + the the values that are given to it when it is called (i.e. when epp() is called). When calling a function, it is good to specify the parameters it accepts instead of just blindly throwing it a set of variables - that way an error is raised instead of something just not rendering as expected.

For inline_epp(), the typical use is to render something from the calling scope. In this case nothing special needs to be declared (no parameter declaration, nor any passing of arguments). If you however plan to later move the template to a file, and you are just using it inline while trying things out then you want to both declare the parameters and call it with arguments even if this initially means unnecessary typing. The end result is that an inline epp works the same as a file based epp when argument are given to it.

Examples

The examples makes use of Puppet Heredoc to specify the template text.

$x = droid
notice inline_epp(@(END))
This is the <%= $x %> you are looking for!
| END

Produces a notice of the string "This is the droid you are looking for!"

$a = world
notice inline_epp(@(END), {x => magic})
  <%-| $x |-%>
  <% Integer[1,3].each |$count| { %>
  hello epp <%= $x %> <%= $a %> <%= $count %>
  <%- } %>
  |- END

Produces the following output:

Notice: Scope(Class[main]): 
hello epp magic world 1
hello epp magic world 2
hello epp magic world 3

(In the example above $a resulted in "world" because all of the logic is in the global scope).

EPP Template files

EPP template files must end with .epp, and they are placed in the same location as where you place .erb templates for use with Puppet. For testing purposes you can specify the location on the command line - below using the current directory:

puppet apply --parser future -e 'notice epp("foo.epp")' --templatedir .

(Obviously also placing the EPP source you want to test in the file foo.epp).

Summary

Puppet Templates are available when using the --parser future feature switch with Puppet 3.5.0 - The functions epp(), and inline_epp() provides EPP templating capabilities using the Puppet Programming Language as opposed to the Ruby based ERB based already existing template, and inline_template functions.

Heredoc is Here !

Starting with Puppet 3.5.0 with --parser future turned on you can now use Puppet Heredoc; basically a way to write strings of text without having to escape/quote special characters. The primary motivation for adding heredoc support to the Puppet Programming Language is to help avoiding the problem known as "backslash hell", where every backslash character in a string may require, two, four or more backslashes to pass an actual backslash through multiple layers of string special character interpretation.

Before talking about the features of Puppet Heredoc, lets look at an example:

 $a = @(END)
 This is the text that gets assigned to $a.
 And this too.
 END

As you probably already figured out, the @() is the heredoc start tag, where you get to define what the end tag is; a text string that marks where the verbatim sequence of text on the lines following the start tag ends. In the example above, the end tags is END. (Obviously you have to select an end tag that does not appear as a separate line inside the actual text).

This blog post is a brief introduction of the Puppet Heredoc features, the full specification is found in the Puppet Heredoc ARM-4 text.

Trying out the examples

If you want to try out the examples, you need Puppet 3.5.0 and then turn on --parser future.

Controlling the Left Margin

A problem with Heredoc is how to deal with text that appears in indented text, but you do not want the indentation in the resulting string.

if $something {
  $a = @(END)
  Text here is indented 2 spaces.
  END
}

Puppet Heredoc solves this by allowing you to define where the left margin is by using a pipe | character on the end-tag line at the position where the first character on each line should be. To fix the example above, we then write:

if $something {
  $a = @(END)
  Text here is not indented 2 spaces.
  | END
}

Controlling trailing new-line

Another problem with heredoc text is how to deal with the line ending of the last line of text (and any trailing whitespace on that line). With Puppet Heredoc you can easily strip out trailing space and the newline by using a - before the end tag. (You can combine the - with | by placing the - after the pipe).

Here is the same example again, now without trailing new-line in the result:

if $something {
  $a = @(END)
  Text here is not indented 2 spaces, and has no newline.
  |- END
}

Interpolating variables

The default mode of Puppet Heredoc is to not interpolate variables (e.g. having $a in the heredoc text does not expand to the value of the variable $a). If you need this, it is possible to turn on interpolation by double quoting the specification of the end tag.

$a = world
notice @("END")
  The $a is an awesome place
  |- END

Will output "The world is an awesome place".

Naturally, since there also needs to be a way to enter a $, escaping is turned on for $ and for \. Thus when using interpolation, a \ must be entered as \\, and a $ as \$.

You can use both styles of interpolation; either just $a, or ${a}. The same rules for interpolation of expression as for double quoted strings apply.

Controlling Special Character Escapes

By default, all character escapes are turned off (when using interpolation, escapes for \ and $ are turned on). Puppet Heredoc also allows you to control escapes in more detail. The possible escapes are t, s, r, n, u, L, and $, and you can control these individually by specifying them in the heredoc start tag like this:

$a = @(END/tL)
  This text has a tab\t and joins this line \
  with this line.
  |-END

Most of the escapes should be familiar, except the L escape which makes it possible to escape the end of line thus effectively joining a line with the next. The charters may appear in any order in the spec. Using one (or more) escapes also always turn on escaping of \.

Specifying the Syntax

The Puppet Heredoc start tag allows specification of the syntax of the contained text. This is done by following the end tag name with ':', and the syntax/language specification as a mime specification string following the ':'. Here is an example:

$a = @(END:json)
["a"]
- END 

The syntax/language tag serves dual purpose; it is an indicator to tools (such as Geppetto) how the tool should perform things like syntax highlighting or syntax checking, and it enables the Puppet Parser to perform syntax checking if there is a plugin that checks the given syntax.

In Puppet 3.5.0, there is a syntax checker for Json, and consequently, if you were to enter the following example, you will see it report the Json syntax error.

$a = @(END:json)
['a']
- END 

You will get the following error:

Error: Invalid produced text having syntax: 'json'. JSON syntax checker: Cannot parse invalid JSON string. "unexpected token in array at ''a']'!"

New syntax checkers can be written in Ruby, and distributed as a Puppet Module. (This will be the topic of a future blog post).

Summary

This blog post is an introduction to Puppet Heredoc. There are some additional features that are documented in the full ARM text, such as how to use multiple heredocs on the same line, the precise semantics of special character escapes and margin control, the details about what is permissible as an end-tag etc.

What better than to end with some poetry...

notice @(Verse 8 of The Raven)
  Then this ebony bird beguiling my sad fancy into smiling,
  By the grave and stern decorum of the countenance it wore,
  `Though thy crest be shorn and shaven, thou,' I said, `art sure no craven.
  Ghastly grim and ancient raven wandering from the nightly shore -
  Tell me what thy lordly name is on the Night's Plutonian shore!'
  Quoth the raven, `Nevermore.'
  | Verse 8 of The Raven