The Ruby side of Puppet - Part 2 - Custom Functions

This is the second post in a series of three, covering the concepts and best practices for extending Puppet using custom facts, custom functions and custom types and providers.

Part 1 covers Custom Facts, which explain how a node can provide information to the Puppet Server.

Part 2 (this post) focuses on Custom Functions, demonstrating how to develop functions for data processing and execution.

Part 3 will cover Custom Types and Providers, showing how to extend Puppet's DSL functionality.

---

Custom Functions in Puppet:

Functions in Puppet are executed on the Puppet server, specifically inside the compiler. These functions have access to all facts and Puppet variables, as long as they exist within the function’s namespace.

There are two main types of functions in Puppet:

• Statement functions
• Return value functions

Statement functions can interact with Puppet internals. The most common statement functions are include or contain, which add classes to the catalog, or noop and notice, which set all resources into simulation mode or print entries into the Puppet server log file.

Return value functions such as lookup or read_url, return values from Hiera or external sources like web servers.
Some return value functions can also manipulate data (each, filter, map, reduce).

Why Create Custom Functions?

Here are some reasons you might develop custom functions in Puppet:

• Writing a custom Hiera lookup function
• Converting data structures for specific needs

Content:

1. Function development
2. Functions in Modules
3. General API
4. Dispatcher and Define
5. Using functions in Puppet DSL
6. Summary

Function development

Since functions are executed only at compile time, it is advisable to first focus on their core functionality before integrating them with the Function API. This allows you to test the logic with local data, avoiding issues on the Puppet server that could break compilation or even crash the server.

For example, imagine you have a data structure that outlines various aspects of a system's configuration. Using a function, you can expose specific parts of this data structure as variables.

Example Data Structure:

server_hash:
  'postfix':
    'config':
      'transport': 'mta'
      'ssl': true
  'apache':
    'config':
      'vhost': 'server.domain.tld'
      'document_root': '/srv/www/html/server.domain.tld'
      'ssl': true
      'proxy_pass': 'http://localhost:8080'
  'tomcat':
    'config':
      'application': 'crm'
      'service': 'frontend'

Assume that this data structure comes from Hiera, and you want to allow other teams, who may write their own modules, to reuse this data without performing additional lookups. This would allow you to modify the data structure internally while still providing teams with the required information.

Example Function:

# test data
server_hash = {
  'postfix' => {
    'config' => {
      'transport' => 'mta',
      'ssl' => true
    }
  },
  'apache' => {
    'config' => {
      'vhost' => 'server.domain.tld',
      'document_root' => '/srv/www/html/server.domain.tld',
      'ssl' => true,
      'proxy_pass' => 'http://localhost:8080'
    }
  },
  'tomcat' => {
    'config' => {
      'application' => 'crm',
      'service' => 'frontend'
    }
  }
}
# function definition
def read_service_config(data, service)
  data[service]
end
# testing the function
puts read_service_config(server_hash, 'tomcat')

Functions in Modules

Puppet provides an API for creating custom functions. To ensure the Puppet agent can locate these functions, they must be placed in specific directories within a module.

Files inside the module's lib directory are synchronized across all agents. Note that it is not possible to limit which files are synchronized.

Custom functions can be added to one of two directory structures:

• lib/puppet/parser/functions - Functions API v1
• lib/puppet/functions - Functions API v2

The newer Functions API v2 supports subdirectories, allowing you to namespace your functions. The filename should match the function name.

In this post, we’ll focus on the modern API v2.

Example Directory Structure:

# <modulepath>/stdlib/
  \- lib/
    \- puppet/
      \- functions/
        \- stdlib/
          \- to_yaml.rb

# stdlib/lib/puppet/functions/stdlib/to_yaml.rb
Puppet::Functions.create_function(:'stdlib::to_yaml') do
  # ...
end

We recommend prefixing custom functions with the module name to avoid naming conflicts.

General API

To create custom functions in Puppet, use the following structure:

Example:

# <modulepath>/betadots/lib/puppet/functions/betadots/resolve.rb
Puppet::Functions.create_function(:'betadots::resolve') do
  # ...
end

Note that the namespace must be identical to the module name.

Dispatcher and Define

Within the do ... end of the function, you need to add a dispatcher. The dispatcher validates the input data and links it to a named definition, which executes the corresponding Ruby code.

The dispatcher can check data types using param and map values to Ruby symbols. The definition uses the dispatcher’s name and the parameters provided in the dispatcher.

The final command in the definition determines the return value.

Example Dispatcher for an IPv4 Address:

# <modulepath>/betadots/lib/puppet/functions/betadots/resolve.rb
Puppet::Functions.create_function(:'betadots::resolve') do
  dispatch :ip do
    param 'Regexp[/^(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})$/]', :ipaddr
  end
end

Here’s a table of common parameter methods used in dispatchers:

Method	Description
param or required_param	A mandatory argument. May occur multiple times. Position: All mandatory arguments must come first.
optional_param	An argument that can be omitted. You can use any number of these. When there are multiple optional arguments, users can only pass latter ones if they also provide values for the prior ones. This also applies to repeated arguments. Position: Must come after any required arguments.
repeated_params or optional_repeated_params	A repeatable argument, which can receive zero or more values. A signature can only use one repeatable argument. Position: Must come after any non-repeating arguments.
required_repeated_param	A repeatable argument, which must receive one or more values. A signature can only use one repeatable argument. Position: Must come after any non-repeating arguments.
block_param or required_block_param	A mandatory lambda (block of Puppet code). A signature can only use one block. Position: Must come after all other arguments.
optional_block_param	An optional lambda. A signature can only use one block. Position: Must come after all other arguments.

Adding a Definition:

Description	Dispatcher	Definition
Name	dispatch :ip	def ip
Parameter	param ..., :ipaddr	def ip(ipaddr)

# <modulepath>/betadots/lib/puppet/functions/betadots/resolve.rb
Puppet::Functions.create_function(:'betadots::resolve') do
  require 'resolv'
  dispatch :ip do
    param 'Regexp[/^(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})$/]', :ipaddr
  end

  def ip(ipaddr)
    Resolv.getname ipaddr
  end
end

Now additional dispatchers and definitions can be added:

# <modulepath>/betadots/lib/puppet/functions/betadots/resolve.rb
Puppet::Functions.create_function(:'betadots::resolve') do
  require 'socket'
  dispatch :ip do
    param 'Regexp[/^(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})$/]', :ipaddr
  end

  dispatch :dnsname do
    param 'Regexp[/.*/]', :dnsname
  end

  dispatch :local_hostname do
    param 'Regexp[//]', :local_hostname
  end

  def ip(ipaddr)
    Addrinfo.tcp(ipaddr, '80').getnameinfo[0]
  end

  def dnsname(dnsname)
    Addrinfo.ip(dnsname).getnameinfo[0]
  end

  def local_hostname
    Socket.gethostname
  end
end

Using functions in Puppet DSL

Within Puppet the functions usually gets executed during compile time.
A special case is the deferred function which gets executed on the agent.

Let's consider two use-cases:

1. using resolv function on the compiler
2. using resolv function on the agent

Use case 1: compiler execution

The function can be used anywhere in the code:

class betadots::resolver (
  Stdlib::Fqdn $local_hostname = betadots::resolve(),
) {
  $ip_from_google = betadots::resolve('www.google.com')
}

Use case 2: agent execution

The function must be declared to run in deferred mode in Puppet. Avoid using the function within the class header, as this could potentially overwrite the function call from hiera data.

class betadots::local_resolve (
) {
  $local_api_ip = Deferred('betadots::resolve', ['api.int.domain.tld'])
}

Summary

Puppet provides a stable API for creating custom functions.
Before creating a new function, check whether a solutions already exists in Stdlib- or Extlib-Module.

Remember to prefix your functions with your module name to avoid conflicts and help identify their origin.

Happy puppetizing,
Martin