Orchestration and Puppet

What is orchestration?

When it comes to IT systems, the term "orchestration" refers either to a work flow process which is executed either on a single system or on multiple systems.
An example of a system based work flow process is application update. The process may look like this:

1. Stop application server service
2. Install the new version of the application software
3. Execute one-time or initial commands (e.g. create caches)
4. Start the application server on localhost
5. Run the scripts to verify functionality
6. Restart the application server service normally

Orchestration on multiple systems can refer to related systems such as application updates and DB schema updates on the DB server. To do this:

1. Run the above - mentioned commands on the application server (Steps 1 to 5)
2. Apply the schema update to the DB server
3. Execute step 6 on the application server

What is Puppet

Puppet is a configuration management solution which is based upon a client server architecture.
Each system which is managed by Puppet needs a Puppet agent.
The Puppet agent fetches its configuration description from the Puppet server.

The main focus of Puppet is the declarative configuration state which means that the user describes the final desired state of a system.
Within Puppet itself, work flow process (stop service, start service) contradict the desired state and are therefore not possible.

How do you do orchestration in Puppet?

Puppet offers work flow process control using Puppet Bolt.
Puppet bolt does not need an agent; instead it uses existing access methods (e.g. SSH or WinRM) or connects to the Puppet messaging client (pxp-agent) and Puppet Enterprise.

Puppet Bolt work flow process control using tasks

The required work flow processes (Tasks) are scripts (Shell, Python, Ruby, Powershell) and these scripts can be added to a Puppet module within tasks folder next to a JSON description file.

Example for an application update (without error handling):

# app/tasks/update.sh
#!/bin/bash

# Set script variables
version=$PT_version

# Stop of the application server service (regardless if systemd or sysv)
puppet resource service tomcat ensure=stopped

# Installation/Update of the application (regerdless if apt, yum or zypper)
puppet resource package app ensure=$version

# Run the initialization and test scripts
/usr/local/bin/app_init.py
/usr/local/bin/app_check.sh

# Start of the application server service
puppet resource service tomcat ensure=running

The JSON description file:

# app/tasks/update.json
{
  "description": "Update APP",
  "input_method": "environment",
  "parameters": {
    "version": {
      "description": "Version to install",
      "type": "String[1]"
    }
  }
}

This task can now be applied onto the target systems:

bolt task run app::update version='1.2.3' --targets <node1>, <node2> --modulepath <path to module>

Puppet Bolt server work flow process using plans

Tasks allow you to bring steps in order within a single system. For server work flow process control we can use Puppet Bolt and Plans.

Plans can be written in Puppet DSL or YAML; these execute Bolt tasks, scripts and commands on remote systems.

Plans must be deployed into the Puppet module plans folder.

An example for an application update including DB schema update (Puppet DSL):

# app/plans/update.pp
plan app::update (
  TargetSpec $appservers,
  TargetSpec $dbserver,
  String[1]  $version,
  Boolean    $update_schema=true,
){
  $update_result = run_task('app::update', $appservers, 'version' => $version)
  out::message($update_result)

  if $update_schema {
    $schema_result = run_task('app::schema', $dbserver, 'version' => $version)
    out::message($schema_result)
  }
}

For starting plans, one can also use Puppet Bolt:

bolt plan run app::update appservers='["<node1>","<node2>"] dbserver='<db>' version='1.2.3' update_schema=true

Alternatively a plan can be written in YAML:

steps:
  - task: app::update
    targets:
      - node1
      - node2
    parameters:
      version: '1.2.3'
  - task: app::schema
    targets:
      - db
    parameters:
      version: '1.2.3'
      update_schema: true

Summary

Puppet configuration management allows us to deploy and set up systems according to existing security guidelines in order to ensure compliance.
The Puppet Agent starts regularly and corrects configuration drifts.

Puppet Bolt is an orchestration tool which allows us to control work flow processes.