Sys Admin Pocket Survival Guide

PUPPET

101

Host config/management using Puppet specify the "what". Essentially, simple YAML file to list all the desired settings the host need to have.
modules (puppet preconfig or 3rd party) takes these declarations and create commands that will be executed by the agent. These typically resides in /etc/puppet/environment/prd/modules/

Puppet is based on Ruby. Dev will likely involve writting snipplet of code based on ruby, and .erb template based on ruby's parsing.

Config

/etc/puppet/puppet.conf
/etc/puppet/manifest/site.pp
/etc/puppet/environment/prd/modules/common/ # classes/group defined here
/etc/puppet/environment/prd/modules/common/core/manifests/init.pp # set of common core settings eg default/core packages.
/etc/puppet/environment/prd/modules/common/srv_oracle/manifests/init.pp # define the classes, what other modules to include, etc.
/etc/puppet/data/enc/ $fqdn.yaml # what class a host is defined into
/etc/puppet/data/hiera/common/common.yaml # hiera store more variable stuff, users that belong to a class, etc
/etc/puppet/data/hiera/site/common.yaml # site specific stuff, like NFS mounts, root pw,
/etc/puppet/data/hiera/site/ $fqdn.yaml # very specific to a given host. probably little stuff here. eg specific user that may need sudo access. host specific mounts.

- what goes into ENC and what goes into Hiera??
  * modules dir has .pp files, 
  * hiera has .yaml
Looking at mountfs module, the .pp module files look at the .yaml and loop to add each mounts defined.
it just that the .pp uses lot of puppet construct to effect the changes on the host.  
how/where these are actually done, not sure... cuz it should be in a platform independent way...

Terminology

ENC: External Node Classifier. One possible methods to organize node (client host) data.
Other possibility include Hiera (for hierachy) and LDAP backed.
1. ENC is a script written in any lang. output is YAML.
2. only assign classes, top-scope var, and env for the node.
Hiera: a way to organize node variable and separate it from data in modules.
1. Optional, activate it in hiera.yaml. This define where all the places where YAML hierarchi are sourced (/etc/puppet/data/hiera, and the order sequence.
2. facts are case sensitive in hiera and template
3. http://docs.puppetlabs.com/hiera/1
YAML: typical output format for puppet.
1. YAML files start with three dashes (---);
2. use colons (:) to separate parameters from values,
3. hyphens (-) to separate multiple values (arrays).

Commands/Troubleshooting

# run on single machine


puppet apply -e 'notify { "I am $fqdn and have $processorcount CPU/core": }'

puppet agent		# this is the main daemon process on node, will be daemonized by def, 
			# but pull/cron mechanism don't need this agent to be running.
puppet agent -t		# should be able to see if environment is set to dev, test, or prd.  
puppet agent --no-daemonize --onetime --noop --test -v   # ? help get some info of what a run would do


puppet help		# list puppet subcommands
puppet status local	# YAML output of whether puppet is running and version info
puppet describe --list	# list of puppet resource types.  eg augeas, computer, sshkey, package...
			# good place to find out what clause and params are usable in .pp files

puppet describe sshkey

facter domain 		# depends on DNS reverse record
facter fqdn

PuppetDB

install as puppet module
...

Examples

Check for OS and version when applying NTP config

NTP definition is defined for the site in
/etc/puppet/data/hiera/usa/common.yaml ::

ntp_servers:
      - 'time1.us.nx.com'
      - 'time2.us.nx.com'
      - 'time1.eu.nx.com'

Then, in the puppet code, it checks for the OS and version, and decides how to apply these changes accordingly. Note where $osname and $osmajor are checked. This kind of check is coded in each class, eg srv_oracle, httpd, cups, etc.
/etc/puppet/environment/prd/modules/common/ntp/manifests/params.pp ::

class ntp::params($ntp_servers='', $ntp_allow_stepping='')
{
  $osmajor = $core::params::osmajor
  $osfamily = $core::params::osfamily

  if empty($ntp_servers) {
    fail('No NTP servers specified (hiera var ntp_servers)')
  }

  case $ntp_allow_stepping {
    'true','false': {}
    default: { fail("Invalid value for ntp_allow_stepping : ${ntp_allow_stepping}. Must be true or false") }
  }

  case $core::params::osname {
    'rhel','centos': {
      $service_name = 'ntpd'
      $conf_file    = '/etc/ntp.conf'
      $conf_file_template = "ntp.conf.${osfamily}.erb"
      $step = $ntp_allow_stepping ? { 'true' => '-x', 'false' => '' }

      case $osmajor {
        '5': {
          $ntp_options = strip("-u ntp:ntp -p /var/run/ntpd.pid ${step}")
          $package_list = 'ntp'
          $has_separate_ntpdate = 'false'
        }
        '6': {
          $ntp_options = strip("-u ntp:ntp -p /var/run/ntpd.pid ${step}")
          $package_list = ['ntp','ntpdate']
          $has_separate_ntpdate = 'true'
        }

        default: {
          fail("Module ${module_name} does not support ${::operatingsystem} ${::operatingsystemrelease}")
        }
      }
    }
    default: {
      fail("Module ${module_name} does not support ${::operatingsystem}")
    }
  }
}

Excluding specific app from installation

/etc/puppet/environment/prd/modules/common/core/manifest/package_blacklist.pp defines the code on how to handle blacklist, while
/etc/puppet/data/hera/common/common.yaml is an example of actual usage of declaring what is to be blacklisted. in this case, all servers (common) across all classes/sites exclude the redhat annoyware:

core_package_blacklist_rhel:
      - 'subscription-manager'

Explicitly call out and include other classes:

# this class implements a generic server
class srv_generic{
  include srv_common
  class {'core::puppet_facts': puppet_class => $module_name}
}

## the srv_common has a large list of include.
## for server conversion from CFEngine into puppet.
## may want to create a srv_bare that has minimal settings
## but then again, the basic site settings should be okay even if they were diff than pre-migration settings.

Mounts

Define a list of commont mount points, location and options for the site at
/etc/puppet/data/hiera/usem/common.yaml ::

---
mountfs_default_options:
  'nfs1.usa.nx.com.net_nfs': 'intr,tcp,rw,rsize=32768,wsize=32768,nfsvers=3,_netdev'
  'nfs2.usa.nx.com.net_nfs': 'intr,udp,rw,nfsvers=2,_netdev'

mountfs_mountpoints:
  'homedirs':
    mountpoint: '/nfs/nfshome'
    group: 'universal'
    type: 'static'
    source: 'nfs://nfs1.usa.nx.com/ifs/usa/nfshome'
  'oldhomedirs':
    mountpoint: '/nfs/oldhome'
    group: 'oldhome'
    type: 'static'
    source: 'nfs://netapp.nx.com/vol/oldhome'
  'ora_global_nfs':
    mountpoint: '/ora_global_nfs'
    group: 'srv_oracle'
    type: 'autofs'
    source: 'nfs://nfs2.usa.nx.com/ifs/usa/sync/ora_core'
    # the ora_global_nfs mount uses an existing class called srv_oracle, so 
    # any server defined in this class will get get this mount included.

# this below means all host on this site (using this common.yaml) will mount the "universal" group, 
# no need to define them in the host-specific hiera file.
mountfs_mount_by_group:
  - 'universal'

Then for machines that need to use such pre-defined mount points, at
/etc/puppet/data/hiera/usa/host/appsvr002.yaml ::

---
# define one off mounts manually, could be nfs or local hd
mountfs_mountpoints:
  appsvr002_apps_splunk:
    mountpoint: '/opt/splunk'
    type: 'static'
    source: 'ext4:///dev/mapper/vg_apps-lv_splunk'
    options: 'defaults'


mountfs_mount_by_group:
  - 'oldhome'
  # no need to spell out the inclusing of the 'universal' group mount

Realize virtual classes

Convention to use @ for virtual class name
realize is to instantiate/materialize the virtual class
/etc/puppet/environment/prd/modules/common/core/manifests/local_users.pp ::

class core::local_users
{
  $core_local_users=hiera_array('core_local_users',[])

  if size($core_local_users) > 0 {
    realize( User[$core_local_users] )
  }

  $set_oracle_password=hiera('core_local_users_oracle_reset_password','true')
  case $set_oracle_password {
    'true': { $oracle_password='!!' }
    'false': { $oracle_password=undef }
    default: { fail("Module ${module_name}: 'core_local_users_oracle_reset_password' must be set to 'true' or 'false'") }
  }

  @user {'oracle':
    ensure   => present,
    uid      => hiera('core_local_users_oracle_uid','666'),
    gid      => hiera('core_local_groups_dba_gid','666'),
    home     => '/u01/home/oracle',
    shell    => '/bin/ksh',
    password => $oracle_password,
    require  => [ Group['dba'], Package['ksh'] ]
  }

  # additional @user can be declared here
}

/etc/puppet/environment/prd/modules/common/srv_oracle/manifests/init.pp ::

class srv_middleware_oracle {
  $osmajor = $core::params::osmajor
  $default_vg_name = $core::params::default_vg_name
  case $::operatingsystem {
    'RedHat': {}
    'CentOS': {}
    default: {fail("Module ${module_name} does not support ${::operatingsystem}") }
  }
  # ...
  realize(User['oracle'])
  realize(Group['dba'])
}

Related tools

mrepo/mcollective
platform
foreman
git

Reference

Uphill, Thomas (2014-07-16). Mastering Puppet (p. 40). Packt Publishing. Kindle Edition.
tba
cfengine

Puppet (OpenSource) version 3.7.4
Last Updated: 2015.02.26

Copyright info about this work

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike2.5 License. Pocket Sys Admin Survival Guide: for content that I wrote, (CC) some rights reserved. 2005,2012 Tin Ho [ tin6150 (at) gmail.com ]
Some contents are "cached" here for easy reference. Sources include man pages, vendor documents, online references, discussion groups, etc. Copyright of those are obviously those of the vendor and original authors. I am merely caching them here for quick reference and avoid broken URL problems.

Where is PSG hosted these days?

http://tin6150.github.io/psg/psg2.html This new home page at github
http://tiny.cc/tin6150/ New home in 2011.06.
http://tin6150.s3-website-us-west-1.amazonaws.com/psg.html (coming soon)
ftp://sn.is-a-geek.com/psg/psg.html My home "server". Up sporadically.
http://tin6150.github.io/psg/psg.html
http://www.fiu.edu/~tho01/psg/psg.html (no longer updated as of 2007-05)