Table of Contents
Fixing Puppet Certificates
See main article on Puppet for more information
There are a number of ways that we can reach a state where identity certificates between a client and the Puppet master are broken and the master cannot validate the client's identity. There are also a number of different ways to fix this problem, but the simplest way to handle this situation is to
- Delete all certs from the client
- Clean the client certs from the master
Fixing the Problem
Automatically
There are new scripts available to fix certificate problems or issue/sign a certificate for the first time in /sw/pkgs/scripts/puppet
:
Initial puppet setup can be performed by either the ubuntu-puppetize.sh
or centos-puppetize.sh
scripts:
[root@centos-host ~]# bash /sw/pkgs/scripts/puppet/centos-puppetize.sh
For a client that already has the puppet packages installed, a cert can be issued/reissued using the script puppet-reissue-cert.sh
Scripts should be called with bash
instead of run directly (eg bash script.sh
instead of ./script.sh
Manually
Client Cert
Identifying problems with Puppet almost always start by running puppet agent -t
from the client. In most cases, Puppet is nice enough to tell you both the problem and an idea of how to fix it in red text. Let's look at an example for the host labsrv03
:
[root@labsrv03 ~]# puppet agent -t Error: Could not request certificate: The certificate retrieved from the master does not match the agent's private key. Certificate fingerprint: 24:A4:1A:7A:87:0C:83:31:3C:F5:E5:82:8D:C3:54:F8:31:F0:78:F3:9C:27:D3:3A:94:51:4F:69:E4:9E:37:90 To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certificate. On the master: puppet cert clean labsrv03.cs.virginia.edu On the agent: 1a. On most platforms: find /etc/puppetlabs/puppet/ssl -name labsrv03.cs.virginia.edu.pem -delete 1b. On Windows: del "\etc\puppetlabs\puppet\ssl\certs\labsrv03.cs.virginia.edu.pem" /f 2. puppet agent -t
So here, our certificate name is labsrv03.cs.virginia.edu
and the instructions listed in the output are exactly what we want to do.
From the Puppet master we want to run:
[root@coresrv04 ~] puppet cert clean labsrv03.cs.virginia.edu
But on the client, let's go one step further and simply delete the entire /etc/puppetlabs/puppet/ssl
directory:
[root@labsrv03 ~] rm -rf /etc/puppetlabs/puppet/ssl
The reason we want to remove this whole directory is that it's possible to have additional certs in /etc/puppetlabs/puppet/ssl
that will cause more problems. Removing everything off the bat might save us a step.
Signing new Certs
Now we need to generate a new certificate request from the client and sign it from the master. On the client, run
[root@labsrv03 ~]# puppet agent -t Info: Creating a new SSL key for labsrv03.cs.virginia.edu Info: Caching certificate for ca Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml Info: Creating a new SSL certificate request for labsrv03.cs.virginia.edu Info: Certificate Request fingerprint (SHA256): A5:81:1A:9F:32:3C:F6:CA:B9:C6:7E:BD:38:CB:42:46:B4:11:1D:5D:82:41:34:32:AE:75:C4:42:48:F9:45:B1 Info: Caching certificate for ca Exiting; no certificate found and waitforcert is disabled
Then from the master, sign the new certificate request:
[root@coresrv04 ~]# puppet cert sign labsrv03.cs.virginia.edu Signing Certificate Request for: "labsrv03.cs.virginia.edu" (SHA256) A5:81:1A:9F:32:3C:F6:CA:B9:C6:7E:BD:38:CB:42:46:B4:11:1D:5D:82:41:34:32:AE:75:C4:42:48:F9:45:B1 Notice: Signed certificate request for labsrv03.cs.virginia.edu Notice: Removing file Puppet::SSL::CertificateRequest labsrv03.cs.virginia.edu at '/etc/puppetlabs/puppet/ssl/ca/requests/labsrv03.cs.virginia.edu.pem'
Verify on Client
Then from the client, run the agent one more time:
[root@labsrv03 ~]# puppet agent -t Info: Caching certificate for labsrv03.cs.virginia.edu Info: Caching certificate_revocation_list for ca Info: Caching certificate for labsrv03.cs.virginia.edu Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter]/ensure: created Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/facter_dot_d.rb]/ensure: defined content as '{md5}6f1934f8e129e77a0ed86c9da1174422' Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/nvidia_fact.rb]/ensure: defined content as '{md5}1ba61c941a715e8950e44a16cdd544ea' Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/package_provider.rb]/ensure: defined content as '{md5}12d57e65f9b90c13e7808c347fbdc0b0' Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/pe_version.rb]/ensure: defined content as '{md5}245bb8ea121e8a55e1a38668cc480530' ... ... Info: Stage[main]: Unscheduling all events on Stage[main] Notice: Applied catalog in 71.65 seconds