<The CFEngine Blog

Discovering SSL Certificates and Expirations

Posted by Nick Anderson
June 19, 2017

With more and more services using SSL keeping track of the certificates in use across a global infrastructure can be challenging. The inventory reporting features in CFEngine Enterprise can be useful in identifying services using SSL as well as when their certificates will expire. cf-monitord provides lists of ports that are listening. We can use openssl to connect to each listening port and if successful we can extract the certificate information for inventory. We won’t be able to find ALL certificates like this. This policy only covers up-front SSL/TLS. From Serverfault:

For up-front SSL/TLS, you can check whether it will accept a TLS ClientHello (i.e. be a TLS server from the start of the connection), but using echo "" | openssl s_client -connect hostname:port (echo "" | is optional, it will just stop openssl as soon as it has established the connection, as you probably don’t want to send anything specific). For “upgraded” SSL/TLS connections, done after a command at the application protocol level (such as STARTTLS), this can be trickier. You can do this for by adding -starttls the_name_of_the_protocol to this openssl s_client command. According to the OpenSSL documentation, “Currently, the only supported [protocol names] are “smtp”, “pop3”, “imap”, and “ftp””. This won’t help you for LDAP (if configured to use Start TLS and not up-front TLS), MySQL, PostgreSQL, … For these, you may simply have to look into their respective configuration files. Automating this process would require a tool that can understand all these protocols, which can be quite difficult.

Additionally, it will not discover SNI certificates because the probe does not provide the server name to in the probe. While there is no silver bullet this policy can aid in discovering a significant number of ssl certificates in use along with their expiration’s and it can be expanded with additional detection mechanisms as well. Some things we would like to collect:

  • Unix formatted date when certificate will expire
  • Human readable and lexically sort-able formatted date (ISO 8601)
  • Ports using SSL
  • Days until port certificate expires

Additionally it would be nice to have a class defined if the number of days until the certificate expiration is less than some threshold. The threshold should be configurable without modifying policy.

Listing 1: services/ssl_port_inventory/main.cf

bundle agent ssl_port_inventory
# @brief Discover and inventory the ports and certificate expiration date where
# SSL seems to be in use.
#
# **Example Augments:**
#
# ```
# {
#   "classes": {
#      "autorun_ssl_port_inventory": [ "any" ],
#       "services_autorun": [ "any" ]
#   },
#   "inputs": [ "services/ssl_port_inventory/main.cf" ],
#   "vars": {
#       "ssl_port_inventory": {
#          "thresholds" {
#            "warning": {
#              "443": "90",
#              "8443": "90",
#            }
#            "critical": {
#              "5308": "365"
#            }
#          }}
#   }
# }
# ```
{
  meta:

  autorun_ssl_port_inventory::
    "tags" slist => { "autorun" };

  vars:

      "ports"
         slist => { @(mon.listening_ports) };

      "regextract"
        string => "notBefore=(?<begin_M>\w+)\s+(?<begin_D>\d+)\s+\S+\s+(?<begin_Y>\d+)\s+.*notAfter=(?<end_M>\w+)\s+(?<end_D>\d+)\s+\S+\s+(?<end_Y>\d+)\s+.*";

      "timeout" string => ifelse( fileexists("/usr/bin/timeout"), "/usr/bin/timeout", "/bin/timeout" );

      "now" string => strftime("gmtime", "%s", now() );

  classes:

      "found_monitord_listening_ports"
        expression => isvariable( "mon.listening_tcp4_ports" );

    found_monitord_listening_ports::

      "port_$(ports)_ssl_connected"
        expression => returnszero("$(timeout) 1 /usr/bin/openssl s_client -connect localhost:$(ports) 2>/dev/null | $(timeout) 1 /usr/bin/openssl x509 -noout 2>/dev/null", "useshell");

  vars:

    "port_$(ports)_ssl_connected"::

     # Example string we are parsing:
     # notBefore=Jun 14 15:54:23 2017 GMT\nnotAfter=Jun 12 15:54:23 2027 GMT

      "date_$(ports)"
        data => data_regextract( $(regextract),
                    execresult("$(timeout) 1 /usr/bin/openssl s_client -connect 127.0.0.1:$(ports) 2>/dev/null | /usr/bin/openssl x509 -noout -dates 2>/dev/null", useshell ) ),
        unless => isvariable( $(this.promiser) );

      # @var ssl_$(ports) The ports that have been found using SSL. **Example:** `[443, 5308]`
      "ssl_$(ports)"
        string => "$(ports)",
        meta => { "inventory", "attribute_name=SSL Ports"},
        if => isvariable( "data_$(ports)" ),
        unless => isvariable( $(this.promiser) );

      # @var inventory_$(ports)_ssl_expiration_unix Date in unix format. **Example:** `1497715408`
       "inventory_$(ports)_ssl_expiration_unix"
         string => execresult("/bin/date -d'$(date_$(ports)[end_M]) $(date_$(ports)[end_D]) $(date_$(ports)[end_Y])' +%s", useshell),
         meta => { "inventory", "attribute_name=Port $(ports) SSL Expires UNIX" },
         unless => isvariable( $(this.promiser) );

      # @var inventory_$(ports)_ssl_expiration_iso_8601 Date in ISO 8601 format (YYYY-MM-DD). **Example:** `2017-06-17`
      "inventory_$(ports)_ssl_expiration_iso_8601"
        string => execresult("/bin/date -d'$(date_$(ports)[end_M]) $(date_$(ports)[end_D]) $(date_$(ports)[end_Y])' +%Y-%m-%d", useshell),
        meta => { "inventory", "attribute_name=Port $(ports) SSL Expires iso 8601" },
        unless => isvariable( $(this.promiser) );

      # @var inventory_$(ports)_ssl_days_until_expiration the number of days until the certificate on `port` expires. **Example:** `05`
      "inventory_$(ports)_ssl_days_until_expiration"
        string => format( "%d", eval("($(inventory_$(ports)_ssl_expiration_unix)-$(now))/86400", math, infix)),
        meta => { "inventory", "attribute_name=Days until $(ports) SSL Expires" },
        unless => isvariable( $(this.promiser) );

      # { "vars": { "ssl_port_inventory": { "thresholds": "warning": { "443": "30" }}}}
      # $(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)])
      "thresholds" slist => getindices( "def.$(this.bundle)[thresholds]" );

  classes:

      # @class ssl_port_inventory_port_$(ports)_threshold_$(thresholds) is defined if inventory_$(ports)_ssl_days_until_expiration < def.ssl_port_inventory[$(thresholds)][$(ports)]
      "$(this.bundle)_port_$(ports)_threshold_$(thresholds)"
        meta => { "$(this.bundle)_threshold", "report" },
        expression => islessthan( "$(inventory_$(ports)_ssl_days_until_expiration)", "$(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)])" );

  vars:
      "_threshold_c" slist => classesmatching( ".*", "$(this.bundle)_threshold" );

  reports:
    !found_monitord_listening_ports::
      "ERROR: Unable to find open ports $(this.bundle) requires cf-monitord to be running.";

    DEBUG|DEBUG_ssl_port_inventory::
      "Found threshold definition: Port $(ports) $(thresholds) if there are less than $(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)]) days until the certificate expires";

      "Detected $(ports) is using SSL because of successful connection using openssl s_client"
        if => "port_$(ports)_ssl_connected";

      "Certificate on port $(ports) expires on $(inventory_$(ports)_ssl_expiration_iso_8601) in $(inventory_$(ports)_ssl_days_until_expiration) days";

      "Threshold Class: $(_threshold_c)";
}

Going through the specifics of the policy: The policy is tagged for autorun if autorun_ssl_port_inventory is defined.

Listing 2: Autorun capable policy

meta:

autorun_ssl_port_inventory::
  "tags" slist => { "autorun" };

Next we define some variables for re-use. Notably we get the list of ports we should be monitoring from mon.listening_ports which is provided by cf-monitord. If cf-monitord isn’t running then the agent will not have any ports to monitor. We also define regextract as our pattern to parse the result of our probe with openssl later, timeout as the path to the gnu timeout utility to prevent commands from getting stuck and the current time in unix format for use in later comparisons.

Listing 3: Define variables for re-use

vars:

    "ports"
       slist => { @(mon.listening_ports) };

    "regextract"
      string => "notBefore=(?<begin_M>\w+)\s+(?<begin_D>\d+)\s+\S+\s+(?<begin_Y>\d+)\s+.*notAfter=(?<end_M>\w+)\s+(?<end_D>\d+)\s+\S+\s+(?<end_Y>\d+)\s+.*";

    "timeout" string => ifelse( fileexists("/usr/bin/timeout"), "/usr/bin/timeout", "/bin/timeout" );

    "now" string => strftime("gmtime", "%s", now() );

Here we define a class if we have found the ports listening provided by cf-monitord and if we have ports we probe them with openssl s_client -connect and define classes if they appear to be speaking SSL.

Listing 4: Determine which ports are SSL enabled

classes:

    "found_monitord_listening_ports"
      expression => isvariable( "mon.listening_tcp4_ports" );

  found_monitord_listening_ports::

    "port_$(ports)_ssl_connected"
      expression => returnszero("$(timeout) 1 /usr/bin/openssl s_client -connect localhost:$(ports) 2>/dev/null | $(timeout) 1 /usr/bin/openssl x509 -noout 2>/dev/null", "useshell");

When we know which ports are speaking ssl we begin to extract the data provided by our probe.

Listing 5: Parse certificate information for SSL ports

vars:

  "port_$(ports)_ssl_connected"::

   # Example string we are parsing:
   # notBefore=Jun 14 15:54:23 2017 GMT\nnotAfter=Jun 12 15:54:23 2027 GMT

    "date_$(ports)"
      data => data_regextract( $(regextract),
                  execresult("$(timeout) 1 /usr/bin/openssl s_client -connect 127.0.0.1:$(ports) 2>/dev/null | /usr/bin/openssl x509 -noout -dates 2>/dev/null", useshell ) ),
      unless => isvariable( $(this.promiser) );

We inventory each port that speaks SSL and then we inventory the expiration dates for each port in our desired formats.

Listing 6: Inventory SSL Ports and expiration dates in multiple formats

# @var ssl_$(ports) The ports that have been found using SSL. **Example:** `[443, 5308]`
"ssl_$(ports)"
  string => "$(ports)",
  meta => { "inventory", "attribute_name=SSL Ports"},
  if => isvariable( "data_$(ports)" ),
  unless => isvariable( $(this.promiser) );

# @var inventory_$(ports)_ssl_expiration_unix Date in unix format. **Example:** `1497715408`
 "inventory_$(ports)_ssl_expiration_unix"
   string => execresult("/bin/date -d'$(date_$(ports)[end_M]) $(date_$(ports)[end_D]) $(date_$(ports)[end_Y])' +%s", useshell),
   meta => { "inventory", "attribute_name=Port $(ports) SSL Expires UNIX" },
   unless => isvariable( $(this.promiser) );

# @var inventory_$(ports)_ssl_expiration_iso_8601 Date in ISO 8601 format (YYYY-MM-DD). **Example:** `2017-06-17`
"inventory_$(ports)_ssl_expiration_iso_8601"
  string => execresult("/bin/date -d'$(date_$(ports)[end_M]) $(date_$(ports)[end_D]) $(date_$(ports)[end_Y])' +%Y-%m-%d", useshell),
  meta => { "inventory", "attribute_name=Port $(ports) SSL Expires iso 8601" },
  unless => isvariable( $(this.promiser) );

# @var inventory_$(ports)_ssl_days_until_expiration the number of days until the certificate on `port` expires. **Example:** `05`
"inventory_$(ports)_ssl_days_until_expiration"
  string => format( "%d", eval("($(inventory_$(ports)_ssl_expiration_unix)-$(now))/86400", math, infix)),
  meta => { "inventory", "attribute_name=Days until $(ports) SSL Expires" },
  unless => isvariable( $(this.promiser) );

Next we define classes for any tripped thresholds as defined in def.ssl_port_inventory data structure in the Example Augments section of the file.

Listing 7: Define threshold based classes

# { "vars": { "ssl_port_inventory": { "thresholds": "warning": { "443": "30" }}}}
    # $(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)])
    "thresholds" slist => getindices( "def.$(this.bundle)[thresholds]" );

classes:

    # @class ssl_port_inventory_port_$(ports)_threshold_$(thresholds) is defined if inventory_$(ports)_ssl_days_until_expiration < def.ssl_port_inventory[$(thresholds)][$(ports)]
    "$(this.bundle)_port_$(ports)_threshold_$(thresholds)"
      meta => { "$(this.bundle)_threshold", "report" },
      expression => islessthan( "$(inventory_$(ports)_ssl_days_until_expiration)", "$(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)])" );

And finally we collect information about any threshold classes defined and report information if relevant.

Listing 8: Report the results

vars:
    "_threshold_c" slist => classesmatching( ".*", "$(this.bundle)_threshold" );

reports:
  !found_monitord_listening_ports::
    "ERROR: Unable to find open ports $(this.bundle) requires cf-monitord to be running.";

  DEBUG|DEBUG_ssl_port_inventory::
    "Found threshold definition: Port $(ports) $(thresholds) if there are less than $(def.ssl_port_inventory[thresholds][$(thresholds)][$(ports)]) days until the certificate expires";

    "Detected $(ports) is using SSL because of successful connection using openssl s_client"
      if => "port_$(ports)_ssl_connected";

    "Certificate on port $(ports) expires on $(inventory_$(ports)_ssl_expiration_iso_8601) in $(inventory_$(ports)_ssl_days_until_expiration) days";

    "Threshold Class: $(_threshold_c)";

We can then report on the SSL information inventoried and build dashboards with alerts based on the thresholds.

Try CFEngine Enterprise for free

Sign up for CFEngine Enterprise and connect up to 25 hosts for free. No credit card required. Get started in minutes.

Try enterprise for free