Hashicorp Nomad to Renew your TLS Certificates

• Prepare the Host
• Nomad Job File
• Run the Job

All my web applications are configured to use a containerized NGINX ingress for the TLS termination. Before I was using a simple Cron Jobs to keep my certificates updated. Now, since I moved all my apps into Nomad/Consul clusters, I wanted to see if I cannot use the option to periodically start a Nomad service to trigger a Certbot Renewal with the EXEC_RAW driver.

Prepare the Host

First, download the Let’s Encrypt client, certbot:

apt update
apt install certbot

Run the following command to generate certificates:

certbot -d example.com -d www.example.com
certbot certificates

Additionally, I will have to enable the EXEC_RAW plugin in my Nomad Client Configuration:

/etc/nomad.d/client.hcl

plugin "raw_exec" {
  config {
    enabled = true
  }
}

Nomad Job File

I currently do not know how to test for whether the certificate was actually renewed. So I cannot add second task to the job below that restarts the Docker service if this case is true. But since all my apps are monitored using Zabbix I will be prompted once a used certificate reaches the end of it's lifespan. So I can manually restart Docker to start using the updated certificate:

job "myapp_ingress_cert" {

    periodic {
        # run every day @5:55am
        cron             = "55 5 * * *"
    }
    type = "batch"
    reschedule {
      attempts  = 0
      unlimited = false
    }
    datacenters = ["mydatacenter"]
    
    group "myapp-ingress-cert" {

        task "cert-renewal" {
            driver = "raw_exec"

            config {
                command = "/usr/bin/certbot"
                args    = ["renew", "--quiet", "--no-self-upgrade"]
            }
        }
    }
}

UPDATE:

Solution for non-interactive certonly renewal:

job "myapp_ingress_cert" {

    periodic {
        # run every day @5:55am
        cron             = "55 5 * * *"
    }
    type = "batch"
    reschedule {
      attempts  = 0
      unlimited = false
    }
    datacenters = ["mydatacenter"]
    
    group "myapp-ingress-cert" {

        task "cert-renewal" {
            driver = "raw_exec"

            config {
                command = "/usr/bin/certbot"
                args    = ["certonly", "--quiet", "--noninteractive", "--standalone", "--cert-name", "my.domain.com"]
            }
        }
    }
}

Run the Job

nomad plan myapp_ingress_cert.tf

+ Job: "myapp_ingress_cert"
+ Task Group: "myapp-ingress-cert" (1 create)
  + Task: "cert-renewal" (forces create)

Scheduler dry-run:
- All tasks successfully allocated.
- If submitted now, next periodic launch would be at 2022-11-14T05:55:00Z (22h52m3s from now).

nomad job run -check-index 0 myapp_ingress_cert.tf

Job registration successful
Approximate next launch time: 2022-11-14T05:55:00Z (22h51m52s from now)