Skip to main content

Hashicorp Terraform - Docker Provider 2024

Shen Zhen, China

Back to Terraform!

I have been experimenting with Terraform as a provider for containerized apps before but got distracted... this is probably going to be largely a repeat of what I did last time... so let's see where I left off.

Installation Linux

I already installed Terraform but it is now outdated:

terraform version
Terraform v1.6.2
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.7.1. You can update by downloading from https://www.terraform.io/downloads.html

So let's update:

wget https://releases.hashicorp.com/terraform/1.7.1/terraform_1.7.1_linux_amd64.zip
wget https://releases.hashicorp.com/terraform/1.7.1/terraform_1.7.1_SHA256SUMS
sha256sum terraform_1.7.1_linux_amd64.zip
sha256sum -c terraform_1.7.1_SHA256SUMS

> terraform_1.7.1_linux_amd64.zip: OK
unzip terraform_1.7.1_linux_amd64.zip
rm terraform_1.7.1_linux_amd64.zip

sudo mv terraform /usr/bin/terraform
terraform version
Terraform v1.7.1
on linux_amd64

Get Started - Docker

Build, change, and destroy Docker infrastructure using Terraform. Step-by-step, command-line tutorials will walk you through the Terraform basics for the first time.

Hello World

Create a file main.tf inside a sub-dir (all job files need to be located in their own directory):

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.2"
    }
  }
}

provider "docker" {}

resource "docker_image" "nginx" {
  name         = "nginx:latest"
  keep_locally = false
}

resource "docker_container" "nginx" {
  image = docker_image.nginx.image_id
  name  = "tutorial"
  ports {
    internal = 80
    external = 8000
  }
}
terraform fmt
> main.tf

terraform validate
> Error: Missing required provider
> 
> This configuration requires provider registry.terraform.io/kreuzwerker/docker, but that provider > isn't available. You may be able to install it automatically by running: terraform init
terraform init

> Initializing provider plugins...
> - Finding kreuzwerker/docker versions matching "~> 3.0.1"...
> - Installing kreuzwerker/docker v3.0.2...
> - Installed kreuzwerker/docker v3.0.2 (self-signed, key ID BD080C4571C6104C)

> Terraform has been successfully initialized!
terraform validate
> Success! The configuration is valid.

Provision the NGINX server container with apply. When Terraform asks you to confirm, type yes and press ENTER:

terraform apply
> Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Inspect the current state using:

terraform show
terraform state list
> docker_container.nginx
> docker_image.nginx

Run docker ps to view the NGINX container running in Docker via Terraform:

docker ps

CONTAINER ID   IMAGE          STATUS         PORTS                  NAMES
dbf501b456a2   a8758716bb6a   Up 3 minutes   0.0.0.0:8000->80/tcp   tutorial
curl localhost:8000

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...

Deployment Variables

Change the docker_container.nginx resource under the provider block in main.tf by replacing the ports.external value of 8000 with 8888 and choose a different name for the container. But to make it a bit more interesting ~ let's use variables to define default values.

Create a new file called variables.tf with a block defining our variables:

variable "ingress_image_version" {
    description = "Version of the NGINX ingress image"
    type = string
    default = "nginx:latest"
}

variable "ingress_http_external" {
    description = "External http port of the NGINX ingress"
    type = number
    default = 8080
}

variable "ingress_http_internal" {
    description = "Internal http port of the NGINX ingress"
    type = number
    default = 80
}

variable "ingress_container_name" {
    description = "Name of the NGINX ingress container"
    type = string
    default = "ingress"
}

These values will now be used if no value is provided, e.g. terraform apply -var "ingress_image_version=nginx:alpine"

resource "docker_image" "nginx" {
  name         = var.ingress_image_version
  keep_locally = false
}

resource "docker_container" "nginx" {
  image = docker_image.nginx.image_id
  name  = var.ingress_container_name
  ports {
    internal = var.ingress_http_internal
    external = var.ingress_http_external
  }
}

After changing the configuration, run terraform apply again to see how Terraform will apply this change to the existing resources:

terraform validate
> Success! The configuration is valid.

terraform apply -var "ingress_image_version=nginx:alpine"
> Apply complete! Resources: 2 added, 0 changed, 2 destroyed.
docker ps

CONTAINER ID   IMAGE          STATUS         PORTS                   NAMES
f6e5cfed277c   a8758716bb6a   Up 17 seconds  0.0.0.0:8888->80/tcp    ingress
docker images
REPOSITORY           TAG       IMAGE ID       CREATED        SIZE
nginx                alpine    2b70e4aaac6b   3 months ago   42.6MB

Variables by String Input

Remove the default value to force a user input when the job is started:

variable "ingress_http_external" {
    description = "External http port of the NGINX ingress"
    type = number
}
terraform apply

var.ingress_http_external
  External http port of the NGINX ingress

  Enter a value:

.tfvsrs File

Instead of specifying the variables inside the variables file we can create a terraform.tfvars file with all the information. We still need to instantiate the variable inside variables.tf:

variable "ingress_image_version" {
    description = "Version of the NGINX ingress image"
    type = string
}

variable "ingress_http_external" {
    description = "External http port of the NGINX ingress"
    type = number
}

variable "ingress_http_internal" {
    description = "Internal http port of the NGINX ingress"
    type = number
}

variable "ingress_container_name" {
    description = "Name of the NGINX ingress container"
    type = string
} 

But all the user editable information will be inside the terraform.tfvars file:

ingress_image_version="nginx:latest"
ingress_http_external=8888
ingress_http_internal=80
ingress_container_name= "ingress"

Environment Variables

To avoid leaking - e.g. credentials - to your source management system you can use environment variables instead of adding those values to your tf code. In the following example I edited the hostname and external port that Docker should use for my container:

resource "docker_container" "nginx" {
  image    = docker_image.nginx.image_id
  name     = var.container_name
  hostname = var.HOSTNAME
  ports {
    internal = 80
    external = var.EXT_PORT
  }
}

These variables need to be declared in the variables.tf file:

variable "HOSTNAME" {
  description = "Name of the Docker host"
  type        = string
}

variable "EXT_PORT" {
  description = "External port forwarded to the ingress container"
  type        = string
}

Now export the values you want to set for those variables from your terminal:

export TF_VAR_HOSTNAME=docker_hostname
export TF_VAR_EXT_PORT=7777

Apply the changes and verify that the container is now using the new external port:

terraform apply
docker ps

CONTAINER ID   IMAGE          STATUS        PORTS                   NAMES
5cd4ef649771   bc649bab30d1   Up 5 seconds  0.0.0.0:7777-->80/tcp   nginx-ingress

Variables Precedence

  1. -var and -var-file option for terraform apply
  2. *.auto.tfvars | *.auto.tfvars.json
  3. terraform.tfvars.json
  4. terraform.tfvars
  5. Environment variables

Query Data with Outputs

Create a file called outputs.tf and add the configuration below to define outputs for your container's ID and the image ID:

output "container_id" {
  description = "ID of the Docker container"
  value       = docker_container.nginx.id
}

output "image_id" {
  description = "ID of the Docker image"
  value       = docker_image.nginx.id
}

You must apply this configuration before you can use these output values. Apply your configuration now. Respond to the confirmation prompt with yes:

terraform apply

Apply complete! Resources: 2 added, 0 changed, 2 destroyed.

Outputs:

container_id = "bec760237e437fb6ed8fe31c57fbfa5b3620f5684e7687d853c86a7049161907"
image_id = "sha256:a8758716bb6aa4d90071160d27028fe4eaee7ce8166221a97d30440c8eac2be6nginx:latest"

Terraform prints output values to the screen when you apply your configuration. Query the outputs with the terraform output command.

container_id = "bec760237e437fb6ed8fe31c57fbfa5b3620f5684e7687d853c86a7049161907"
image_id = "sha256:a8758716bb6aa4d90071160d27028fe4eaee7ce8166221a97d30440c8eac2be6nginx:latest"

Destroy Infrastructure

To stop the container and destroy the resources created in this tutorial, run terraform destroy. When Terraform asks you to confirm, type yes and press ENTER:

terraform destroy
Destroy complete! Resources: 2 destroyed.

Real World

ntfy Server

Let's try to deploy ntfy using Terraform. The Docker-Compose file for it looks like this:

docker-compose.yml

version: "3"

services:
  ntfy:
    image: binwiederhier/ntfy
    container_name: ntfy
    command:
      - serve
    environment:
      - TZ=UTC    # optional: set desired timezone
    user: 1002:1002 # replace with the user/group or uid/gid
    volumes:
      - ./cache:/var/cache/ntfy
      - ./config:/etc/ntfy
      - ./db:/var/lib/ntfy/
    ports:
      - 8000:80 # exposed on port 8000 (you can change it)
    restart: unless-stopped

There are a lot more parameters in here that need to be added - see Terraform Docs:

./variables.tf

variable "image_version" {
  description = "Version of the ntfy Docker image"
  type        = string
}

variable "container_name" {
  description = "Name of the ntfy Docker container"
  type        = string
}

variable "http_internal" {
  description = "Internal http port of the nfty service"
  type        = number
}

variable "http_external" {
  description = "External http port of the nfty service"
  type        = number
}

// variable "volume_mounts" {
//   description = "List of all the volume mounts"
//   type        = list(any)
//   default = [
//     {
//       volume_name    = "cache"
//       host_path      = "/opt/nfty/cache/"
//       container_path = "/var/cache/ntfy/"
//       type           = "bind"
//       read_only      = false
//     },
//     {
//       volume_name    = "user_db"
//       host_path      = "/opt/nfty/lib/"
//       container_path = "/var/lib/ntfy/"
//       type           = "bind"
//       read_only      = false
//     },
//     {
//       volume_name    = "log"
//       host_path      = "/opt/nfty/log/"
//       container_path = "/var/log/"
//       type           = "bind"
//       read_only      = false
//     }
//   ]
// }

./terraform.tfvars

image_version  = "binwiederhier/ntfy:latest"
container_name = "ntfy"
http_internal  = 80
http_external  = 8080

./main.tf

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.1"
    }
  }
}

provider "docker" {}

resource "docker_image" "ntfy" {
  name         = var.image_version
  keep_locally = false
}

resource "docker_container" "ntfy" {
  image    = docker_image.ntfy.image_id
  name     = var.container_name
  user     = "1002:1002"
  start    = true
  must_run = true
  restart  = "unless-stopped"

  ports {
    internal = var.http_internal
    external = var.http_external
  }

  command = [
    "serve"
  ]

  env = [
    "TZ=UTC"
  ]

  // dynamic "volumes" {
  //   for_each = var.volume_mounts
  //   iterator = each
  //   content {
  //     volume_name    = each.value.volume_name
  //     host_path      = each.value.host_path
  //     container_path = each.value.container_path
  //     read_only      = each.value.read_only
  //   }
  // }

    upload {
    file = "/var/cache/ntfy/cache.db"
    source = "./cache/cache.db"
  }

  upload {
    file = "/var/lib/ntfy/user.db"
    source = "./lib/user.db"
  }

  upload {
    content = <<EOF
base-url: "http://192.168.2.112"
listen-http: ":80"
cache-file: "/var/cache/ntfy/cache.db"
cache-duration: "12h"
auth-file: /var/lib/ntfy/user.db
auth-default-access: "deny-all"
behind-proxy: false
attachment-cache-dir: "/var/cache/ntfy/attachments"
attachment-total-size-limit: "5G"
attachment-file-size-limit: "15M"
attachment-expiry-duration: "3h"
web-root: /ntfy
enable-signup: false
enable-login: true
enable-reservations: true
log-level: info
log-format: json
log-file: /var/log/ntfy.log
      EOF

    file = "/etc/ntfy/server.yml"
  }

  healthcheck {
    test     = ["CMD", "curl", "-f", "http://localhost:8080/ntfy"]
    interval = "15s"
    timeout  = "2s"
    retries  = 5
  }
}

I was not able to get the volume mounts to work. I guess that Terraform handles the file system similar to Nomad - which means that you would have to configure Terraform to access the file system instead of being locked in inside the Docker overlay. But I cannot find any documentation for it.

So I replaced all volume mounts with file uploads. Note that you have to start ntfy once and create your users, subscriptions and permissions. Then copy the cache.db and user.db to where you want to Terraform to pick them up on your next deployment.

Run the Container

terraform init
> Terraform has been successfully initialized!
terraform fmt
> main.tf
> terraform.tfvars
> variables.tf
terraform validate
> Success! The configuration is valid.
terraform apply
> Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
terraform destroy
> Destroy complete! Resources: 2 destroyed.