Skip to main content

Hashicorp Consul Refresher - Services

Guangzhou, China

Registering a Service​

Services can registered with Consul as part of the Nomand Job that sets them up. But you can also directly register a service with Consul using the REST API consul services register with inline parameters or sending a separate service file using the -config-file or -config-dir flag. It can also be placed inside the Consul agent configuration directory and be read on start-up.

Service Definition​

A service definition contains a set of parameters that specify various aspects of the service, including how it is discovered by other services in the network:

  • id: Name of the node. If not specified, the value of the name field will be used.
  • name: Name of the service - use valid DNS labels for service definition names for compatibility with external DNSs.
  • tags: List of string values that can be used to add service-level labels. Use valid DNS labels for service definition IDs
  • address: String value that specifies a service-specific IP address or hostname. If no value is specified, the IP address of the agent node is used by default.
  • port: Integer value that specifies a service-specific port number.
  • checks: Array of objects that define health checks for the service.
nano ~/consul_services/test_service.hcl
{
  "service": {
    "id": "webservice01",
    "name": "webservice-test",
    "tags": ["this", "is", "a", "test"],
    "address": "192.168.2.111",
    "port": 9200
  }
}

This definition registers a web service running on port 9200. I am going to create this with Node.js module npm install httpster that allows me to quickly spin up a web service from my command line on the minion server 192.168.2.111:

httpster -p 9200

Back on my Consul master I can now register this service with Consul:

consul services register ~/consul_services/test_service.hcl

The new service immediately shows up as healthy in the nomad UI:

Hashicorp Consul Services

Health Checks​

But the consul is not yet keeping a eye on the service - even if you shut down the web server it will remain "green" since we don't have a health check configured:

Types of Health Checks​

  • Application Level: Check if the service is still running.
  • System Level: Check if the Node and Port is available.

There are several different kinds of checks:

  • Scripts: These checks depend on invoking an external application that performs the health check, exits with an appropriate exit code, and potentially generates some output. A script is paired with an invocation interval
  • HTTP Status: These checks curl HTTP GET request to the specified URL, waiting the specified interval amount of time between requests. The status of the service depends on the HTTP response code: any 2xx code is considered passing, a 429 Too ManyRequests is a warning, and anything else is a failure.
  • TCP Connection: These checks make a TCP connection attempt to the specified IP/hostname and port, waiting interval amount of time between attempts. If no hostname is specified, it defaults to "localhost".
  • Docker: These checks depend on invoking an external application which is packaged within a Docker Container. The application is triggered within the running container via the Docker Exec API.

So let's deregister the service and actually add a check to the service declaration that periodically checks if the web service is still online:

consul services deregister ~/consul_services/test_service.hcl
nano ~/consul_services/test_service.hcl
{
  "service": {
    "id": "webservice01",
    "name": "webservice-test",
    "tags": ["this", "is", "a", "test"],
    "address": "192.168.2.111",
    "port": 9200,
    "check": {
      "id": "web",
      "name": "check web service on port 9200",
      "tcp": "192.168.2.111:9200",
      "interval": "10s",
      "timeout": "1s"
    }
  }
}

And run the service again:

consul services register ~/consul_services/test_service.hcl

Hashicorp Consul Services

And this time we have both the verification that the Consul agent on our minion is running and the TCP Health Check telling us the web service is online. The latter will alert us within 10s when we shut down our web service:

dial tcp 192.168.2.111:9200: connect: connection refused

Check Service Status​

DNS Request​

Consul provides an DNS Service that allows you to query for services, e.g. our webservice-test.service.consul:

dig @192.168.2.110 -p 8600 webservice-test.service.consul

; <<>> DiG 9.11.26-RedHat-9.11.26-4.el8_4 <<>> @192.168.2.110 -p 8600 webservice-test.service.consul

;; ANSWER SECTION:
webservice-test.service.consul. 0 IN    A       192.168.2.111

The response tells us that there was one server found on IP 192.168.2.111 that provides the service webservice-test.

Consul REST API​

Alternatively, you can send a GET request to the Consul API:

curl --request GET http://192.168.2.110:8500/v1/catalog/service/webservice-test

[{"ID":"d561f8d4-9606-8c9c-40d4-a5350857801e","Node":"consul-master","Address":"192.168.2.110","Datacenter":"instaryun","TaggedAddresses":{"lan":"192.168.2.110","lan_ipv4":"192.168.2.110","wan":"192.168.2.110","wan_ipv4":"192.168.2.110"},"NodeMeta":{"consul-network-segment":""},"ServiceKind":"","ServiceID":"webservice01","ServiceName":"webservice-test","ServiceTags":["this","is","a","test"],"ServiceAddress":"192.168.2.111","ServiceTaggedAddresses":{"lan_ipv4":{"Address":"192.168.2.111","Port":9200},"wan_ipv4":{"Address":"192.168.2.111","Port":9200}},"ServiceWeights":{"Passing":1,"Warning":1},"ServiceMeta":{},"ServicePort":9200,"ServiceSocketPath":"","ServiceEnableTagOverride":false,"ServiceProxy":{"Mode":"","MeshGateway":{},"Expose":{}},"ServiceConnect":{},"CreateIndex":10150,"ModifyIndex":10150}]%

Prepared Queries​

You can also write common queries down into files as Prepared Queries and run them from there - nano ~/consul_services/prepared-query.hcl:

{
  "Name": "thisisatest",
  "Service": {
    "Service": "webservice-test",
    "Tags": ["this","is","a","test"]
  }
}

You can register this query through the Consul REST API:

curl --request POST --data @prepared-query.hcl http://192.168.2.110:8500/v1/query

{"ID":"d3b2063d-df45-7649-c0ec-e3d671deced3"}%

And the query was saved under the ID d3b2063d-df45-7649-c0ec-e3d671deced3:

curl --request GET http://192.168.2.110:8500/v1/query/d3b2063d-df45-7649-c0ec-e3d671deced3 | jq

[
  {
    "ID": "d3b2063d-df45-7649-c0ec-e3d671deced3",
    "Name": "thisisatest",
    "Session": "",
    "Token": "",
    "Template": {
      "Type": "",
      "Regexp": "",
      "RemoveEmptyTags": false
    },
    "Service": {
      "Service": "webservice-test",
      "Failover": {
        "NearestN": 0,
        "Datacenters": null
      },
      "OnlyPassing": false,
      "IgnoreCheckIDs": null,
      "Near": "",
      "Tags": [
        "this",
        "is",
        "a",
        "test"
      ],
      "NodeMeta": null,
      "ServiceMeta": null,
      "Connect": false
    },
    "DNS": {
      "TTL": ""
    },
    "CreateIndex": 10622,
    "ModifyIndex": 10622
  }
]

We can now use this query, under it's registered name thisisatest, in our applications to find an address where the service webservice-test is hosted:

dig @192.168.2.110 -p 8600 thisisatest.query.consul

;; ANSWER SECTION:
thisisatest.query.consul. 0     IN      A       192.168.2.111

And the server that is a best match is 192.168.2.111.

To update the query - e.g. if we want our application to start using the version 2 of our webservice - we just have to update the prepared query file:

{
  "Name": "thisisatest",
  "Service": {
    "Service": "webservice-test",
    "Tags": ["this","is","a","test", "v2"]
  }
}

and then use PUT to upload the update to Consul:

curl --request PUT --data @prepared-query.hcl http://192.168.2.110:8500/v1/query/d3b2063d-df45-7649-c0ec-e3d671deced3