Managing Config Files with Consul-template

Consul template is a system for managing configuration files using templates that pull data from the Consul key value store or Vault.  The template language is simple to use and understand.  Consul-template can also be used to as a process manager so that changes to configurations in Consult or Vault cause reloads (or restarts) of the processes which depend on the configuration changes.  Consul template is open source and written in Go.

Consul Templates

Consul templates have access to a fairly powerful set of functions when rendering templates.  Simple key value lookups can be doing the following:

{{key "service/redis/maxconns"}}

The contents of a local file can be read into the template like this:

{{file "/path/to/local/file"}}

Values can be looked up in Vault using the following syntax

{{with secret "secret/passwords"}}{{.Data.password}}{{end}}

Consul can manage services with monitoring of those services as well as simple key value pairs.  A service can be queried from a template as well so that a process configuration can be determined dynamically by the Consul service monitoring system.

{{range service "web"}}
server {{.Name}} {{.Address}}:{{.Port}}{{end}}

This could produce output like:

server nyc_web_01 123.456.789.10:8080
server nyc_web_02 456.789.101.213:8080

Example Template for Django to connect to Postgresql

DATABASES['default'] = {
   'ENGINE': 'django.db.backends.postgresql_psycopg2',
   'NAME': 'blnd',
{{ with secret "pg_blnd/creds/blend" }}
   'USER': '{{ .Data.username }}',
   'PASSWORD': '{{ .Data.password }}',
{{ end }}
{{ with secret "secret/blend/database/default" }}
   'HOST': '{{ .Data.hostname }}',
{{ end }}
   'PORT': '',
   'OPTIONS': {
       'sslmode': 'require',

Managing Processes with Consul Template

When Consul-template starts it reads it’s template files, queries Consul and/or Vault and renders the templates into configuration files.  Once this is done Consul-template can start a process that depends on the templates it has rendered.  It then watches Consul for changes.  If any are detected the templates are re-rendered and the dependent process is sent a reload signal or is restarted if none exists.  There is only one consul-template process per child process and consult-template is not a process supervisor which will restart a failing child process.

Configuring Consul Template

The consul-template configuration file is written in yaml and defines the connection information for consul and vault.  It also has the mapping for the list of templates to be rendered. Finally, it determines the child process to be run if any.