Skip to main content

Salt Execution Order

Guangzhou, China

Complex State Trees

You can combine *.sls files by using import statements or by the use of Top Files. We can use this data for example in our Apache landing page (see previous tutorial):

welcome.sls

# Adding a blank front page

include:
  - apache

{% set name = salt.pillar.get('name') %}

check_pillar_values:
  test.check_pillar:
    - present:
      - name
    - failhard: True

welcome_page:
  file.managed:
    - name: /var/www/html/index.html
    - contents: |
        <!doctype html>
        <body>
            <h1>{{ name }}.</h1>
        </body>

The include statement on top will add the Apache installation - we can now execute the welcome.sls directly and get the complete Apache setup:

sudo salt ubuntuAsus state.sls apache.welcome

You can accomplish the same by creating a /srv/salt/top.sls file:

base:
  '*':
    - apache
    - apache.welcome

The Apache setup can now be executed by running:

sudo salt '*' state.highstate

The state.highstate command will setup the whole infrastructure as defined inside your top file. If you create multiple top files for different setup use the state.top command and specify the file you want to execute:

sudo salt '*' state.top prod_top.sls

Execution Order

Salt's YAML render picks up every instruction inside an *.sls file and assigns an order key to them. This makes sure that they are executed in the same order they are written down inside your file. If you need to make sure that one of the instruction is ALWAYS either executed FIRST or LAST, you can specify this inside your file:

init.sls

# Install vanilla Apache on Debian/RedHat

{% from 'apache/map.sls' import apache with context %}

install_apache:
  pkg.installed:
   - name: {{ apache.pkg }}
   - order: last

enable_apache:
  service.running:
    - name: {{ apache.srv }}

    # Will be enabled automatically on Debian but has to be enabled manually on RedHat
    - enable: True
    - order: first

This order stops working reliably once you have include or require statements inside your file.

Requisites

Requisites bring explicit ordering to your file execution:

init.sls

# Install vanilla Apache on Debian/RedHat

{% from 'apache/map.sls' import apache with context %}

install_apache:
  pkg.installed:
   - name: {{ apache.pkg }}

enable_apache:
  service.running:
    - name: {{ apache.srv }}

    # Will be enabled automatically on Debian but has to be enabled manually on RedHat
    - enable: True
    - require:
      - pkg: install_apache

The require statement makes sure that Apache is installed before it will attempt to enable the Apache service.

Watch

The watch module reacts to a specified instruction being executed and then triggers another function. A practical use case is to restart the Apache service once the Apache configuration was modified:

init.sls

# Install vanilla Apache on Debian/RedHat

{% from 'apache/map.sls' import apache with context %}

install_apache:
  pkg.installed:
   - name: {{ apache.pkg }}

enable_apache:
  service.running:
    - name: {{ apache.srv }}

    # Will be enabled automatically on Debian but has to be enabled manually on RedHat
    - enable: True
    - watch:
      - file: danger_config
    
danger_config:
  file.managed:
    - name /bar/foo
    - contents: foo
    

We can also extend the watch service to another SLS file:

mods.sls

include:
  - apache

extend:
  start_apache:
    service:
      - watch:
        - file: danger_config

{% for conf in ['status', 'info'] %}

mod_{{ conf }}:
  file.managed:
    - name: /etc/apache2/conf-available/mod_{{ conf }}.conf
    - contents: |
        <Location '/{{ conf }}'>
            SetHandler server-{{ conf }}
        </Location>

  {% if salt.grains.get('os_family') == 'Debian' %}
  cmd.run:
    - name: a2enmod {{ conf }} && a2enconf mod_{{ conf }}
    - creates: /etc/apache2/conf-enabled/mod_{{ conf }}.conf
  {% endif %}

{% endfor %}

The mods.sls configures Apache - by including the init.sls file we can now execute mods and be certain that Apache will be installed first before the configuration is attempted. We can now define the watch task here instead of the init file.

We can also use the watch_in statement:

mods.sls

include:
  - apache

{% for conf in ['status', 'info'] %}

mod_{{ conf }}:
  file.managed:
    - name: /etc/apache2/conf-available/mod_{{ conf }}.conf
    - contents: |
        <Location '/{{ conf }}'>
            SetHandler server-{{ conf }}
        </Location>
    - watch_in:
      - service: enable_apache

  {% if salt.grains.get('os_family') == 'Debian' %}
  cmd.run:
    - name: a2enmod {{ conf }} && a2enconf mod_{{ conf }}
    - creates: /etc/apache2/conf-enabled/mod_{{ conf }}.conf
    - watch_in:
      - service: enable_apache
  {% endif %}

{% endfor %}

If mod_status or mod_info should always be changed BEFORE Apache is restarted with enable_apache.

You can test the execution order by doing a dry-run:

salt '*' state.sls apache.mods test=true

ubuntuAsus:
----------
          ID: install_apache
    Function: pkg.installed
        Name: apache2
      Result: True
     Comment: All specified packages are already installed
     Started: 18:26:02.852277
    Duration: 38.55 ms
     Changes:   
----------
          ID: mod_status
    Function: file.managed
        Name: /etc/apache2/conf-available/mod_status.conf
      Result: None
     Comment: The file /etc/apache2/conf-available/mod_status.conf is set to be changed
              Note: No changes made, actual changes may
              be different due to other states.
     Started: 18:26:02.899635
    Duration: 1.456 ms
     Changes:   
              ----------
              newfile:
                  /etc/apache2/conf-available/mod_status.conf
----------
          ID: mod_info
    Function: file.managed
        Name: /etc/apache2/conf-available/mod_info.conf
      Result: None
     Comment: The file /etc/apache2/conf-available/mod_info.conf is set to be changed
              Note: No changes made, actual changes may
              be different due to other states.
     Started: 18:26:02.901184
    Duration: 1.077 ms
     Changes:   
              ----------
              newfile:
                  /etc/apache2/conf-available/mod_info.conf
----------
          ID: enable_apache
    Function: service.running
        Name: apache2
      Result: None
     Comment: Service is set to be restarted
     Started: 18:26:02.940131
    Duration: 19.022 ms
     Changes:   
----------
          ID: mod_status
    Function: cmd.run
        Name: a2enmod status && a2enconf mod_status
      Result: None
     Comment: Command 'a2enmod status && a2enconf mod_status' would have been executed
     Started: 18:26:02.961747
    Duration: 378.228 ms
     Changes:   
----------
          ID: mod_info
    Function: cmd.run
        Name: a2enmod info && a2enconf mod_info
      Result: None
     Comment: Command 'a2enmod info && a2enconf mod_info' would have been executed
     Started: 18:26:03.340098
    Duration: 4.92 ms
     Changes:   

Summary for ubuntuAsus
------------
Succeeded: 6 (unchanged=5, changed=2)
Failed:    0
------------
Total states run:     6
Total run time: 443.253 ms

Conditionals and Branching

onChanges

We can use onChanges to watch a state and trigger actions when required. For example the Apache modules only need to be (re) enabled if their content changed.

include:
  - apache

{% for conf in ['status', 'info'] %}

mod_{{ conf }}:
  file.managed:
    - name: /etc/apache2/conf-available/mod_{{ conf }}.conf
    - contents: |
        <Location "/{{ conf }}">
            SetHandler server-{{ conf }}
        </Location>
    - watch_in:
      - service: enable_apache

  {% if salt.grains.get('os_family') == 'Debian' %}
  cmd.run:
    - name: a2enmod {{ conf }} && a2enconf mod_{{ conf }}
    - onchanges:
      - file: mod_{{ conf }}
    - watch_in:
      - service: enable_apache
  {% endif %}

{% endfor %}

You can now re-run the command and the see that the execution was suppressed:

salt ubuntuAsus state.sls apache.mods
----------
          ID: mod_status
    Function: cmd.run
        Name: a2enmod status && a2enconf mod_status
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 10:54:17.354494
    Duration: 0.004 ms
     Changes:   
----------
          ID: mod_info
    Function: cmd.run
        Name: a2enmod info && a2enconf mod_info
      Result: True
     Comment: State was not run because none of the onchanges reqs changed
     Started: 10:54:17.354603
    Duration: 0.002 ms
     Changes:

You can now change the configuration file (by adding some meaningless white space) and re-run the command:

...

<Location "/{{ conf }}">
    SetHandler server-{{ conf }}

</Location>

...

The change will be picked up and the modules will be executed once again:

salt ubuntuAsus state.sls apache.mods

                                                                    
ubuntuAsus:
----------
          ID: mod_status
    Function: file.managed
        Name: /etc/apache2/conf-available/mod_status.conf
      Result: True
     Comment: File /etc/apache2/conf-available/mod_status.conf updated
     Started: 11:00:09.916316
    Duration: 31.164 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,3 +1,4 @@
                   <Location "/status">
                       SetHandler server-status
                  +
                   </Location>
----------
          ID: enable_apache
    Function: service.running
        Name: apache2
      Result: True
     Comment: Service restarted
     Started: 11:00:10.005666
    Duration: 133.762 ms
     Changes:   
              ----------
              apache2:
                  True
----------
          ID: mod_info
    Function: file.managed
        Name: /etc/apache2/conf-available/mod_info.conf
      Result: True
     Comment: File /etc/apache2/conf-available/mod_info.conf updated
     Started: 11:00:09.947841
    Duration: 11.464 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -1,3 +1,4 @@
                   <Location "/info">
                       SetHandler server-info
                  +
                   </Location>
----------
          ID: mod_status
    Function: cmd.run
        Name: a2enmod status && a2enconf mod_status
      Result: True
     Comment: Command "a2enmod status && a2enconf mod_status" run
     Started: 11:00:10.142729
    Duration: 59.029 ms
     Changes:   
              ----------
              pid:
                  83853
              retcode:
                  0
              stderr:
              stdout:
                  Module status already enabled
                  Conf mod_status already enabled
----------
          ID: mod_info
    Function: cmd.run
        Name: a2enmod info && a2enconf mod_info
      Result: True
     Comment: Command "a2enmod info && a2enconf mod_info" run
     Started: 11:00:10.202081
    Duration: 52.738 ms
     Changes:   
              ----------
              pid:
                  83864
              retcode:
                  0
              stderr:
              stdout:
                  Module info already enabled
                  Conf mod_info already enabled

onFail

With onFail we can run a state if another state does not complete successfully. Create a file nano /srv/salt/apptest.sls and let it download a git repository for us:

apptest:
  git.latest:
    - name: https://github.com/mpolinowski/docker-elk.git
    - rev: master
    - target: /opt/apptest

notify_of_fail:
  event.send:
    - name: apptest/failed
    - onfail:
      - git: apptest

Then run the state:

salt ubuntuAsus state.sls apptest
ubuntuAsus:
----------
          ID: apptest
    Function: git.latest
        Name: https://github.com/mpolinowski/docker-elk.git
      Result: True
     Comment: https://github.com/mpolinowski/docker-elk.git cloned to /opt/apptest
     Started: 11:24:51.598681
    Duration: 13472.643 ms
     Changes:   
              ----------
              new:
                  https://github.com/mpolinowski/docker-elk.git => /opt/apptest
              revision:
                  ----------
                  new:
                      2358839589c36223984b6a0528289d9efefd5189
                  old:
                      None
----------
          ID: notify_of_fail
    Function: event.send
        Name: apptest/failed
      Result: True
     Comment: State was not run because onfail req did not change
     Started: 11:25:05.080430
    Duration: 0.013 ms
     Changes:   

Summary for ubuntuAsus
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:  13.473 s

The repository cloning was successful and the notification was NOT triggered.

prereq

Only run a state when another state will change the system. E.g. reload your webserver only if a new app version was downloaded by git:

{% from "apache/map.sls" import apache with context %}

include:
  - apache

apptest:
  git.latest:
    - name: https://github.com/mpolinowski/docker-elk.git
    - rev: master
    - target: /opt/apptest
    - watch_in:
      - service: enable_apache

reload_apache:
  module.run:
    - name: service.stop
    - m_name: {{ apache.srv }}
    - prereq:
      - git: apptest

notify_of_fail:
  event.send:
    - name: apptest/failed
    - onfail:
      - git: apptest

The prerequisite will run apptest in Dry-Run. If the code changed, reload_apache will be triggered once the new source code was cloned by git.

salt ubuntuAsus state.sls apptest                                                                             
ubuntuAsus:
----------
          ID: install_apache
    Function: pkg.installed
        Name: apache2
      Result: True
     Comment: All specified packages are already installed
     Started: 11:44:57.106219
    Duration: 40.43 ms
     Changes:   
----------
          ID: reload_apache
    Function: module.run
        Name: service.stop
      Result: True
     Comment: Module function service.stop executed
     Started: 11:44:58.331330
    Duration: 97.635 ms
     Changes:   
              ----------
              ret:
                  True
----------
          ID: apptest
    Function: git.latest
        Name: https://github.com/mpolinowski/docker-elk.git
      Result: True
     Comment: https://github.com/mpolinowski/docker-elk.git cloned to /opt/apptest
     Started: 11:44:58.429290
    Duration: 4591.687 ms
     Changes:   
              ----------
              new:
                  https://github.com/mpolinowski/docker-elk.git => /opt/apptest
              revision:
                  ----------
                  new:
                      2358839589c36223984b6a0528289d9efefd5189
                  old:
                      None
----------
          ID: enable_apache
    Function: service.running
        Name: apache2
      Result: True
     Comment: Service apache2 is already enabled, and is running
     Started: 11:45:03.021324
    Duration: 118.316 ms
     Changes:   
              ----------
              apache2:
                  True
----------
          ID: notify_of_fail
    Function: event.send
        Name: apptest/failed
      Result: True
     Comment: State was not run because onfail req did not change
     Started: 11:45:03.141194
    Duration: 0.004 ms
     Changes:   

Summary for ubuntuAsus
------------
Succeeded: 5 (changed=3)
Failed:    0
------------
Total states run:     5
Total run time:   4.848 s

reload_apache detected that apptest will add new source code and stopped the webserver. Once the code was cloned the watch task reloaded the webserver.