Ansible Overview

These pages explain the Ansible concepts I used to deploy roboshop, a sample e-commerce application made of 10 components:

  • Databases and messaging: mongodb, redis, mysql, rabbitmq
  • Application services: catalogue, user, cart, shipping, payment
  • Web: frontend

Each component runs on its own server, and each one needs software installed, configuration files copied, a system user created and a service started.

Source code: github.com/MadireddyManojKumar/ansible_roboshop/ansible_concepts


Deploying with Linux commands and shell scripts

The first way to deploy roboshop is by hand: log in to each server and run Linux commands, then put those commands into shell scripts. It works, but it gets hard quickly:

  • One server at a time. You copy the script to each of the 10 servers and run it there over SSH. There is no single place to run everything from.
  • Running a script twice can break things. Commands like useradd fail if the user already exists, and appending to a config file adds the same line again. You have to write an if check around almost every command.
  • Errors are easy to miss. By default a script keeps going after a command fails, and there is no built-in way to roll back.
  • No clear report. At the end you don't get a summary of what changed and what was already in place.
  • Different systems need different commands. For example, dnf on RHEL and apt on Ubuntu, and the script has to handle each one by hand.
  • Hard to maintain. As scripts grow, there is no standard structure, so they become difficult for someone else to read and reuse.

For example, creating the roboshop system user safely in a shell script needs a check first:

id roboshop &>/dev/null
if [ $? -ne 0 ]; then
  useradd --system --home /app --shell /sbin/nologin roboshop
fi

Why Ansible

Ansible solves these problems. You describe the end result you want, and Ansible makes each server match it.

  • Agentless. Nothing to install on the servers. Ansible connects over SSH from one machine, the control node.
  • All servers in one place. The inventory file lists every server, and one command configures all of them, several at the same time.
  • Safe to run again (idempotent). Modules check the current state first and only change what is different. The output shows ok for things already in place and changed for what was updated.
  • Easy to read. Playbooks are written in YAML, so they read almost like a list of steps.
  • Built-in tools. Variables, conditions, loops, functions and error handling are part of the language, and roles make the code reusable.
  • Handles different systems. Facts tell Ansible the operating system of each server, so one playbook can work on all of them.

The same roboshop system user in Ansible. There is no if check: if the user already exists, Ansible simply reports ok and moves on.

- name: Create roboshop system user
  ansible.builtin.user:
    name: roboshop
    comment: "roboshop system user"
    system: true
    home: /app
    shell: /sbin/nologin
    create_home: false

Where Ansible falls short: creating infrastructure

Ansible can also create and delete the roboshop EC2 instances and their DNS records (see Launching AWS EC2). But it is hard to maintain:

  • It doesn't remember what it created. Ansible keeps no record of the infrastructure, so to clean up you have to write separate delete tasks yourself.
  • Changes are hard. Changing something, like the instance type or a security group, often means editing the playbook and handling the existing resources yourself.
  • No preview. It can't show you what will be created, changed or deleted before it runs.

To overcome this, we use Terraform. Terraform keeps a state file of everything it created, shows a plan before making any change, and can update or destroy the whole setup with one command.

So the two tools work together: Terraform creates the infrastructure (servers, networks, DNS), and Ansible configures the servers (packages, users, services).

← Back to portfolio

Different Ways of Setting Variables in Ansible

There are 6 ways to give values to variables in Ansible.

  1. Playbook level variables
  2. Task level variables
  3. Variable file (vars_files)
  4. Prompt (vars_prompt)
  5. Inventory file
  6. Command line

Wherever a variable is defined, you use it the same way: wrap its name in double curly braces, like {{ first_name }}. When Ansible sees the double curly braces, it knows you are referring to a variable and replaces it with the variable's value.


1. Playbook level variables

Define variables under vars at the play level. These variables are available to every task in the playbook.

- name: "Ansible variables"
  hosts: localhost
  connection: local
  vars:
    my_var: "Hello, Ansible!"
    first_name: "Manoj"
    last_name: "Madireddy"

  tasks:
    - name: "playbook level variables"
      ansible.builtin.debug:
        msg: "{{ my_var }} My name is {{ first_name }} {{ last_name }}"

2. Task level variables

Define variables under vars inside a single task. These variables are available only to that task.

    - name: "task level variables"
      vars:
        first_name: "Rajesh"
        middle_name: "kumar"
        location: "Dallas, Texas, USA"
      ansible.builtin.debug:
        msg: "{{ my_var }} My name is {{ first_name }} {{ middle_name }} and I live in {{ location }}"

If the same variable is defined at both the playbook level and the task level, the task level value wins. Here first_name prints Rajesh, not Manoj.


3. Variable file (vars_files)

When you have hundreds of variables, keep them in a separate file and refer to that file from the playbook with vars_files. The same file can be reused by any playbook.

# global_variables.yaml
first_name: "Manoj"
last_name: "Madireddy"
location: "Dallas, Texas, USA"
occupation: "Site Reliability Engineer"
years_of_experience: 9
# playbook
  vars_files:
    - global_variables.yaml

  tasks:
    - name: "global variables from variable file"
      ansible.builtin.debug:
        msg: "I am a {{ occupation }} with {{ years_of_experience }} years of experience."

occupation and years_of_experience are not defined in the playbook or the task, so Ansible takes their values from the variable file.


4. Prompt (vars_prompt)

Use vars_prompt to ask the user for values when the playbook runs. Ansible shows the prompt text, reads what the user types and stores it in the variable named by name. The value is available to every task in the playbook.

  vars_prompt:
    - name: "user_name"
      prompt: "What is your name?"
      private: false

    - name: "user_occupation"
      prompt: "What is your occupation?"
      private: true

  tasks:
    - name: "variables from prompt"
      ansible.builtin.debug:
        msg: "Your name is {{ user_name }} and your occupation is {{ user_occupation }}"
  • private: false shows what the user types on the screen.
  • private: true hides what the user types, like a password. Use it for sensitive values.

5. Inventory file

Variables can be defined for a group of hosts in inventory.ini. Ansible loads them automatically when it loads the inventory, and they can be used in the playbook.

# inventory.ini
[frontend]
35.146.170.65

[frontend:vars]
appname=roboshop
version=v1

6. Command line

If a playbook expects a variable that isn't declared anywhere, pass it with -e when you run the playbook:

ansible-playbook -i inventory.ini ansible_variables.yaml -e "app_version=v2.0"

Now v2.0 is used wherever the playbook refers to {{ app_version }}.


Which one takes precedence?

If the same variable is defined in more than one place, Ansible uses this order, with the last one winning:

  1. Inventory file (lowest)
  2. Playbook level vars
  3. vars_prompt
  4. vars_files
  5. Task level vars
  6. -e on the command line (highest)

So the command line always wins.

If a variable isn't given a value anywhere, the playbook fails with an error saying the variable is undefined.

← Back to portfolio

Data Types in Ansible

Ansible playbooks are written in YAML, so Ansible's data types come from YAML. There are 6 main data types:

  1. String
  2. Number
  3. Boolean
  4. List
  5. Dictionary
  6. Null

You don't declare a type in Ansible. It works out the type from how you write the value. To check which type Ansible picked, add the type_debug filter: {{ my_var | type_debug }}.

All the examples below come from one playbook:

- name: ansible data types
  hosts: local
  connection: local
  gather_facts: false   # we don't need facts here, so skip them to run faster

1. String

A string is plain text. Quotes are optional, but use them if the value has special characters like : or #.

vars:
  # Quotes are optional, but use them when the value has special characters (: { } # etc.)
  course_name: "DevSecOps"
  tool: ansible                 # no quotes, still a string
  version_as_string: "2.17"     # quoted, so it's a STRING, not a number
- name: print string
  debug:
    msg: "course={{ course_name }}, tool={{ tool }}, type={{ course_name | type_debug }}"   # type -> str

- name: quoted number is a string
  debug:
    msg: "{{ version_as_string }} is {{ version_as_string | type_debug }}"   # str, because it's in quotes

Watch out for "2.17": it looks like a number, but the quotes make it a string.

Multi-line strings

For text that runs over several lines, YAML gives you two options:

  • | keeps each line on its own line.
  • > joins all the lines into one line.
  # Multi-line strings:
  # "|" (literal) keeps the line breaks as they are -> good for scripts and config files
  literal_text: |
    line one
    line two
  # ">" (folded) joins the lines into one line with spaces -> good for long sentences
  folded_text: >
    this is a long sentence
    that becomes one line
- name: multi-line strings
  debug:
    msg:
      - "literal: {{ literal_text }}"   # keeps \n between lines
      - "folded: {{ folded_text }}"     # one line

2. Number

A number without quotes is either an integer (whole number) or a float (decimal). You can do math with numbers.

  port: 8080          # integer
  cpu_limit: 1.5      # float (decimal)
- name: print numbers and do math
  debug:
    msg: "port={{ port }} ({{ port | type_debug }}), cpu={{ cpu_limit }} ({{ cpu_limit | type_debug }}), port+1={{ port + 1 }}"
    # int and float support arithmetic. A string like "8080" would need "| int" first.

3. Boolean

A boolean is either true or false. You'll mostly use booleans with when to decide whether a task runs.

  # Use true/false (lowercase) as best practice.
  # YAML also accepts yes/no, True/False, on/off, but keep it consistent.
  is_production: false
  enable_firewall: true
- name: use boolean in a condition
  debug:
    msg: "Firewall is enabled"
  when: enable_firewall      # booleans are mostly used with "when"

- name: skipped because is_production is false
  debug:
    msg: "This runs only in production"
  when: is_production

The first task runs. The second one is skipped, because is_production is false.


4. List

A list holds several values in order. Each item goes on its own line starting with -, or you can write the whole list on one line in square brackets.

  # Each item starts with "-". Items are read by index, starting at 0.
  packages:
    - nginx
    - git
    - python3
  # Inline (flow) style for the same kind of list:
  ports_list: [80, 443, 8080]

You can read one item by its position (counting from 0) or count the items with length.

- name: print list and access items by index
  debug:
    msg:
      - "full list: {{ packages }}"
      - "first item: {{ packages[0] }}"        # index starts at 0
      - "count: {{ packages | length }}"       # length filter gives the number of items
      - "type: {{ packages | type_debug }}"    # list

To run a task once for every item, use loop. Each time round, item holds the current value.

- name: loop over a list
  debug:
    msg: "installing {{ item }}" # item is a reserved word in ansible
  loop: "{{ packages }}"     # "item" holds the current element on each pass

5. Dictionary

A dictionary groups related values under names (keys). Think of it as one object with several fields, like a database with a host, a port and an SSL setting.

  # key: value pairs. Read with dot notation (db.host) or brackets (db['host']).
  db:
    host: mongodb.example.com
    port: 27017
    ssl: true
  # Inline (flow) style:
  app: { name: catalogue, replicas: 2 }

Read a value with a dot (db.host) or with brackets (db['port']):

- name: access dictionary values
  debug:
    msg:
      - "db host (dot): {{ db.host }}"
      - "db port (bracket): {{ db['port'] }}"   # bracket style is safer when keys have - or spaces
      - "app name: {{ app.name }}"
      - "type: {{ db | type_debug }}"           # dict

loop only works on lists, so to loop over a dictionary, first turn it into a list with dict2items. Each item then has a key and a value.

- name: loop over a dictionary
  debug:
    msg: "{{ item.key }} = {{ item.value }}"
  loop: "{{ db | dict2items }}"   # dict2items turns a dict into a list of {key, value} so loop can use it

List of dictionaries

In real playbooks you'll often combine the two: a list where each item is a dictionary. For example, a list of users, each with a name, a uid and groups.

  # Very common in real playbooks, e.g. creating many users with loop.
  users:
    - name: manoj
      uid: 1001
      groups: [wheel, docker]
    - name: roboshop
      uid: 1002
      groups: [roboshop]
- name: loop over a list of dictionaries
  debug:
    msg: "user={{ item.name }}, uid={{ item.uid }}, groups={{ item.groups | join(',') }}"
  loop: "{{ users }}"

6. Null

Null means "no value". Use it for a variable that exists but hasn't been given a value yet. Check for it with is none.

  # Empty value, "null", or "~" all mean null (Python None).
  optional_value: null
- name: check null value
  debug:
    msg: "optional_value is null -> {{ optional_value is none }}"   # "is none" tests for null

Converting between types

Sometimes a value has the wrong type, for example a number stored as a string. Filters like int, string, bool and float convert it.

- name: type conversion with filters
  debug:
    msg:
      - "{{ '100' | int + 5 }}"          # string -> int   => 105
      - "{{ port | string }}"            # int -> string
      - "{{ 'yes' | bool }}"             # string -> boolean => True
      - "{{ '3.14' | float }}"           # string -> float

Running the playbook

ansible-playbook -i inventory.ini ansible_datatypes.yaml

# Syntax check only:
ansible-playbook -i inventory.ini ansible_datatypes.yaml --syntax-check
← Back to portfolio

Conditions in Ansible

A condition decides whether a task runs. In Ansible you add it to a task with the "when:" keyword.

  • If the condition is true, the task runs.
  • If the condition is false, the task is skipped. It is not marked as failed.

Ansible checks the condition for every host separately, so the same task can run on one server and be skipped on another.


Rules for writing conditions

1. Don't use {{ }} inside when

when already understands variables, so write the variable name on its own.

right : when: env == "dev"
wrong : when: "{{ env }}" == "dev"

2. Quote the whole condition if it starts with a quote

If the condition starts with ' or ", put quotes around the whole thing. Otherwise YAML can't read the line.

when: "'git' in packages"

3. Convert values from -e and the inventory

Values passed with -e or set in the inventory arrive as strings. Convert them with | int or | bool before you compare them, as in sections 1 and 2 below.


The example playbook

All the examples on this page come from one playbook. These are the variables it uses:

- name: Ansible conditions examples
  hosts: local
  connection: local            # run on this machine without SSH
  gather_facts: true           # needed for the ansible_facts examples below
  vars:
    env: dev
    app_port: 8080
    enable_debug: true
    db_password: ""            # left empty on purpose to demo the length check
    packages: [nginx, git, curl]
    users:
      - { name: alice, admin: true }
      - { name: bob, admin: false }

1. Basic comparisons

Compare a variable with a value using ==, !=, >, <, >= or <=.

- name: Equal - runs only in dev
  ansible.builtin.debug:
    msg: "Environment is {{ env }}"
  when: env == "dev"

- name: Not equal - runs everywhere except prod
  ansible.builtin.debug:
    msg: "Not a production run"
  when: env != "prod"

# Values from inventory or -e come in as strings, so convert with "| int" before comparing numbers
- name: Number compare - port is not a privileged port
  ansible.builtin.debug:
    msg: "Port {{ app_port }} does not need root"
  when: app_port | int > 1024

2. Booleans and not

A true/false variable can be the whole condition. Add not to flip it. | bool makes sure strings like "yes" or "true" become a real true/false.

# "| bool" turns "yes", "true", "1" (strings from -e) into a real true/false
- name: Boolean - debug mode on
  ansible.builtin.debug:
    msg: "Debug mode is enabled"
  when: enable_debug | bool

- name: Negation - debug mode off
  ansible.builtin.debug:
    msg: "Debug mode is disabled"
  when: not (enable_debug | bool)

3. Combining conditions: and / or

Sometimes one check isn't enough and you need to combine two or more. There are two ways to combine them:

  • AND: the task runs only if all the conditions are true. Write each condition on its own line under when, as a list.
  • OR: the task runs if at least one condition is true. Write the conditions on one line, joined with or.

In the example below, the first task runs only when the environment is dev and the port is 8080. The second task runs when the environment is dev or qa.

# A list under "when" means AND - every line must be true
- name: AND (list form) - dev on port 8080
  ansible.builtin.debug:
    msg: "Dev app on 8080"
  when:
    - env == "dev"
    - app_port | int == 8080

- name: OR - non-production environments
  ansible.builtin.debug:
    msg: "Lower environment: {{ env }}"
  when: env == "dev" or env == "qa"

4. Membership: in / not in

Use in when you want to ask "is this value one of these?". The task runs only if the value is found in the list. not in does the opposite: the task runs only if the value is not in the list.

For example, both of these lines mean "run if the environment is dev or qa". The second one is shorter and stays easy to read even when the list gets longer:

when: env == "dev" or env == "qa"
when: env in ["dev", "qa"]

In the example below, the first task checks whether env is one of the allowed environments. The second checks whether git is one of the items in the packages list.

# Cleaner than writing many "or" checks
- name: IN list - env is one of dev/qa
  ansible.builtin.debug:
    msg: "{{ env }} is a test environment"
  when: env in ["dev", "qa"]

# Starts with a quote, so the whole condition is quoted
- name: IN list - git is in the package list
  ansible.builtin.debug:
    msg: "git will be installed"
  when: "'git' in packages"

5. Is the variable defined or empty?

Use is defined and is not defined for optional variables, so the playbook doesn't fail with an "undefined variable" error. Try running it with and without -e app_version=v2.

# Avoids "undefined variable" errors for optional vars (try: -e app_version=v2)
- name: Defined - optional variable was passed
  ansible.builtin.debug:
    msg: "Deploying version {{ app_version }}"
  when: app_version is defined

- name: Not defined - fall back to latest
  ansible.builtin.debug:
    msg: "app_version not given, using latest"
  when: app_version is not defined

A variable set to "" still counts as defined. To catch an empty value, check its length:

# "is defined" is true even for "", so also check the length for empty strings
- name: Empty string - password not set
  ansible.builtin.debug:
    msg: "WARNING: db_password is empty"
  when: db_password | length == 0

6. Conditions on facts

Facts are details Ansible collects about each host, like its OS and memory. The most common use is running the right commands for each OS. Facts need gather_facts: true.

# Most common real use: pick the right commands for the OS
- name: Facts - only on macOS
  ansible.builtin.debug:
    msg: "macOS host, use homebrew"
  when: ansible_facts['os_family'] == "Darwin"

- name: Facts - only on RedHat family (skipped on a Mac)
  ansible.builtin.debug:
    msg: "RedHat host, use dnf"
  when: ansible_facts['os_family'] == "RedHat"

- name: Facts - enough memory
  ansible.builtin.debug:
    msg: "Host has {{ ansible_facts['memtotal_mb'] }} MB RAM"
  when: ansible_facts['memtotal_mb'] | int >= 4096

7. Conditions on an earlier task's result

register saves a task's result in a variable. Later tasks can check it, for example the return code (rc, where 0 means success) or what the command printed (stdout).

# failed_when: false -> never fail, we only want the return code
# changed_when: false -> a read-only check should not show "changed"
- name: Register - check if git is installed
  ansible.builtin.command: which git
  register: git_check
  failed_when: false
  changed_when: false

- name: Register - rc is 0 means command worked
  ansible.builtin.debug:
    msg: "git found at {{ git_check.stdout }}"
  when: git_check.rc == 0

- name: Register - rc not 0 means git is missing
  ansible.builtin.debug:
    msg: "git is NOT installed"
  when: git_check.rc != 0
# Check what the command printed
- name: Register - read OS name
  ansible.builtin.command: uname -s
  register: os_name
  changed_when: false

- name: Register - output contains a word
  ansible.builtin.debug:
    msg: "uname says Darwin"
  when: "'Darwin' in os_name.stdout"

8. Does a file or directory exist?

The stat module looks at a path and registers what it finds. Then stat.exists tells you whether it is there.

- name: Stat - look at /etc/hosts
  ansible.builtin.stat:
    path: /etc/hosts
  register: hosts_file

- name: Stat - file exists
  ansible.builtin.debug:
    msg: "/etc/hosts exists"
  when: hosts_file.stat.exists

9. when with loop

When a task has both loop and when, the condition is checked for each item. Here only alice is printed, because bob is not an admin.

- name: Loop - print only admin users
  ansible.builtin.debug:
    msg: "{{ item.name }} is an admin"
  loop: "{{ users }}"
  when: item.admin

10. when on a block

To apply one condition to several tasks, group them in a block and put when on the block, instead of repeating it on every task.

- name: Block - dev-only setup
  when: env == "dev"
  block:
    - name: Block task 1
      ansible.builtin.debug:
        msg: "Enabling verbose logs"

    - name: Block task 2
      ansible.builtin.debug:
        msg: "Seeding test data"

11. Inventory groups

group_names lists the inventory groups the current host belongs to. Use it to run a task only on certain groups, like frontend or database servers.

# group_names = list of groups the current host belongs to (from inventory.ini)
- name: Inventory - host is in the local group
  ansible.builtin.debug:
    msg: "{{ inventory_hostname }} is in group(s) {{ group_names }}"
  when: "'local' in group_names"

12. failed_when, changed_when and assert

Conditions can also decide a task's status:

  • failed_when sets your own rule for when a task counts as failed.
  • changed_when sets when a task reports "changed". Read-only commands should use false.
  • assert checks your inputs and stops the play early with a clear message if they are wrong.
# failed_when: decide yourself what counts as a failure
- name: failed_when - fail only if disk usage output is empty
  ansible.builtin.command: df -h /
  register: disk
  changed_when: false
  failed_when: disk.stdout | length == 0

# assert: stop the play early with a clear message if inputs are wrong
- name: Assert - validate inputs before deploying
  ansible.builtin.assert:
    that:
      - env in ["dev", "qa", "prod"]
      - app_port | int > 0
    fail_msg: "Invalid env or port"
    success_msg: "Inputs look good"

Running the playbook

Run it once as it is, then again with env=prod to see which tasks change from running to skipped.

ansible-playbook -i inventory.ini ansible_conditions.yaml
ansible-playbook -i inventory.ini ansible_conditions.yaml -e env=prod   # see what changes
← Back to portfolio

Loops in Ansible

A loop runs the same task once for every item in a list. Roboshop has 10 components. Instead of writing 10 tasks, you write one task and give it the list of components.

  • Put the list under loop:.
  • On each pass, the current item is in the variable item.
  • loop only accepts a list. Other data, like a dictionary or a range of numbers, has to be turned into a list first with a filter.

These are the ways to loop covered on this page:

  1. Simple list
  2. List of dictionaries
  3. Dictionary
  4. Range of numbers
  5. Nested loop
  6. loop_control
  7. loop with when
  8. loop with register
  9. until (retry loop)
  10. Old style: with_items

All the examples come from one roboshop playbook. These are the variables it uses:

- name: Ansible loops examples - roboshop
  hosts: local
  connection: local            # run on this machine without SSH
  gather_facts: false          # no facts needed for these examples
  vars:
    components: [mongodb, rabbitmq, mysql, redis, catalogue, cart, user, shipping, payment, frontend]
    servers:
      - { name: mongodb, port: 27017, database: true }
      - { name: redis, port: 6379, database: true }
      - { name: catalogue, port: 8080, database: false }
      - { name: frontend, port: 80, database: false }
    db_ports:
      mongodb: 27017
      rabbitmq: 5672
      mysql: 3306
      redis: 6379
    app_users:
      - { server: catalogue, user: roboshop, home: /app, shell: /sbin/nologin }
      - { server: cart, user: roboshop, home: /app, shell: /sbin/nologin }
      - { server: user, user: roboshop, home: /app, shell: /sbin/nologin }
      - { server: shipping, user: roboshop, home: /app, shell: /sbin/nologin }
      - { server: payment, user: roboshop, home: /app, shell: /sbin/nologin }
    envs: [dev, qa]
    app_components: [catalogue, cart, user, shipping, payment]

1. Simple list

The most common loop. The task runs once for each value in the list, and item holds the current value. Here it runs once for each of the 10 roboshop components.

- name: Simple loop - one task per roboshop component
  ansible.builtin.debug:
    msg: "Setting up {{ item }}"
  loop: "{{ components }}"

The output shows one line per component:

TASK [Simple loop - one task per roboshop component] ****************
ok: [localhost] => (item=mongodb)  => { "msg": "Setting up mongodb" }
ok: [localhost] => (item=rabbitmq) => { "msg": "Setting up rabbitmq" }
ok: [localhost] => (item=mysql)    => { "msg": "Setting up mysql" }
...
ok: [localhost] => (item=frontend) => { "msg": "Setting up frontend" }

The list doesn't have to be a variable. You can write it directly under loop:

- name: Inline list
  ansible.builtin.debug:
    msg: "Creating instance {{ item }}"
  loop:
    - mongodb
    - catalogue
    - frontend

2. List of dictionaries

When each item has several fields, like a server with a name and a port, each item is a dictionary. Read a field with item.<key>.

- name: List of dictionaries - one task per server
  ansible.builtin.debug:
    msg: "{{ item.name }} listens on port {{ item.port }}"
  loop: "{{ servers }}"

Real example: the roboshop system user

In roboshop, every app component (catalogue, cart, user, shipping, payment) runs as a system user called roboshop. A system user is meant for running an application, not for people: it has no login shell (/sbin/nologin), and its home is /app, where the app code lives.

Each item in app_users describes the user for one app server:

- name: roboshop system user - one entry per app server
  ansible.builtin.debug:
    msg: "{{ item.server }}: create user {{ item.user }}, home {{ item.home }}, shell {{ item.shell }}"
  loop: "{{ app_users }}"
  loop_control:
    label: "{{ item.server }}"
catalogue: create user roboshop, home /app, shell /sbin/nologin
cart: create user roboshop, home /app, shell /sbin/nologin
user: create user roboshop, home /app, shell /sbin/nologin
shipping: create user roboshop, home /app, shell /sbin/nologin
payment: create user roboshop, home /app, shell /sbin/nologin

On a real server, the task that creates the user looks like this. You run it on each app server, and Ansible already repeats a play for every host in the group, so you don't need a loop for the servers:

- name: Create the roboshop system user
  hosts: catalogue
  become: true
  tasks:
    - name: Create roboshop system user
      ansible.builtin.user:
        name: roboshop
        comment: "roboshop system user"
        system: true
        home: /app
        shell: /sbin/nologin
        create_home: false

    - name: Create /app directory owned by roboshop
      ansible.builtin.file:
        path: /app
        state: directory
        owner: roboshop
        group: roboshop

3. Dictionary

loop can't read a dictionary directly. The dict2items filter turns it into a list where each item has a key and a value. Here the key is the database and the value is its port.

- name: Dictionary - database name and port
  ansible.builtin.debug:
    msg: "{{ item.key }} runs on port {{ item.value }}"
  loop: "{{ db_ports | dict2items }}"
mongodb runs on port 27017
rabbitmq runs on port 5672
mysql runs on port 3306
redis runs on port 6379

4. Range of numbers

To loop over numbers, use range(start, end). It stops before the end number, so range(1, 4) gives 1, 2 and 3. Here it names three catalogue instances.

- name: Range - create 3 catalogue instances
  ansible.builtin.debug:
    msg: "catalogue-{{ item }}"
  loop: "{{ range(1, 4) | list }}"
catalogue-1
catalogue-2
catalogue-3

5. Nested loop

Sometimes you need every combination of two lists, for example every app component in every environment. The product filter builds those pairs. In each pair, item.0 is from the first list (the environment) and item.1 is from the second (the component).

- name: Nested loop - every environment with every component
  ansible.builtin.debug:
    msg: "Deploying {{ item.1 }} to {{ item.0 }}"
  loop: "{{ envs | product(app_components) | list }}"
Deploying catalogue to dev
Deploying cart to dev
...
Deploying payment to dev
Deploying catalogue to qa
...
Deploying payment to qa

6. loop_control

loop_control changes how the loop behaves. The three options you'll use most:

  • index_var: gives you the position of the current item, starting at 0.
  • label: what to print for each item in the output. Without it, Ansible prints the whole item, which gets long for dictionaries.
  • loop_var: renames item. You need this when one loop runs inside another, so the two item variables don't clash.
- name: loop_control - index and a short label
  ansible.builtin.debug:
    msg: "Server {{ idx + 1 }} of {{ servers | length }} is {{ item.name }}"
  loop: "{{ servers }}"
  loop_control:
    index_var: idx
    label: "{{ item.name }}"

- name: loop_control - rename item
  ansible.builtin.debug:
    msg: "Component: {{ component }}"
  loop: "{{ app_components }}"
  loop_control:
    loop_var: component
Server 1 of 4 is mongodb
Server 2 of 4 is redis
Server 3 of 4 is catalogue
Server 4 of 4 is frontend

7. loop with when

When a task has both loop and when, the condition is checked for each item. Items where it is false are skipped, and the rest still run. Here only the database servers are printed.

- name: loop with when - only database servers
  ansible.builtin.debug:
    msg: "{{ item.name }} is a database server"
  loop: "{{ servers }}"
  when: item.database
  loop_control:
    label: "{{ item.name }}"
ok: [localhost] => (item=mongodb) => { "msg": "mongodb is a database server" }
ok: [localhost] => (item=redis)   => { "msg": "redis is a database server" }
skipping: [localhost] => (item=catalogue)
skipping: [localhost] => (item=frontend)

8. loop with register

When you register a task that loops, the result of every pass is saved in a list called results. Each entry also remembers which item it was for, in item.

- name: loop with register - run a command for each app component
  ansible.builtin.command: "echo starting {{ item }}"
  loop: "{{ app_components }}"
  register: start_out
  changed_when: false

- name: Read the saved results
  ansible.builtin.debug:
    msg: "{{ item.item }} -> {{ item.stdout }}"
  loop: "{{ start_out.results }}"
  loop_control:
    label: "{{ item.item }}"
catalogue -> starting catalogue
cart -> starting cart
user -> starting user
shipping -> starting shipping
payment -> starting payment

9. until (retry loop)

until is a different kind of loop: it runs one task again and again until a condition becomes true. It is mostly used to wait. For example, catalogue needs mongodb, so you wait until mongodb is ready before starting catalogue.

  • until: the condition to wait for.
  • retries: the maximum number of tries. If the condition is still false after that, the task fails.
  • delay: seconds to wait between tries.

In this example a counter goes up by 1 on each try, so the condition becomes true on the third try:

- name: until - wait for mongodb to be ready
  ansible.builtin.shell: "n=$(cat /tmp/mongodb_check 2>/dev/null || echo 0); n=$((n+1)); echo $n > /tmp/mongodb_check; echo $n"
  register: mongodb_check
  until: mongodb_check.stdout | int >= 3
  retries: 5               # try at most 5 times
  delay: 1                 # wait 1 second between tries
  changed_when: false

- name: Clean up the counter file
  ansible.builtin.file:
    path: /tmp/mongodb_check
    state: absent
FAILED - RETRYING: [localhost]: until - wait for mongodb to be ready (4 retries left).
FAILED - RETRYING: [localhost]: until - wait for mongodb to be ready (3 retries left).
ok: [localhost]

The "FAILED - RETRYING" lines are normal. They only mean mongodb wasn't ready yet.


10. Old style: with_items

Older playbooks use with_* keywords instead of loop. They still work, so you will see them in existing code, but loop is recommended for new playbooks.

- name: Old style - with_items
  ansible.builtin.debug:
    msg: "Setting up {{ item }}"
  with_items: "{{ app_components }}"

How the old keywords map to loop:

  • with_items → loop
  • with_dict → loop with | dict2items
  • with_sequence → loop with range(...) | list
  • with_nested → loop with | product(...) | list

A tip for packages

Modules like dnf, apt and package accept a whole list in one go. Passing the list is faster than looping, because Ansible installs everything in one step instead of one package at a time. For example, on the frontend server:

- name: Install frontend packages in one step
  ansible.builtin.dnf:
    name: [nginx, git, curl]
    state: present

Running the playbook

ansible-playbook -i inventory.ini ansible_loops.yaml
← Back to portfolio

Functions in Ansible

Functions let you change a value before you use it: make it uppercase, remove duplicates from a list, give a default when a variable is missing, and so on. In Ansible, functions come in two kinds: filters and lookups.

Filters

A filter takes the value on the left of | and gives back a new value.

{{ value | filter_name(args) }}
{{ "roboshop" | upper }}   ->  ROBOSHOP

You can chain filters. They run from left to right, each one working on the result of the one before:

{{ list | unique | sort | join(',') }}

Lookups

A lookup reads data from outside the playbook, such as an environment variable, a file or the output of a command. Lookups always run on your own machine (the control node), not on the servers you are configuring.

{{ lookup('plugin_name', 'arg') }}

You can't write your own function inside a playbook. To reuse logic, use include_tasks, a role, or a custom filter plugin written in Python.


The example playbook

All the examples on this page come from one roboshop playbook. These are the variables it uses:

- name: Ansible functions (filters and lookups) examples
  hosts: local
  connection: local            # run on this machine without SSH
  gather_facts: false          # not needed here, makes the run faster
  vars:
    project: roboshop
    version: "v1.2.3"
    app_port: "8080"             # a string on purpose, to show the | int conversion
    packages: [nginx, git, curl, git, nginx]     # has duplicates on purpose
    components:
      - { name: mongodb, database: true }
      - { name: catalogue, database: false }
      - { name: mysql, database: true }
      - { name: cart, database: false }
      - { name: redis, database: true }
    default_config: { port: 80, debug: false }
    override_config: { debug: true }
    log_path: /var/log/roboshop/catalogue.log

1. String functions

Change text: make it upper case or title case, count its characters, replace part of it, or split it into a list.

- name: String - upper / lower / title / length
  ansible.builtin.debug:
    msg:
      - "upper  : {{ project | upper }}"        # ROBOSHOP
      - "title  : {{ project | title }}"        # Roboshop
      - "length : {{ project | length }}"       # 8

- name: String - replace / split / regex_replace
  ansible.builtin.debug:
    msg:
      - "replace : {{ version | replace('v', '') }}"            # 1.2.3
      - "split   : {{ version | replace('v', '') | split('.') }}" # ['1','2','3']
      - "major   : {{ version | regex_replace('^v(\\d+).*', '\\1') }}"  # 1

regex_replace uses a pattern to pull out part of the text. Here it takes the major version 1 out of v1.2.3.


2. Type conversion

Values from the inventory or from -e arrive as strings. To do math or compare numbers, convert them first. app_port is the string "8080", so app_port + 1 would fail, but app_port | int + 1 gives 8081.

- name: Convert - int / bool / string
  ansible.builtin.debug:
    msg:
      - "port + 1 : {{ app_port | int + 1 }}"       # 8081 (without | int this fails)
      - "bool     : {{ 'yes' | bool }}"          # True
      - "string   : {{ 42 | string }}"           # '42'

3. default() and ternary()

default is the filter you'll use most. It gives a fallback value when a variable isn't set, so the playbook doesn't fail with an "undefined variable" error.

- name: Default - use 'frontend' if app_name is not passed
  ansible.builtin.debug:
    msg: "App name is {{ app_name | default('frontend') }}"

Run it normally and it prints App name is frontend. Run it with -e app_name=cart and it prints App name is cart.

ternary is a one-line if/else: (condition) | ternary(value_if_true, value_if_false).

- name: Ternary - pick a value based on a condition
  ansible.builtin.debug:
    msg: "Mode is {{ (app_port | int > 1024) | ternary('non-root', 'root') }}"

8080 is greater than 1024, so this prints Mode is non-root.


4. List functions

Clean up and read lists: remove duplicates, sort, join into one string, take the first or last item, or count the items.

- name: List - unique / sort / join / first / last / count
  ansible.builtin.debug:
    msg:
      - "unique : {{ packages | unique }}"                      # [nginx, git, curl]
      - "sorted : {{ packages | unique | sort }}"               # [curl, git, nginx]
      - "joined : {{ packages | unique | join(', ') }}"         # nginx, git, curl
      - "first  : {{ packages | first }} / last: {{ packages | last }}"
      - "count  : {{ packages | length }}"                      # 5

For a list of numbers you can also get the smallest, largest and total:

- name: List - math on numbers
  ansible.builtin.debug:
    msg: "min={{ [4, 9, 2] | min }} max={{ [4, 9, 2] | max }} sum={{ [4, 9, 2] | sum }}"

Output: min=2 max=9 sum=15.


5. Picking items from a list of dictionaries

Each roboshop component in components has a name and a database flag. Three filters, chained together, pull out just what you need:

  • selectattr('database') keeps only the items where database is true. rejectattr does the opposite.
  • map(attribute='name') keeps only the name of each item.
  • list turns the result into a normal list. Always add it at the end.
- name: Select - names of database components only
  ansible.builtin.debug:
    msg: "Databases: {{ components | selectattr('database') | map(attribute='name') | list }}"   # [mongodb, mysql, redis]

- name: Reject - names of app components
  ansible.builtin.debug:
    msg: "Apps: {{ components | rejectattr('database') | map(attribute='name') | list }}"   # [catalogue, cart]

6. Dictionary functions

combine merges two dictionaries. If both have the same key, the value from the one on the right wins. Here debug becomes true and port stays 80.

- name: Dict - combine (merge) two dicts
  ansible.builtin.debug:
    msg: "{{ default_config | combine(override_config) }}"    # {port: 80, debug: true}

dict2items turns a dictionary into a list, so you can loop over it:

- name: Dict - dict2items to loop over key/value
  ansible.builtin.debug:
    msg: "{{ item.key }} = {{ item.value }}"
  loop: "{{ default_config | dict2items }}"

7. Paths, encoding and formats

basename gives the file name from a path, and dirname gives the folder it is in.

- name: Path - basename / dirname
  ansible.builtin.debug:
    msg:
      - "file name : {{ log_path | basename }}"   # catalogue.log
      - "directory : {{ log_path | dirname }}"    # /var/log/roboshop

b64encode and b64decode convert text to and from base64. hash creates a checksum, such as sha256.

- name: Encode - base64 and hash
  ansible.builtin.debug:
    msg:
      - "base64 : {{ 'secret' | b64encode }}"
      - "decode : {{ 'c2VjcmV0' | b64decode }}"
      - "sha256 : {{ 'secret' | hash('sha256') }}"

to_nice_json (and to_nice_yaml) print data in a readable format:

- name: Format - convert to JSON / YAML
  ansible.builtin.debug:
    msg: "{{ default_config | to_nice_json }}"

8. Lookups

Lookups read data from your own machine: env reads an environment variable, pipe runs a command and returns its output, and file reads a file.

- name: Lookup - environment variable / command output / file
  ansible.builtin.debug:
    msg:
      - "HOME   : {{ lookup('env', 'HOME') }}"
      - "whoami : {{ lookup('pipe', 'whoami') }}"
      - "hosts  : {{ lookup('file', '/etc/hosts') | length }} characters"

9. Global functions

A few functions are called directly, without |. range creates a list of numbers and now gives the current date and time.

- name: Global - range() and now()
  ansible.builtin.debug:
    msg:
      - "range : {{ range(1, 4) | list }}"             # [1, 2, 3]
      - "today : {{ now(fmt='%Y-%m-%d') }}"            # current date

10. Real use: building a value with set_fact

Functions are most useful when you combine them. Here three functions build a release name, and set_fact saves it as a new variable that later tasks can use.

- name: set_fact - make a clean release name
  ansible.builtin.set_fact:
    release_name: "{{ project | upper }}-{{ version | replace('v', '') }}-{{ now(fmt='%Y%m%d') }}"

- name: Show release name
  ansible.builtin.debug:
    var: release_name       # e.g. ROBOSHOP-1.2.3-20261101

Trying filters quickly

To test a single filter without writing a playbook, use an ad-hoc command:

ansible localhost -m debug -a "msg={{ 'roboshop' | upper }}"
ansible localhost -m debug -a "msg={{ [3,1,2] | sort }}"

To see every filter and lookup that is available, or the docs for one filter:

ansible-doc -t filter -l
ansible-doc -t lookup -l
ansible-doc -t filter ansible.builtin.combine

Running the playbook

ansible-playbook -i inventory.ini ansible_functions.yaml
ansible-playbook -i inventory.ini ansible_functions.yaml -e app_name=cart   # see default() change
ansible-playbook -i inventory.ini ansible_functions.yaml -v                 # more output
← Back to portfolio

Exception Handling in Ansible

By default, when a task fails on a host, Ansible stops running the rest of the play on that host. Sometimes you want to react to the failure instead, for example roll back a bad deployment.

Ansible does this with three sections that work like try / catch / finally in other languages:

  • block (try): the tasks you want to run.
  • rescue (catch): runs only if a task in block failed.
  • always (finally): runs every time, whether block worked or rescue ran.

The example: deploying catalogue with rollback

This playbook deploys a new version of the roboshop catalogue service. If anything goes wrong, it puts the old version back. In short:

  1. block: back up the current code, download and extract the new version, restart catalogue, and check it is healthy.
  2. rescue: if any of those steps failed, show the error, restore the backup, restart catalogue on the old version, and mark the run as failed.
  3. always: delete the downloaded zip file.
- name: Deploy catalogue service with rollback on failure
  hosts: catalogue
  become: yes
  vars:
    app_dir: /app
    backup_dir: /tmp/app_backup
    artifact_url: https://roboshop-artifacts.s3.amazonaws.com/catalogue-v3.zip

1. block: the deployment

These are the normal deployment steps. If any task fails, Ansible skips the remaining tasks in the block and jumps straight to rescue. It does not stop the play.

- name: Safe deployment
  block:
    - name: Back up current application code
      ansible.builtin.copy:
        src: "{{ app_dir }}/"
        dest: "{{ backup_dir }}/"
        remote_src: yes          # copy on the remote host, not from the controller

    - name: Download new artifact
      ansible.builtin.get_url:
        url: "{{ artifact_url }}"
        dest: /tmp/catalogue.zip
        timeout: 30              # fail fast instead of hanging

    - name: Extract new code into app directory
      ansible.builtin.unarchive:
        src: /tmp/catalogue.zip
        dest: "{{ app_dir }}"
        remote_src: yes

    - name: Restart catalogue service
      ansible.builtin.systemd:
        name: catalogue
        state: restarted
        daemon_reload: yes

    - name: Health check - app must answer HTTP 200
      ansible.builtin.uri:
        url: http://localhost:8080/health
        status_code: 200
      register: health
      retries: 5                 # try up to 5 times
      delay: 5                   # wait 5 seconds between tries
      until: health.status == 200
  • remote_src: yes means the files are already on the server, so copy or extract them there instead of from your machine.
  • The health check uses until, retries and delay to give catalogue time to start. If it still doesn't answer HTTP 200 after 5 tries, the task fails and rescue takes over.

2. rescue: roll back

These tasks run only when a task in block failed.

rescue:
  - name: Show what went wrong
    ansible.builtin.debug:
      msg: "Task '{{ ansible_failed_task.name }}' failed: {{ ansible_failed_result.msg | default('no message') }}"

  - name: Restore previous code from backup
    ansible.builtin.copy:
      src: "{{ backup_dir }}/"
      dest: "{{ app_dir }}/"
      remote_src: yes

  - name: Restart service on old version
    ansible.builtin.systemd:
      name: catalogue
      state: restarted

  - name: Mark deployment as failed after rollback
    ansible.builtin.fail:
      msg: "Deployment failed and was rolled back to the previous version."
  • ansible_failed_task and ansible_failed_result are special variables Ansible fills in inside rescue. They tell you which task failed and why.
  • The last task fails on purpose. Once rescue succeeds, Ansible counts the host as "rescued", which is treated as a success. The deployment didn't really work, so fail makes sure the play recap and your CI/CD pipeline show it as failed.

3. always: clean up

This task runs whether the deployment worked or was rolled back, so the downloaded zip never gets left behind.

always:
  - name: Clean up downloaded artifact
    ansible.builtin.file:
      path: /tmp/catalogue.zip
      state: absent

What a failed deployment looks like

Say catalogue doesn't come up after the restart. The health check fails, the rollback runs, and clean-up still happens at the end:

TASK [Health check - app must answer HTTP 200] *******************
FAILED - RETRYING: ... (4 retries left).
...
fatal: [catalogue]: FAILED! => {"msg": "Status code was 503 and not [200] ..."}

TASK [Show what went wrong] **************************************
"msg": "Task 'Health check - app must answer HTTP 200' failed: Status code was 503 and not [200] ..."

TASK [Restore previous code from backup] *************************
TASK [Restart service on old version] ****************************

TASK [Mark deployment as failed after rollback] ******************
fatal: [catalogue]: FAILED! => {"msg": "Deployment failed and was rolled back to the previous version."}

TASK [Clean up downloaded artifact] ******************************

PLAY RECAP *******************************************************
catalogue : ok=...  failed=1  rescued=1

rescued=1 shows the rollback ran, and failed=1 shows the deployment itself failed.

If everything in block works, rescue is skipped completely, and only always runs after the block.


Other ways to handle failures

block / rescue / always is for groups of tasks. For a single task there are simpler options:

  • ignore_errors: yes: if this task fails, carry on as if it hadn't.
  • failed_when: decide yourself when a task counts as failed (see Conditions).
  • any_errors_fatal: true: on a play, stop the play on every host as soon as one host fails.

Running the playbook

ansible-playbook -i inventory.ini ansible_exception_handling.yaml
← Back to portfolio

Launching AWS EC2 Using Ansible

This section shows how tools talk to AWS, how to connect the AWS CLI to your account, and how Ansible uses the same setup to create the roboshop EC2 instances and their DNS records.

How everyone talks to AWS

Every AWS service, such as EC2, S3 or Route 53, is controlled through an API: a set of web addresses that accept requests over HTTPS. Whatever tool you use, in the end it sends a request to that API.

  • AWS Console: the website. You click, and the website sends the API request.
  • AWS CLI: the aws command in your terminal.
  • SDKs: libraries for programming languages, like boto3 for Python.
  • Ansible and Terraform: automation tools that call the API for you.

Because they all use the same API, anything you can do in the console you can also do from the CLI or from Ansible. For every request, AWS checks two things: who is asking (your access keys) and whether they are allowed (IAM permissions).


Architecture

Ansible and the AWS CLI on your laptop both send signed HTTPS requests to the AWS API, which creates the roboshop EC2 instances and Route 53 records Your laptop (control node) ansible-playbook roboshop_aws_infra.yaml amazon.aws collection ec2_instance, route53 modules boto3 AWS SDK for Python AWS CLI aws configure, aws ec2 ... ~/.aws/credentials + ~/.aws/config access key ID, secret access key region: us-east-1 both read the same keys HTTPS API requests, signed with your keys AWS (us-east-1) AWS API endpoints ec2.us-east-1.amazonaws.com route53.amazonaws.com IAM checks the keys and permissions EC2: 10 roboshop instances roboshop-mongodbroboshop-redisroboshop-mysqlroboshop-rabbitmqroboshop-catalogueroboshop-userroboshop-cartroboshop-shippingroboshop-paymentroboshop-frontend Route 53: manojkumar.blog mongodb.manojkumar.blog → private IP catalogue.manojkumar.blog → private IP ... one record per instance

What happens when you run the playbook:

  1. You run ansible-playbook on your laptop.
  2. The ec2_instance and route53 modules use boto3 to build the API requests.
  3. boto3 reads your access keys and region from ~/.aws, the same files the AWS CLI uses.
  4. The requests go over HTTPS to the AWS API endpoints, signed with your secret key.
  5. IAM checks that your keys are valid and allowed to do this. Then EC2 creates the instances and Route 53 creates the DNS records.
  6. AWS sends the results (instance IDs, IP addresses) back to Ansible, which can use them in the next task.

Connecting with the AWS CLI

1. Create an access key

In the AWS Console, go to IAM → Users → your user → Security credentials → Create access key. AWS gives you an access key ID and a secret access key. The secret is shown only once, so save it somewhere safe.

2. Check the CLI is installed

aws --version

3. Run aws configure

It asks four questions:

$ aws configure
AWS Access Key ID [None]: AKIA****************
AWS Secret Access Key [None]: ****************************************
Default region name [None]: us-east-1
Default output format [None]: json

4. Test the connection

This shows which AWS account and user your keys belong to. If it works, the CLI is connected:

aws sts get-caller-identity

What the AWS configuration means

  • Access key ID: like a username, but for programs. It usually starts with AKIA.
  • Secret access key: like the password. The CLI uses it to sign every request; the secret itself is never sent to AWS.
  • Region: which AWS location to work in, for example us-east-1 (N. Virginia). EC2 instances live in one region, so you have to say which.
  • Output format: how the CLI prints results: json, table or text.

aws configure saves these in two files in your home folder:

# ~/.aws/credentials
[default]
aws_access_key_id = AKIA****************
aws_secret_access_key = ****************************************

# ~/.aws/config
[default]
region = us-east-1
output = json

Keep your keys safe:

  • Never put access keys in a playbook, a variable file or Git. Anyone who has them can use your AWS account.
  • If a key leaks, deactivate it right away in IAM and create a new one.
  • When Ansible runs on an EC2 server instead of your laptop, give the server an IAM role instead of keys.

Domain names

Two kinds of domain names are involved, and they are easy to mix up.

AWS endpoint names

Every AWS service has its own address, and for most services the region is part of it, for example ec2.us-east-1.amazonaws.com. You don't type these yourself: the CLI and boto3 build the address from the region you configured. That is why the region matters.

Your own domain in Route 53

Roboshop uses the domain manojkumar.blog, hosted in Route 53. The playbook creates one DNS record per instance, such as mongodb.manojkumar.blog, pointing to that instance's private IP address.

Every time an instance is recreated it gets a new IP address. With DNS names, the components find each other by name (catalogue connects to mongodb.manojkumar.blog), so nothing else needs to change.


How Ansible talks to AWS

Ansible does not run aws CLI commands. The modules in the amazon.aws collection use boto3, the AWS SDK for Python. boto3 and the AWS CLI read the same ~/.aws files, so once aws configure works, Ansible works too.

What you need:

ansible-galaxy collection install amazon.aws
pip install boto3 botocore

boto3 must be installed for the Python that Ansible uses (ansible --version shows it). Ansible installed with Homebrew already includes it.

The playbook runs on your own machine (hosts: local, connection: local). There is no server to SSH into yet: Ansible's "target" is the AWS API.


Creating the roboshop instances

This is the playbook that creates the whole roboshop infrastructure. These are its variables:

- name: roboshop-mongodb create ec2 instances
  connection: local
  hosts: local
  vars:
    ami_id: ami-0055977a6e828a4f4
    instance_type: t3.micro
    instances: ["mongodb", "redis", "mysql", "rabbitmq", "catalogue", "user", "cart", "shipping", "payment", "frontend"]
    domain_name: manojkumar.blog
    action: create
  • ami_id: the machine image (operating system) to start each instance from.
  • instance_type: the size of each instance.
  • instances: the 10 roboshop components; one instance is created for each.
  • domain_name: the Route 53 domain for the DNS records.
  • action: create builds everything, delete removes it.

1. Create the EC2 instances

- name: create ec2 instances
  amazon.aws.ec2_instance:
    name: roboshop-{{item}}
    image_id: "{{ ami_id }}"
    instance_type: "{{instance_type}}"
    security_groups: 
    - "roboshop-{{item}}"
    - "roboshop-common"
    region: us-east-1
  loop: "{{instances}}"
  when: action == "create"
  register: ec2_instance_creation_output
  • The loop runs once per component, so this one task creates all 10 instances, named roboshop-mongodb, roboshop-redis and so on.
  • Each instance gets two security groups: its own (roboshop-<component>) and one shared by all (roboshop-common).
  • ec2_instance looks for an existing instance with the same name first, so running the playbook again doesn't create duplicates.
  • register saves the result of every pass, including each new instance's IP address, for the next task.

2. Create the Route 53 records

- name: route 53 record creation
  amazon.aws.route53:
    state: present
    zone: "{{domain_name}}"
    record: "{{item.item}}.{{domain_name}}"
    type: A
    ttl: 1
    value: "{{item.instances[0].private_ip_address}}"
    overwrite: true
  loop: "{{ec2_instance_creation_output.results}}"
  when: action == "create"
  • This loops over the saved results from the previous task.
  • item.item is the component name the instance was created for, so the record becomes mongodb.manojkumar.blog.
  • item.instances[0].private_ip_address is the new instance's private IP. The roboshop components talk to each other inside AWS, so they use private IPs.
  • overwrite: true updates the record if it already exists, for example after an instance was recreated.

3. Delete everything

With -e action=delete, the two create tasks are skipped and these run instead. The DNS records are removed first, then the instances.

- name: route 53 record deletion
  amazon.aws.route53:
    state: absent
    zone: "{{domain_name}}"
    record: "{{item}}.{{domain_name}}"
    type: A
    ttl: 1
  loop: "{{instances}}"
  when: action == "delete"

- name: delete ec2 instances
  amazon.aws.ec2_instance:
    name: roboshop-{{item}}
    state: absent
    region: us-east-1
    wait: false
  loop: "{{instances}}"
  when: action == "delete"

Running the playbook

ansible-playbook -i inventory.ini roboshop_aws_infra.yaml                     # create
ansible-playbook -i inventory.ini roboshop_aws_infra.yaml -e action=delete    # delete

To check the result from the AWS CLI:

aws ec2 describe-instances --filters "Name=tag:Name,Values=roboshop-*" --query "Reservations[].Instances[].[Tags[?Key=='Name']|[0].Value,PrivateIpAddress,State.Name]" --output table

Running instances cost money, so delete them when you have finished practising.

← Back to portfolio

Inventory (inventory.ini)

What is the inventory?

The inventory is the list of servers Ansible manages. It holds the host names or IP addresses of all the target servers that a playbook pushes changes to, organised into groups.

Why it is required

  • A playbook only says what to do and names a group, like hosts: mongodb. The inventory says where: which actual server is in the mongodb group.
  • Ansible connects only to servers listed in the inventory. If a server isn't in it, Ansible can't reach it.
  • Because the "where" lives in a separate file, the same playbook can deploy to dev or prod just by using a different inventory.

The roboshop inventory

[frontend:vars]
appname=roboshop
version=v1

[local]
localhost

[mongodb]
mongodb.manojkumar.blog

[redis]
redis.manojkumar.blog

[mysql]
mysql.manojkumar.blog

[rabbitmq]
rabbitmq.manojkumar.blog

[catalogue]
catalogue.manojkumar.blog

[user]
user.manojkumar.blog

[cart]
cart.manojkumar.blog

[shipping]
shipping.manojkumar.blog

[payment]
payment.manojkumar.blog

[dispatch]
dispatch.manojkumar.blog

[frontend]
frontend.manojkumar.blog

[all:vars]
ansible_user=ec2-user
ansible_password=*****
mysql_root_password=*****

Passwords are masked on this page.

How to read it

  • [mongodb] is a group. The lines under it are the servers in that group. Roboshop has one group per component, with one server each.
  • Servers are listed by DNS name (mongodb.manojkumar.blog), not IP address. The next section explains why.
  • [local] holds localhost, your own machine. Playbooks that call the AWS API, like the EC2 one, run here.
  • [frontend:vars] sets variables for every server in the frontend group. Ansible loads them automatically when it reads the inventory, and playbooks can use them like any other variable, for example {{ appname }}.
  • [all:vars] sets variables for every server. all is a built-in group that always contains every host.

Variables that start with ansible_ are special: they tell Ansible how to connect. ansible_user is the SSH user name and ansible_password its password. Other variables, like mysql_root_password, are just values your playbooks use.


How Ansible uses the inventory

Roboshop's main.yaml takes the group name from a variable, so one playbook can configure any component:

- name: configure "{{ component }}""
  hosts: "{{ component }}"
  become: yes
  roles:
    - "{{component}}"
ansible-playbook -i inventory.ini main.yaml -e component=mongodb

When you run this, Ansible:

  1. Reads inventory.ini (the -i option), including its variables.
  2. Finds the group named in hosts:, here mongodb, and the servers in it: mongodb.manojkumar.blog.
  3. Looks up that name in DNS to get the server's IP address.
  4. Connects to it over SSH as ansible_user with ansible_password.
  5. Runs the tasks on that server and prints the result.

Useful commands to check an inventory before running anything:

ansible-inventory -i inventory.ini --graph                     # show groups and their servers
ansible-inventory -i inventory.ini --host frontend.manojkumar.blog   # variables one server gets
ansible -i inventory.ini mongodb --list-hosts                  # servers a group points to
ansible -i inventory.ini mongodb -m ping                       # test the SSH connection

To avoid typing -i inventory.ini every time, set inventory = inventory.ini in ansible.cfg.


How the names resolve to EC2 instances

The inventory never contains an IP address. The connection between a name like mongodb.manojkumar.blog and the real server is made by Route 53, using the output of the EC2 playbook:

The EC2 playbook creates the instance and a Route 53 record; later, Ansible reads the name from inventory.ini, looks it up in DNS, gets the private IP and connects over SSH 1. Create the server (once) roboshop_aws_infra.yaml ec2_instance creates roboshop-mongodb EC2 instance roboshop-mongodb private IP: 172.31.20.15 Route 53 record mongodb.manojkumar.blog → 172.31.20.15 register → route53 task 2. Configure the server (every run) inventory.ini [mongodb] mongodb.manojkumar.blog ansible-playbook -e component=mongodb hosts: mongodb DNS lookup mongodb.manojkumar.blog = 172.31.20.15 SSH to 172.31.20.15 as ec2-user runs the tasks Route 53 answers
  1. The EC2 playbook creates the roboshop-mongodb instance. AWS gives it a private IP, for example 172.31.20.15.
  2. The playbook registers the creation output. The Route 53 task reads the IP from it (item.instances[0].private_ip_address) and creates the record mongodb.manojkumar.blog → 172.31.20.15.
  3. Later, Ansible reads mongodb.manojkumar.blog from the inventory and asks DNS for its address. Route 53 answers with 172.31.20.15.
  4. Ansible connects to that IP over SSH and runs the tasks.

Why names instead of IPs

Every time an instance is deleted and created again, it gets a new IP address. If the inventory held IPs, you would have to edit it after every rebuild. With names, the EC2 playbook updates the Route 53 record (overwrite: true, and a ttl of 1 second so the change shows up straight away), and the inventory never changes.

Private IPs need a machine inside AWS

The records point to private IP addresses, which can only be reached from inside the same AWS network (VPC). So the configuration playbooks have to run from a machine inside AWS, such as a workstation EC2 instance, not from your laptop over the internet.


Keep passwords out of the inventory

Storing ansible_password and mysql_root_password in plain text is fine for practice, but not for real projects, because the inventory usually goes into Git. Better options:

  • Use SSH keys instead of a password (ansible_ssh_private_key_file, or private_key_file in ansible.cfg).
  • Encrypt secrets with Ansible Vault, for example ansible-vault encrypt_string, and keep them in an encrypted variable file.
← Back to portfolio

ansible.cfg: The Ansible Configuration File

What is ansible.cfg?

ansible.cfg is the settings file for Ansible. It holds the default behaviour for every run, like which inventory file to use, which user to log in as, and how the output looks.

Without it, you would have to type the same options on the command line every time. With it, you set them once and every ansible and ansible-playbook command picks them up.


When is it used?

Ansible reads the file once, at the start of every ansible or ansible-playbook command, before it runs any task. If you change the file, the change applies from the next run.


Where does Ansible look for it?

Ansible configuration can be loaded from 4 places. Ansible checks them in this order and uses the first file it finds. It only ever uses one file; it does not combine them.

1. The ANSIBLE_CONFIG environment variable

If ANSIBLE_CONFIG is set, Ansible loads the file it points to, and the other three places are ignored.

  • If you set it in your shell profile, that one file applies to every project and every playbook you run.
  • If you don't want that, remove it:
export ANSIBLE_CONFIG=/path/to/ansible.cfg   # use this file for everything
unset ANSIBLE_CONFIG                         # stop using it

2. ansible.cfg in the current folder

If the environment variable isn't set, Ansible looks for ansible.cfg in the folder you run the command from, which is usually your project or repo folder.

3. .ansible.cfg in your home folder

Next, it checks for ~/.ansible.cfg. Note the dot at the start of the name.

4. /etc/ansible/ansible.cfg

Last, it checks /etc/ansible/, the standard system-wide location on Linux. If none of the four exist, Ansible uses its built-in defaults.

On a Mac, Homebrew installs Ansible in its own location, /opt/homebrew/opt/ansible/ (the commands are in its bin folder), and doesn't create /etc/ansible/. So on a Mac you normally rely on a project ansible.cfg.

Best practice

Always keep a config file specific to each project: put ansible.cfg in the project folder next to your playbooks and inventory.ini, and commit it to Git. That way everyone running the project gets the same settings, and one project's settings don't affect another's.

To see which file Ansible is actually using, run:

ansible --version          # the "config file =" line shows the file in use

How is it written?

The file uses the INI format:

  • Settings are grouped under section names in square brackets, like [defaults].
  • Each setting is one line: name = value.
  • A line starting with # is a comment, so Ansible ignores it. In the example file almost every setting starts with #, which means Ansible uses its built-in default. To change a setting, remove the # and set your value.
  • Put comments on their own line. Ansible does not strip a # comment written after a value, so it becomes part of the value, and the setting is broken or ignored.
# right
forks = 10

# wrong: Ansible reads the value as "10   # work on 10 hosts" and stops with an error
forks = 10   # work on 10 hosts

Almost every setting can also be overridden for a single run, from the playbook or with a command line flag.

Two handy commands:

ansible-config list                  # every setting Ansible supports, with its default
ansible-config dump --only-changed   # only the settings you have changed

My ansible.cfg

My file is based on the example config. The one setting I turned on makes the playbook output easier to read:

stdout_callback = ansible.builtin.default
callback_result_format = yaml

By default Ansible prints task results as one long JSON line. callback_result_format = yaml prints them as YAML instead, spread over several lines, which is much easier to read. The older way, stdout_callback = yaml, has been removed, so that line stays commented out in my file:

#stdout_callback = yaml          # removed in community.general 12.0.0

Sample ansible.cfg

Here is a sample ansible.cfg for a project, with the settings that are used the most. Copy it into your project folder and change the values (user, key, paths) to match your setup.

[defaults]
# inventory file, so you don't need -i every time
inventory = inventory.ini
# user to log in to the servers as, and its SSH key
remote_user = ec2-user
private_key_file = ~/.ssh/roboshop.pem
# don't stop to ask "do you trust this host?" on the first connection
host_key_checking = False
# work on 10 hosts at the same time (default is 5)
forks = 10
# seconds to wait for an SSH connection
timeout = 30
# collect facts once and reuse them
gathering = smart
# where to find roles
roles_path = ./roles
# save the output of every run to a log file
log_path = ./ansible.log
# don't create .retry files when a playbook fails
retry_files_enabled = False
# find Python on the host automatically, without a warning
interpreter_python = auto_silent
# readable YAML output instead of one long JSON line
stdout_callback = ansible.builtin.default
callback_result_format = yaml

[privilege_escalation]
# run tasks as root (sudo) by default
become = True
become_method = sudo
become_user = root
# don't ask for the sudo password
become_ask_pass = False

[ssh_connection]
# fewer SSH round trips per task, so playbooks run faster
pipelining = True
# reuse one SSH connection for 60 seconds instead of opening a new one per task
ssh_args = -o ControlMaster=auto -o ControlPersist=60s
# retry an unreachable host 3 times
retries = 3

A few things to keep in mind:

  • host_key_checking = False is convenient for labs and short-lived cloud servers. In production, leave it on, so you notice if a server's identity changes.
  • become = True makes every task run as root. If only a few tasks need root, leave it off here and add become: true to those tasks instead.
  • Never put passwords in this file. It usually goes into Git with the project.

To check that Ansible picked up your values, run this from the project folder:

ansible-config dump --only-changed

Settings worth knowing

The example file has hundreds of settings, but you will only ever change a few. These are the useful ones, as they appear in my file, grouped by section.

[defaults]: general behaviour

[defaults]
#inventory       = /etc/ansible/hosts
#forks           = 5
#remote_user = root
#private_key_file = /path/to/file
#host_key_checking = True
#timeout = 10
#log_path = /var/log/ansible.log
#roles_path = ~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
#vault_password_file = /path/to/vault_password_file
  • inventory: the inventory file to use, so you don't need -i inventory.ini every time.
  • forks: how many hosts Ansible works on at the same time. The default is 5; raise it for many servers.
  • remote_user: the user Ansible logs in to the servers as.
  • private_key_file: the SSH key used to log in.
  • host_key_checking: when True, Ansible stops at a server it hasn't seen before and asks to trust it. Setting it to False skips that question (handy for labs, less safe in production).
  • timeout: how many seconds to wait when connecting to a server.
  • log_path: save the output of every run to a log file. Logging is off unless you set this.
  • roles_path: the folders Ansible searches for roles.
  • vault_password_file: a file holding the Ansible Vault password, so you don't have to type it.

Gathering facts

Facts are the details Ansible collects about each host (OS, IP, memory) at the start of a play. These settings control that step:

# smart - gather by default, but don't regather if already gathered
# implicit - gather by default, turn off with gather_facts: False
# explicit - do not gather by default, must say gather_facts: True
#gathering = implicit

#gather_timeout = 10

Collecting facts takes time, so smart is a common choice to avoid collecting them twice.

Output

#display_skipped_hosts = True
#nocolor = 1
  • display_skipped_hosts: set to False to hide the "Skipping" lines for tasks whose when was false.
  • nocolor: set to 1 to turn off coloured output.

[privilege_escalation]: running as root

[privilege_escalation]
#become = False
#become_method = sudo
#become_ask_pass = False
  • become: when True, tasks run as root (with sudo) by default. Most installs need this.
  • become_method: how to switch user, normally sudo.
  • become_ask_pass: when True, Ansible asks for the sudo password.

[ssh_connection]: speed and reliability

[ssh_connection]
#pipelining = False
#retries = 3
  • pipelining: when True, Ansible needs fewer SSH round trips per task, so playbooks run noticeably faster.
  • retries: how many times to retry a server that can't be reached.

[colors] and [diff]

[colors]
#ok = green
#changed = yellow
#error = red

[diff]
# Always print diff when running ( same as always running with -D/--diff )
#always = False
  • [colors]: change the colour used for each result, such as ok, changed and error.
  • [diff] always: when True, Ansible shows what changed inside files on every run.

The other sections in the file ([inventory], [paramiko_connection], [persistent_connection], [sudo_become_plugin], [selinux]) are for special setups. You can leave them as they are.

← Back to portfolio