Why automate Ansible

Ansible can be used for many things. There are only a few things I have on my bucket list of things I would like to do, where Ansible cannot help me.

One of my most urgent things to handle was the increasing complexity of Ansible, its configuration and in particular the role development. As I got deeper into Ansible, more and more factors needed to be taken into consideration when setting up a role: the role structure, linting issues, molecule testing, etc.

When writing a new role, I usually start with the tool ansible-galaxy in order to create the basic structure.

$ ansible-galaxy role init dbv.test
- Role dbv.test was created successfully
$ tree dbv.test
├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── README.md
├── tasks
│   └── main.yml
├── templates
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

This gives me the basic structure, files and directories that I will most likely need. It also gives me some stuff that I do not need. I also lack some things like molecule testing.

It does not only make my life easier, but also gives me some extra work.

If I ran yamllint on the freshly create role, I immediately get a list of problems.

$ yamllint .
./meta/main.yml
  1:1       warning  missing document start "---"  (document-start)
  21:81     error    line too long (84 > 80 characters)  (line-length)
  25:5      warning  comment not indented like content  (comments-indentation)
  25:81     error    line too long (83 > 80 characters)  (line-length)
  26:81     error    line too long (82 > 80 characters)  (line-length)
  29:81     error    line too long (83 > 80 characters)  (line-length)
  33:3      warning  comment not indented like content  (comments-indentation)
  33:81     error    line too long (85 > 80 characters)  (line-length)

The file ./meta/main.yml is the troublemaker here. If I run ansible-lint on top of that against the blank role, I get even more issues with the same file:

$ ansible-lint .
ERROR    Computed fully qualified role name of test does not follow current galaxy requirements.
Please edit meta/main.yml and assure we can correctly determine full role name:

galaxy_info:
role_name: my_name  # if absent directory name hosting role is used instead
namespace: my_galaxy_namespace  # if absent, author is used instead

Namespace: https://old-galaxy.ansible.com/docs/contributing/namespaces.html#galaxy-namespace-limitations
Role: https://old-galaxy.ansible.com/docs/contributing/creating_role.html#role-names

As an alternative, you can add 'role-name' to either skip_list or warn_list.

Computed fully qualified role name of test does not follow current galaxy requirements.
Please edit meta/main.yml and assure we can correctly determine full role name:

galaxy_info:
role_name: my_name  # if absent directory name hosting role is used instead
namespace: my_galaxy_namespace  # if absent, author is used instead

Namespace: https://old-galaxy.ansible.com/docs/contributing/namespaces.html#galaxy-namespace-limitations
Role: https://old-galaxy.ansible.com/docs/contributing/creating_role.html#role-names

As an alternative, you can add 'role-name' to either skip_list or warn_list.

I foolishly tend to share my roles. Without that I could delete the whole subdirectory ./meta and just do not have to care about these initial problems. But since I know how that would make my role less resilient, usable and shareable, I usually fix those problems.

Why the ./meta/main.yml file is not passing the basic yamllint or ansible-lint test, I cannot understand. Surely the Ansible project should be able to create lint-valid code? Spoiler: It isn’t that easy.

Since I am already working on ./meta/main.yml, I also update the role author, platforms, the minimum required Ansible version, add my desired license and delete all the comments.

That makes me pass at least the ansible-lint check for this file.

But I still cannot start working on my role. It would be easy just to write some code, hope it works and call it a day. But since I stepped my toe into Ansible molecule, I have to create the required file-structure there as well.

 tree molecule/
molecule/
└── default
    ├── converge.yml
    ├── molecule.yml
    ├── prepare.yml
    ├── variables.yml
    └── verify.yml

The file variables.yml is my personal little amendment to molecule in order to set the variables I use in a role. For each testing scenario I have to create a new directory with at least these five files again. And again. And again.

The file molecule.yml also has some references to my local Podman containers which I use for development and testing. Each time I test on a new platform/container, I have to update these in all my roles as well. Some additional work that this development brings forth.

I hope you can see how this very quickly starts to get out of hand. Each new feature and functionality increases the initial workload every time I create a role.

But I am not done yet.

While the basic role file structure now passes the linting tests, the structure itself requires some changes.

The directory ./tests and the file ./.travis.yml I usually do not need. I delete them. Instead I add a configuration file for yamllint into the root directory of the role.

A file CHANGELOG.md should not be missing either.

Last but not least I structure my roles in installation, configuration and service tasks. Following that pattern I create corresponding files in ./tasks/, called install.yml, config.yml and service.yml and include those in this order in ./tasks/main.yml.

At this point I have already done quite some changes to the role, without actually writing any application related code:

• Delete the subdirectory ./tests.
• Update the role name in ./meta/main.yml.
• Update the namespace in ./meta/main.yml.
• Update the author in ./meta/main.yml
• Update the compatible platforms for the role in ./meta/main.yml.
• Update the required license in ./meta/main.yml.
• Update the compatible Ansible version in ./meta/main.yml and put it in hyphens.
• Add additional task files in ./tasks.
• Add a Changelog file.
• Add a basic yamllint configuration to the role.
• Remove the ./travis.yml file.
• Create a molecule configuration and add required scenarios.

The file structure of the (still simple) role now looks like this:

$ tree
~/git/dbv.test/
├── CHANGELOG.md
├── defaults
│   └── main.yml
├── meta
│   └── main.yml
├── molecule
│   └── default
│       ├── converge.yml
│       ├── molecule.yml
│       ├── prepare.yml
│       ├── variables.yml
│       └── verify.yml
├── README.md
├── tasks
│   ├── config.yml
│   ├── install.yml
│   ├── main.yml
│   └── requirements.yml
├── templates
└── vars
    └── main.yml

Roughly after the third time when I had to do this, I realized that this amount of work will in the end stop me from writing good roles. The tasks were repetitive, stupid and simple. They were begging for automation.

That’s what I did.

I wrote an Ansible role to create Ansible roles.

I regret nothing. Nothing.

The idea is simple. Instead of using the ansible-galaxy command to create a new role, I use an Ansible ad-hoc command instead.

The Ansible role I created to handle this I called Fule.¹

Now, this command will create a new role in /data/ansible_roles:

ansible -i localhost, all \
  --connection local \
  -m include_role \
  -a "name=dbv.fule" \
  -e fule_working_dir=/data/ansible_roles \
  -e fule_role_name=dbv.test

The role has become quite complex itself, considering all the steps it has to complete to create and adjust new roles. In order to simplify using the role, I also had to add some features I did originally not think of.

Let me take you through it:

The only two required parameter are fule_working_dir which specifies in which directory to create Ansible role and fule_role_name, setting the name of the new role.

These two parameters are also the only ones that I usually specify. If I forget them , the role outputs an error message reminding me to specify these two parameters.

For any other configuration of a new role I have two options:

1. Specify a global configuration file in e.g. ~/.ansible/fule.yml (or any other five allowed filenames), or
2. Specify local configuration settings in ./molecule/fule_variables.yml within the newly created role.

The personal configuration file in my home directory ~/.ansible/fule.yml is used to specify most of the content of the file molecule.yml in each molecule scenario. It contains the basic specifications of each container and which Podman/Docker image to use. Any update here will also update all scenarios in all managed Ansible rules with new containers to test on if I choose to run the role on an existing role again.

Because existing roles are not being destroyed, but sensible updated. Anything else would be tremendously stupid.

The more role specific settings like which scenarios a role should have are stored in ./molecule/fule_variables.yml. Additional scenarios that I need are simply listed in there and fule will just create those scenarios if they are missing.

The local configuration file keeps data like the specify molecule scenarios required.

Remember that I had to create five files in a specific directory within ./molecule every time I require a new scenario? Fule does that for me. The role reads ./molecule/fule_variables and will create additional scenarios if they are missing (beside the default scenario, of course).

As mentioned Fule does not modify existing code and leaves the role in a worse state than it was before. Therefore Fule will not

• Remove files or directories (unless instructed to).
• Overwrite existing file (unless instructed to).

I have some further adjustments to make to Fule to address even more issues during the development, but just using the role makes it very easy to update any existing role that already has been used with this setup.[^2]

I also tried to address the linting issues at the source: I filed some change request for Ansible and suggested to fix some of the issue, but I got shot down. The reasoning followed these three points:

• The Ansible project and the tools yamllint/Ansible-lint are not part of the same projects and disagree to some point about what is considered to be acceptable and what not. This occasionally leads to contrary opinions about what is preferable in terms of coding styles.
• The file templates are meant as a starting point to make further edits and are not required to be perfekt form the start.
• It does not make sense to fix some issues, but leave other (e.g. linting issues) unaddressed.

I partly disagree with these points from the Ansible team, but understand the angle where they are coming from. So, so long thee issues, that show up every time I create a new role, are being fixed, I will use other solutions like this Ansible role to work around it.