Ansible filters for taming lists (part 1)
When running through a series of tasks in Ansible it is common to want to have data available in a few formats - sometimes as a simple list and sometimes as a dictionary. What we don't really want to do is to have the same data in two or more separate lists (that would be hard to maintain).
So say I have a task that updates a series of images on the remote host and I need the data in the format -
{ src: filename, type: filetype}But in another task I want a list of the types we updated. I don't want to have to manage two lists and I don't want to repeat an action if the types are duplicated across a number of file instances.
Ideally we would like to be able to get from the list a list of unique values from one of the elements - and that is where Jinja filters can help.
Firstly I am bound to say that Ansible is not a programming language, so we should always think of the simplest way to do this and not have loads of Jinja processing within a task.
Also in some cases it is worth considering creating/using a plugin to get the processing out of your tasks and templates (and that is the subject of part 2).
So back to the task - Lets consider a fictitious scenario. I have a site that has pictures of fruit and you can download archives of all of the images of fruit combined by type. A routine on the server builds archives for the fruit types that contain all of the images of fruit of that type. So I can request a download of a zip containing all citrus fruit images or all non citrus fruit images. These archives are cached so they don't have to be built more than once - but the cached files need to be dropped if more fruit images are added.
Occasionally I want to upload more fruity pictures and when I do that I want to drop the archives for the type (so they will get rebuilt when next requested). But I don't want to maintain two vars; one for the list of images and one for the list of types of archive to drop. What I want is one list of fruit images that lists the name of the files and the type of the fruit.
So I could easily use a with_items iterator for both a copy task (to put up the images) and the drop archive task (file state=absent), but that would repeat the second task necessarily.
So I create a dictionary var for the fruit images:
images_to_update:
- { name: "oranges-001.jpg", type: "Citrus"}
- { name: "oranges-002.jpg", type: "Citrus"}
- { name: "apples-001.jpg", type: "Non-citrus"}
- { name: "bananas-001.jpg", type: "Non-citrus"}
- { name: "grapes-001.jpg", type: "Non-citrus"}
- { name: "grapefruit-001.jpg", type: "Citrus"}
- { name: "pears-001.jpg", type: "Non-citrus"}and then create one task to use a with_items to copy each of these up to the server.
Then I use a filter to set a fact (a new variable that persists only during the scope of the play) to create a unique list of the type values so that I can drop the relevant archives.
This uses the filter:
fruit_archives_to_drop: "{{images_to_update|map(attribute='type')|list|unique }}"This says take the dictionary structure (images_to_update) and create a list from the 'type' attribute and further make it unique - Cool huh! You can then use this generated list to iterate without repetition.
"['Citrus', 'Non-citrus']"See a working demo of this in the gist below.
In the next part I will show how we can use this sort of thing to get a list of unique IPs from your inventory (hostvars) to add to a firewall whitelist or similar in partnership with a plug-in.