Adding support for Ansible Collections to dhall-ansible


#1

Hello,

I am planning to update the dhall-ansible package to support Ansible Collections. Here is some context and my plan, please let me know if the proposition sounds good to you.

Context

Ansible configuration is mostly composed of Play containing lists of Task. A play is defined as follow:

-- Play.dhall
{ hosts : List Text
, pre_tasks : List Ansible.Task.Type
, roles : Optional (List Text)
, tasks : List Ansible.Task.Type
}

And a task is defined as follow:

-- Task.dhall
{ name : Text
, when : Optional Text
, vars : Optional JSON.Type
, module_name : ModuleOptions
}

Example configuration:

- hosts: localhost
  tasks:
    - name: create a directory
      file:
        path: /var/lib/data
        state: directory

    - name: copy a file
      copy:
        src: file.txt
        dest: /var/lib/data/

Previously I added the most common modules to a big Task record type with Optional module options, for example:

-- Task.dhall
{ name : Text
, when : Optional Text
, vars : Optional JSON.Type
, file : Optional Ansible.File.Type
, copy : Optional Ansible.Copy.Type
, command : Optional Ansible.Command.Type
, ...
}

This caused two issues:

  • Adding all the modules (more than 4500 today) is not very practical.
  • Extending the Task type is not compatible with the current Play definition because //\\ can’t be used to modify the existing Play.tasks type.

Here is the status-quo to mitigate the first issue: custom.dhall

The plan

Since ansible version 2.10, the modules are now divided into logical collections (you can find the list here). And thanks to the ansible-doc command, it is possible to get all their options and convert them to Dhall types.

Thus I’m working on a tool to generate the following structure:

  • Ansible/Collection/$collectionName/{$collectionModules...}/{Type,default,package}.dhall : the module options schemas
  • Ansible/Collection/$collectionName/Task/{Type,default,package}.dhall : a record type with all the collection module, for example:
-- Ansible/Collection/OpenstackCloud/Task/Type.dhall
{ `openstack.cloud.server` : Optional ../Server/Type.dhall
, `openstack.cloud.volume` : Optional ../Volume/Type.dhall
, ...
}

The Play will no longer define a task attribute, and the Task will only contain the few builtin module. To use a collection, here is how it will work:

--| To use a collection, or a custom module, the task type needs to be extented

let Ansible = ../package.dhall

let --| Here the task type is extended with the OpenStack collection
    Task =
      { Type =    Ansible.Task.Type    //\\ Ansible.Collection.OpenstackCloud.Task.Type
      , default = Ansible.Task.default //\\ Ansible.Collection.OpenstackCloud.Task.default
      }

let --| The new task type can be added to the Play too
    Play =
      { Type = Ansible.Play.Type //\\ { tasks : List Task.Type }
      , default = Ansible.Play.default
      }

in  Play::{
    , hosts = "localhost"
    , tasks =
      [ Task::{
        , name = "Create a server"
        , `openstack.cloud.server` = Some Ansible.Collection.OpenstackCloud.Server::{
          , name = "my-server"
          }
        }
      , Task::{
        , name = "Create a volume"
        , `openstack.cloud.volume` = Some Ansible.Collection.OpenstackCloud.Volume::{
          , name = "my-volume"
          , size = 42
          }
        }
      ]
    }

That’s all, folks!


#2

The change has been in proposed in this gerrit topic. Note that the complete package weight more than 100MB so I split the collections in dedicated projects to ease the maintenance. It seems like the implementation is correct and I’ll merge the change soon.

Cheers!


#3

I released and froze the change with dhall-ansible version 0.2.1, and here is a PR to add the new version to the store.dhall-lang.org . There is a script to generate individual collection bindings here, and you can find some of them there. Let me know if you need a missing one, I can create more packages.