Executing Ansible playbooks in your Amazon EC2 Image Builder pipeline

Amazon Web Services Feed
Executing Ansible playbooks in your Amazon EC2 Image Builder pipeline

This post is contributed by Andrew Pearce – Sr. Systems Dev Engineer, AWS

Since launching Amazon EC2 Image Builder, many customers say they want to re-use existing investments in configuration management technologies (such as Ansible, Chef, or Puppet) with Image Builder pipelines.

In this blog, I walk through creating a document that can execute an Ansible playbook in an Image Builder pipeline. I then use the document to create an Image Builder component that can be added to an Image Builder image recipe. In addition to this blog, you can find further samples in the amazon-ec2-image-builder-samples GitHub repository.

Ansible playbook requirements

There are likely some changes required so your existing Ansible playbooks can work in Image Builder.

When executing Ansible within Image Builder, the playbook must be configured to execute using the localhost. In a traditional Ansible environment, the control server executes playbooks against remote hosts using a protocol like SSH or WinRM. In Image Builder, the host executing the playbook is also the host that must be configured.

There are a number of ways to accomplish this in Ansible. In this example, I set the value of hosts to 127.0.0.1, gather_facts to false, and connection to local. These values set execution at the localhost, prevent gathering Ansible facts about remote hosts, and force local execution.

In this walk through, I use the following sample playbook. This installs Apache, configures a default webpage, and ensures that Apache is enabled and running.

---
- name: Apache Hello World hosts: 127.0.0.1 gather_facts: false connection: local tasks: - name: Install Apache yum: name: httpd state: latest - name: Create a default page shell: echo "<h1>Hello world from EC2 Image Builder and Ansible</h1>" > /var/www/html/index.html - name: Enable Apache service: name=httpd enabled=yes state=started

Creating the Ansible document

This example uses the build phase to install Ansible, download the playbook from an S3 bucket, execute the playbook and cleanup the system. It also ensures that Apache is returning the correct content in both the validate and test phases.

I use Amazon Linux 2 as the target operating system, and assume that the playbook is already uploaded to my S3 bucket.

Document design diagram

To start, I must specify the top-level properties for the document. I specify a name and description to describe the document’s purpose, and a schemaVersion, which at the time of writing is 1.0. I also include a build phase, as I want this document to be used when building the image.

name: 'Ansible Playbook Execution on Amazon Linux 2'
description: 'This is a sample component that demonstrates how to download and execute an Ansible playbook against Amazon Linux 2.'
schemaVersion: 1.0
phases: - name: build steps:

The first required step is to install Ansible. With Amazon Linux 2, I can install Ansible using Amazon Linux Extras, so I use the ExecuteBash action to perform the work.

 - name: InstallAnsible action: ExecuteBash inputs: commands: - sudo amazon-linux-extras install -y ansible2

Once Ansible is installed, I must download the playbook for execution. I use the S3Download action to download the playbook and store it in a temporary location. Note that the S3Download action accepts a list of inputs, so a single S3Download step could download multiple files.

 - name: DownloadPlaybook action: S3Download inputs: - source: 's3://mybucket/my-playbook.yml' destination: '/tmp/my-playbook.yml'

Now Ansible is installed and the playbook is downloaded. I use the ExecuteBinary action to invoke Ansible and perform the work described in the playbook. This step uses a feature of the component management application called chaining.

Chaining allows you to reference the input or output values of a step in subsequent steps. In the following example, {{build.DownloadPlaybook.inputs[0].destination}} passes the string of the first “destination” property value in the DownloadPlaybook step, which executes in the build phase. The “[0]” indicates the first value in the array (position zero).

For more information about how chaining works, review the Component Manager documentation. In this example, I refer to the S3Download destination path to avoid mistyping the value, which causes the build to fail.

 - name: InvokeAnsible action: ExecuteBinary inputs: path: ansible-playbook arguments: - '{{build.DownloadPlaybook.inputs[0].destination}}'

Finally, I clean up the system, so I use another ExecuteBash action combined with chaining to delete the downloaded playbook.

 - name: DeletePlaybook action: ExecuteBash inputs: commands: - rm '{{build.DownloadPlaybook.inputs[0].destination}}'

Now, I have installed Ansible, downloaded the playbook, executed it, and cleaned up the system. The next step is to add a validate phase where I use curl to ensure that Apache responds with the desired content. Including a validate phase allows you to identify build issues more quickly. This fails the build before Image Builder creates an AMI and moves to the test phase.

In the validate phase, I use grep to confirm that Apache returned a value containing my required string. If the command fails, a non-zero exit code is returned, indicating a failure. This failure is returned to Image Builder and the image build is marked as “Failed”.

 - name: validate steps: - name: ValidateResponse action: ExecuteBash inputs: commands: - curl -s http://127.0.0.1 | grep "Hello world from EC2 Image Builder and Ansible"

After the validate phase executes, the EC2 instance is stopped and an Amazon Machine Image (AMI) is created. Image Builder moves to the test stage where a new EC2 instance is launched using the AMI that was created in the previous step. During this stage, the test phase of a document is executed.

To ensure that Apache is still functioning as required, I add a test phase to the document using the same curl command as the validate phase. This ensures that Apache is started and still returning the correct content.

 - name: test steps: - name: ValidateResponse action: ExecuteBash inputs: commands: - curl -s http://127.0.0.1 | grep "Hello world from EC2 Image Builder and Ansible"

Here is the complete document:

name: 'Ansible Playbook Execution on Amazon Linux 2'
description: 'This is a sample component that demonstrates how to download and execute an Ansible playbook against Amazon Linux 2.'
schemaVersion: 1.0
phases: - name: build steps: - name: InstallAnsible action: ExecuteBash inputs: commands: - sudo amazon-linux-extras install -y ansible2 - name: DownloadPlaybook action: S3Download inputs: - source: 's3://mybucket/playbooks/my-playbook.yml' destination: '/tmp/my-playbook.yml' - name: InvokeAnsible action: ExecuteBinary inputs: path: ansible-playbook arguments: - '{{build.DownloadPlaybook.inputs[0].destination}}' - name: DeletePlaybook action: ExecuteBash inputs: commands: - rm '{{build.DownloadPlaybook.inputs[0].destination}}' - name: validate steps: - name: ValidateResponse action: ExecuteBash inputs: commands: - curl -s http://127.0.0.1 | grep "Hello world from EC2 Image Builder and Ansible" - name: test steps: - name: ValidateResponse action: ExecuteBash inputs: commands: - curl -s http://127.0.0.1 | grep "Hello world from EC2 Image Builder and Ansible"

I now have a document that installs Ansible, downloads a playbook from an S3 bucket, and invokes Ansible to perform the work described in the playbook. I am also validating the installation was successful by ensuring Apache is returning the desired content.

Next, I use this document to create a component in Image Builder and assign the component to an image recipe. The image recipe could then be used when creating an image pipeline. The blog post Automate OS Image Build Pipelines with EC2 Image Builder provides a tutorial demonstrating how to create an image pipeline with the EC2 Image Builder console. For more information, review Getting started with EC2 Image Builder.

Conclusion

In this blog, I walk through creating a document that can execute Ansible playbooks for use in Image Builder. I define the required changes for Ansible playbooks and when each phase of the document would execute. I also note the amazon-ec2-image-builder-samples GitHub repository, which provides sample documents for executing Ansible playbooks and Chef recipes.

We continue to use this repository for publishing sample content. We hope Image Builder is providing value and making it easier for you to build virtual machine (VM) images. As always, keep sending us your feedback!