Keeping Ansible effortless with AWS Systems Manager

Amazon Web Services Feed
Keeping Ansible effortless with AWS Systems Manager

Introduction

Ansible is a powerful tool because it lets you handle many complicated tasks with minimal effort. Some time ago, I published running Ansible playbooks using Systems Manager blog when the first version of the AWS Systems Manager (SSM) document was released, which enabled support for Ansible. In that blog, I discussed the tight integration of SSM with other AWS services like AWS identity and Access Management (IAM) and AWS CloudTrail. The integration did not only simplify management, but also improved the security posture of your infrastructure. Since then, we received feedback on that first iteration of the integration. Customers requested additional features, such as GitHub integration, complex playbooks support, and more. Customers also provided ideas on how to improve the integration. As is customary with AWS, we listened.

We recently introduced more robust support for Ansible with Systems Manager, with the integration provided by a new SSM document called AWS-ApplyAnsiblePlaybooks. This new version incorporates many of the features our customers requested, including support for GitHub and complex playbooks. If you would like more information on all the new features, check creating associations that run Ansible playbooks documentation.

Overview of how Ansible works

Let’s walk through an example of how to use this enhanced integration. Keep in mind, this blog is not intended to teach you Ansible, rather focus on how SSM works with it. However, if you are interested in learning more, you can access the Ansible documentation.

For our example, let’s say you want to configure an Amazon EC2 instance to run an Apache web server and serve a custom page. With a brand-new instance, there are a few things you must do make that happen. Here is a description of what is required, in addition to the commands needed.

The following command can be used to install Apache in an Amazon Linux EC2 instance.

sudo yum install httpd

For Ubuntu or a Debian based distribution, the command is as follows.

sudo apt-get install apache

So that means you must know which operating system you are running, to determine which command to use. Ansible simplifies the installation process, and I and allows you to introduce some logic to solve this problem. More on that later.

After you install the software, you must make sure that the service is running. You can do that with the following command.

systemctl start httpd #for Amazon Linux

systemctl start apache2 #for Ubuntu

The next step would be to customize and copy the home webpage for your web server. You would typically do this with a text editor and HTML. We don’t go into the details of HTML coding, so for the purpose of this post we use a simple page, with some variables for customization to showcase the Ansible automation.

As you can see, several steps are required to get started. If the process is automated to support multiple operating systems, then some logic handling must be added to account for the different ways of installing software (yum vs apt-get).

Let’s discuss how we can do this using Ansible and the new SSM document to run Ansible playbooks.

Without going into too much detail on how Ansible works, you can use a playbook with the following tasks:

- name: Gather ec2 facts ec2_metadata_facts:
- name: install apache on redhat or centos instances yum: name=httpd state=present when: ansible_os_family == "RedHat"
- name: install apache on debian or ubuntu instances apt: name=apache2 state=present when: ansible_os_family == "Debian"
- name: template the index file for debian template: src=index.html.j2 dest=/var/www/html/index.html owner=www-data group=www-data mode=0644 when: ansible_os_family == "Debian"
- name: template the index file for Redhat template: src=index.html.j2 dest=/var/www/html/index.html owner=apache group=apache mode=0644 when: ansible_os_family == "RedHat"
- name: enable apache on startup and start service for redhat or centos service: name=httpd enabled=yes state=started when: ansible_os_family == "RedHat" - name: enable apache on startup and start service for debian or ubuntu service: name=apache2 enabled=yes state=started when: ansible_os_family == "Debian"

The tasks defined in the playbook perform the following:

1. Gather Amazon EC2 metadata facts and load them into a dictionary variable. This variable can then be used anywhere in the automation.
2. There are two tasks to install Apache. One for distributions that require the yum installer. Another one is for Debian based distributions, which use a different package manager. The tasks are executed conditionally, with Ansible running only one of the tasks, depending on the operating system.
3. Copy a template file for the home page (index.html), using the distribution-specific process.
4. There are also two tasks to start the service, appropriate for the specific Linux distributions.

As you can see, with this playbook we were able to install and configure a web server. We also extended the automation to be flexible enough to support multiple Linux distributions.

All that automation could be expressed in a single Ansible playbook. However, because we are templating a file (index.html.j2), we also must include that file as part of the automation. To make our automation more modular, we could use the Ansible concept of roles. A role is a framework to provide an independent set of files, that can be used for sharing and reusing automation files. In our example, we could break the automation we designed, create a role, and call it apache. This role can now be shared and distributed, so others don’t have to write all the code necessary to set up an Apache server. There is more interesting information about how roles work in the Ansible roles documentation.

Putting it all together, we can structure a folder called “automation” that looks like this:

Automation
├server.yml
├apache+ ├tasks ├ main.yml ├templates ├index.html.j2

The file main.yml has the same tasks we outlined before. As you can see, the file and the template source are now grouped in a file structure for a role called apache. Then at the top level, we have a file called server.yml; the main playbook. That file calls the role. It looks as follows.

--- - hosts: all become: true become_method: sudo roles: - apache

To use and maintain this automation file, you could keep it in a source control system, and make it part of a deployment pipeline. For the purposes of our example, let’s say your pipeline compresses the automation files and stores them in Amazon S3 with the name automation.zip. Now let us discuss how can we use SSM to deploy the automation?

Integration with AWS Systems Manager

Imagine you have a fleet of Apache web servers that must be configured to serve the same web content. They all have a resource tag with a key called “role” and value of “webserver” to identify them. Your goal is to set up a process that verifies if these instances are configured properly all the time. Let’s run through an example of how the new Ansible SSM document can be used to simplify operations.

1. Log in to the AWS Management Console.
2. Go to State Manager.
3. Click on the Association button.
4. Provide an optional name for the association. You can call it webserver.
5. Select AWS-ApplyAnsiblePlaybooks from the list of SSM documents
6. In the source type field, select S3.
7. In the source info field, enter the path to where the zip file is located in Amazon S3. Enter the file location using JSON notation in the following format. For example: {“path”:”https://example-bucket.s3.amazonaws.com/automation.zip”}
8. On the install dependencies field, select true. This installs Ansible and all its dependencies for you.
9. On the playbook file, we must specify the full relative path of where the main playbook is located. Since we compressed the files into a folder called automation, and the main file is called server.yml. We specify “automation/server.yml” in this field.
10. On the extra variable field, we can specify an additional variable to be used during execution.
11. On the targets section, we select the specify tags option. Then, enter the tag definition for our web server. That is tag key: role, and tag value: webserver. This targets the execution of Systems Manager State Manager to all the instances that have a tag with the key role and the value webserver.
12. On specify schedule, define how often you would like to run this association.
13. Now click on Create Association.

You can also create the Systems Manager association using AWS SDK tools or AWS CLI. Here is the command to use for the AWS CLI:

aws ssm create-association --name "AWS-ApplyAnsiblePlaybooks" --parameters '{"SourceType":["S3"],"SourceInfo":["{"path": "https://example-bucket.s3.amazonaws.com/automation.zip"}"],"InstallDependencies":["True"],"PlaybookFile":["automation/playbook.yml"],"ExtraVariables":["myregion=us-east-1"],"Check":["False"],"Verbose":["-v"]}' --targets '[{"Key":"tag:role","Values":["webserver"]}]' --max-concurrency "50" --max-errors "0" --region us-east-1

After the command is run, it returns an AssociationDescription JSON structure, which contains the details of the new association.

This is how SSM Association can be created that runs Ansible playbooks on desired instances. Now you have full control of your Ansible automation using SSM. You can track the results of the automation execution from the Systems Manager console and also view the results. You can optionally enable all the output of the execution to an Amazon S3 bucket.

Cleanup

Be sure to terminate any ec2 instances you launched to walk through this example.

Conclusion

In this blog post, we’ve reviewed some basic functionality of Ansible and went over the new Ansible integration with Systems Manager. We highlighted how the SSM scalability features extend the power of Ansible, and provide you more granular control and enhanced security of your configuration management. To learn more about the integration, visit the Systems Manager documentation.

About the Author

Andres Silva is a Principal Specialist Solutions Architect with the Management Tools team at AWS .He has been working with AWS technology for more than 9 years. Andres works closely with the  service teams to design solutions at scale that help customers implement and support complex cloud infrastructures.When he is not building cloud automation he enjoys skateboarding with his 2 kids.