AWS Feed
GoDaddy’s journey to the cloud and their Standard Cloud Platform
In this blog post, we explore GoDaddy’s journey to the cloud and their Public Cloud Portal, an application created to onboard engineering teams to AWS. GoDaddy started this journey in early 2018 when they announced their partnership with AWS. We’ll focus on how GoDaddy created a service to enable thousands of employees and hundreds of Scrum teams to develop and migrate to AWS.
Balancing agility and quality
GoDaddy realized early in the migration that in order to scale, they had to create a model that balanced agility and quality. They needed to focus on scaling DevOps, financial responsibility, and security requirements to the large number of teams that were migrating to AWS. What they built is an application that can be used by engineering teams to quickly and responsibly onboard to AWS. The engineering team made the application and its code available internally, thereby creating a feedback loop with their customers.
Pillars for decision-making
GoDaddy focused on some core pillars to guide their decision-making.
Security
GoDaddy used the AWS migration to level up their security stance by using AWS Systems Manager, AWS Config, AWS Organizations, AWS Identity and Access Management (IAM), AWS CloudFormation, and AWS Service Catalog. These services and resources are engineered to provide the appropriate access to the appropriate teams. For example, GoDaddy is using Systems Manager to help operations teams perform tasks in accounts. They use AWS Config for constant configuration monitoring. Standard IAM roles and policies are provided based on the needs of the engineering teams. Lastly, GoDaddy uses AWS Service Catalog so that engineering teams can deploy approved service offerings to their accounts.
Financial
GoDaddy focused on making financially responsible decisions as they moved to the cloud. When application teams set up their accounts, they must pick budget constraints, which become more defined and monitored as the teams move toward production. The application teams have more freedom in development environments so they can determine their architecture and start measuring their spending. After they have their accounts, the teams’ financial dashboard is updated every eight hours so they can look at their spending across lines of business, environments, or by application.
Agility and internal contribution
While adhering to financial and security guidance, GoDaddy still wanted their application teams to have the freedom to run at their own pace. To scale their offerings and knowledge, GoDaddy decided to use an internal open contribution model for their development. By using internal code repositories, they were able to scale architectural standards to all the engineering teams. These repositories became the place to discuss changes for an approved architecture. All the engineering teams were able to take part in the discussion. GoDaddy also created a process to review and create patterns quickly. They establish and maintain all the automation to deploy and manage that pattern through its lifecycle.
Multi-account Landing Zone for teams
For compliance and auditing reasons, GoDaddy has separate organizations in AWS Organizations for PCI and non-PCI accounts. The accounts of GoDaddy development teams that have an application dealing with PCI data are built in the PCI organization. All other applications have accounts in the non-PCI organization.
Figure 1: GoDaddy organization model
Under a particular product or service in the organization is where each development team gets its Landing Zone of accounts for application deployment. GoDaddy uses an automate-everything approach. They use AWS CloudFormation and AWS Service Catalog to deploy the default multi-account Landing Zone, depending on the needs of the development team as it moves to production. The team starts with a DevPrivate or sandbox account. When the team moves on to development, it receives dev and test accounts. In production, the team receives stage, OTE, and prod accounts to support their full application lifecycle environment. The team uses an account creation wizard to make its architecture deployment choices. The wizard collects operational information required to create the project teams’ baseline account structure and AWS service regional deployment and configuration.
Global accounts and shared services
GoDaddy recognized a need to create shared capabilities that they could provision to all accounts.
Payer account is the root organization account, has access to all linked accounts, and is the account for consolidated billing.
Logging account contains consolidated logs (AWS CloudTrail, AWS Config, VPC flow logs, and so on) from every account in the organization.
Audit account runs security and vulnerability scans across the accounts using an IAM role that has access to all the accounts.
Automation account runs account setup scripts. Contains information and metadata about all the accounts inside the organization. Contains global buckets that other accounts inside the organization get read-only access to. This account also has access to all other accounts in the organization.
Security account contains replica logs sent from the logging account.
Networking account enables AWS connectivity to the GoDaddy data center through AWS Direct Connect. The automation account fetches AWS Direct Connect information from this account.
KMS account contains the AWS Key Management Service (AWS KMS) keys for all the accounts. An individual account get access to their keys only inside this account.
Management account is a jump account for global privileged roles for access to all accounts.
GoldenAmi account facilitates node rotation capabilities across all accounts. Fetches account metadata from the automation account and has access through a role to all accounts.
Landing Zone capabilities
GoDaddy created capabilities through their Landing Zone that allow them to securely and reliably scale AWS usage to their developers. GoDaddy needed a way to manage the lifecycle of AMIs so that developers could trust the instances they were running in AWS are approved for use and meet GoDaddy compliance standards. GoDaddy also needed a way to manage architectural patterns as building blocks. Because they want their developers to focus on application development, the use of standard building blocks gives developers confidence that the resources they deploy follow GoDaddy best practices.
Golden AMIs
Because immutability and security are a top priority for their applications, GoDaddy created a way to manage their instances and AMIs at scale. Their process allows for AMIs to be patched regularly and to be lifecycled on a regular basis. They created a standard deployment mechanism and pattern with automation to rotate instances daily. It’s used across all teams. This AMI automation and automatic 24-hour node rotation allows them to meet security and compliance requirements. It also promotes highly resilient applications.
By using a highly automated process, GoDaddy can deliver consistent and secure AMIs to their development teams with high reliability. These are the steps GoDaddy follows to build a golden AMI:
- GoDaddy starts with the most recently available AMI from AWS Marketplace.
- GoDaddy uses automation to patch and apply security hardening scripts per their standards.
- After the hardening is complete, GoDaddy builds their golden AMI.
- GoDaddy provisions the golden AMI and uses their security scans to validate compliance before they publish it broadly. GoDaddy is expanding the testing to include standard functional tests for their application teams. These scans allow them to validate the golden AMI before it is provisioned and used by GoDaddy teams.
- After the golden AMI passes the scans, it is copied to AWS Regions and then shared to the accounts that need them.
- The GoDaddy automation stores the AMI IDs in Systems Manager, providing a standard publishing mechanism for teams. This fully automated process makes it possible to quickly build and distribute AMIs to their accounts on a regular basis.
GoDaddy uses Systems Manager to create an AMI rotation process so all teams can safely rotate their AMIs each day. This process allows teams to pick the most recent AMIs as they become available and rotate all instances daily, even if a new AMI is not available. This process runs on any offering the teams create. New AMIs are published at least one per month (more often depending on available patches and source AMI updates).
Architectural pattern deployment
To make the consumption of AWS easier for their developers, GoDaddy wanted to create and manage building blocks that were approved for use in AWS. They created a process to distribute these building blocks to all of their AWS accounts and the Regions in which they operate. They didn’t want the process to hinder their application teams from improving on these building blocks or creating their own. GoDaddy uses AWS Service Catalog to manage the lifecycle of their approved patterns. To allow their developers to create building blocks, GoDaddy uses automation and internal public repositories to hold these patterns. Their developers can suggest changes to existing building blocks or even propose new ones. The use of code repositories fosters collaboration among the many people in the GoDaddy engineering community. After a building block is approved for use in AWS, it is deployed through AWS Service Catalog. This process provides version control of their offerings. Developers can use AWS CloudFormation to consume these building blocks.
Conclusion
GoDaddy continues to evolve their cloud offerings by keeping up with AWS innovation and listening to their customers. If you would like to build similar capabilities, be sure to look at AWS Config, AWS Organizations, AWS Service Catalog, AWS CloudFormation, and AWS Systems Manager. If you would like to start with a managed solution, you can use AWS Control Tower.