Amazon Web Services Feed
Field Notes: Choosing a Rehost Migration Tool – CloudEndure or AWS SMS
Field Notes provides hands-on technical guidance from AWS Solutions Architects, consultants, and technical account managers, based on their experiences in the field solving real-world business problems for customers.
For customers that choose a rehost migration strategy (also known as lift-and-shift) for their large-scale workloads, the recommendation is to select a tool that will orchestrate, automate and schedule the migration with minimum downtime. CloudEndure Migration and AWS Server Migration Service (AWS SMS) are two services provided by AWS that both automate rehost migration, but customers often struggle determining which service is a best fit for their migration use case.
CloudEndure Migration is a block-level replication tool that simplifies the process of migrating applications from physical, virtual, and cloud-based servers to AWS.
Figure 1 – CloudEndure Migration
AWS Server Migration Service is an agentless migration service to migrate on-premises virtual machines to AWS using virtual appliance.
Figure 2 – AWS Server Migration Service
In this post, I provide some considerations and patterns where it’s recommended based on your migration requirements to choose one tool over the other.
What type of infrastructure are you migrating?
First, consider your source infrastructure:
CloudEndure Migration supports any source infrastructure as long as it runs on x86 operating systems supported by Amazon Elastic Cloud Compute (EC2). This includes physical servers, P2V (virtual servers converted from physical), VMware, Hyper-V, and other cloud providers like Azure, GCP, IBM, or Oracle (see Supported Operating Systems). Be aware of some limits that are specific to some the operating systems. For example, dynamic type Windows disks that have GPT partitions are not supported. For more information, see How do I determine if my disks are GPT or dynamic?
AWS SMS supports migrations of virtual machines from VMware, Hyper-V or Microsoft Azure. AWS SMS doesn’t currently support physical infrastructure or other public cloud provider except Microsoft Azure. For a full list of supported operating systems, see Operating Systems Supported by AWS SMS.
If your source environment includes bare metal servers, and you can install agents (more on agents in the next section), the recommendation is to use CloudEndure Migration. You can use a combination of CloudEndure Migration and AWS SMS in an environment that has a mix of physical and virtual servers, but this approach adds unnecessary complexity to your operation– you must maintain both as opposed to having a unified solution that works for all sources.
Do you require agent or agentless solution?
CloudEndure Migration requires that you install CloudEndure Agent individually on each server or virtual machine with root/admin access. CloudEndure Agent is a lightweight software (utilizes approximately 5% CPU and 250MB of RAM) and non-disruptive. CloudEndure Agent performs an initial block-level read for the content of the volumes attached to the server and replicates it to the replication server. After the initial sync is complete, the agent then captures only the writes and synchronizes any block-level modification to the replication server. For handshake, monitoring, and management purposes, the agent requires egress access on TCP port 443 with the CloudEndure User Console. For communicating with the replication server in your account in AWS, it requires egress access on TCP port 1500. For list of networking requirements, see Networking and Ports.
AWS SMS doesn’t require an agent on each VM. Instead, it uses the Server Migration Connector (SMS Connector), which is a pre-configured FreeBSD virtual machine that you install on your on-premises virtualization environment, on the hypervisor level, before the migration can start. The format of the SMS Connector varies based on your source environment (see Install the Server Migration Connector for types). The SMS Connector is responsible for taking snapshots, sending them to AWS SMS, and converting them to an Amazon Machine Image (AMI).
In some situations you can’t meet some of the agent requirements. For example, you may not be able to have admin access on source machines to install the agent, or maybe you just prefer an agentless solution that requires less administration tasks on the operating system level. In such cases, assuming you have the necessary access to the hypervisor (or Azure Portal), AWS SMS is a better fit.
How frequently do you require source environment to be replicated?
With CloudEndure Migration, once the agent is installed and activated, it begins initial replication, reading all of the data on source machines at the block level and replicating it to a staging environment in the customer’s individual account in their preferred target region on AWS. The initial replication can take anywhere from several minutes to several days, depending on the amount of data to be replicated and the bandwidth available between the source and target infrastructure. After the initial replication is complete, the source machines are continuously monitored to ensure constant synchronization, up to the last second. Any changes to source machines are continuously and asynchronously replicated into the “staging area” in the target infrastructure. The replication at this point enters a Continuous Data Replication (CDR) state.
The block-level replication can be performed on any type of disk as long as it’s presented to CloudEndure Migration as a block device. This includes physical disks, disks hosted on Storage Area Network (SAN), VMDK, VHD and iSCSI.
The continuous, asynchronous, block-level replication maintains the data on AWS up-to-date with sub-second latency, and that enables you to achieve a cutover time between 5-20 minutes, depending on the operating systems and network latency. When the replication reaches the CDR state, you can launch a target machine in Test mode. Once you complete all your testing and are ready to fully transition your machine to the target cloud, you can perform the Cutover mode action. This process is fully automated. The following image shows the steps CloudEndure Migration performs during the conversion. For more details, see Performing a Migration Cutover.
Figure 3 – CloudEndure Target Machine Launch
The block-level replication and CDR make CloudEndure Migration a better fit for use cases where you require near real-time replication (i.e., the data in the destination environment is replicated sub-seconds behind the source environment.)
AWS SMS replication is done by taking snapshots of source machines. After you configure the SMS Connector in the virtualized environment and you create the replication job, the replication job starts and performs the following steps:
- Take a snapshot for the source VM
- Export VM to VMDK/VHD file for vCenter/Hyper-V respectively
- Upload VMDK /VHD to the S3 bucket
- Clean the snapshot
- Convert VMDK/VHD file to EBS Snapshot using VM import/export
- Delete the VMDK/VHD file in the S3 bucket
- Create an AMI
These steps are fully automated and are repeated for each recurrence of the replication job. Since AWS SMS uses snapshot-based replication, the subsequent snapshot is taken incrementally and, unlike CloudEndure Migration, there is no need for a staging area to receive the replication data.
Currently, the minimum time between reach replication in SMS is 1-hour, and the maximum is 24-hours. Using this setup, you still can build the migrated environment in minutes, but the migrated environment is at least 1-hour behind the source environment. If your migration requirements tolerate the 1-hour window, then AWS SMS can be a good fit, otherwise you should use CloudEndure Migration.
Do you require migration to be managed entirely from AWS Console?
Currently, CloudEndure Migration is a SaaS offering that you administer in the CloudEndure Console outside of the AWS Management Console, so you need two separate accounts, an AWS account and a CloudEndure account.
Figure 4 – CloudEndure Console
AWS SMS is a managed AWS service, so you operate it entirely from inside your AWS account without having to use an external portal or additional accounts.
If you have such a requirement to operate centrally from within AWS Console, currently, AWS SMS is the right choice.
What level of customization do you require after launching target environment?
In many cases, customers run discovery tools (like TSO Logic) to gather information about on-premises data centers. This information includes server utilizations, dependency mapping, or Total Cost of Ownership (TCO) estimate. Most discovery tools provide rightsizing for on-premises servers and recommend instance types and configurations when migrating these servers to Amazon EC2 on AWS. Both CloudEndure Migration and AWS SMS include these capabilities in different ways.
CloudEndure Migration uses Blueprint, which is a set of instructions on how to launch a target machine for a selected source machine. The Blueprint settings serve as the base settings for the creation of the Target machine. Using Blueprint, you can customize EC2 properties like: machine type, launch type, subnet, security group, private or public IP, disks type (standard, SSD or provisioned SSD) and others. CloudEndure Migration can also do the rightsizing by launching target machines that best match the current hardware (RAM, CPU, Cores) on source machine.
Figure 5 – CloudEndure Blueprint Configurations
In some cases, you may want to import the recommendations from a discovery tool without having to customize the blueprint manually. This is also supported by CloudEndure Migration using the mass blueprint setter script which takes the inventory and rightsizing configurations in csv format and imports them into the blueprint configurations using CloudEndure API. The configuration is then used to launch target machines. For more details, see CloudEndure API.
AWS SMS supports similar functionality when choosing Application Migration. Whereas server migration is accomplished by replicating a single server as an Amazon Machine Image (AMI), application migration replicates all of the servers in an application as AMIs and generates an AWS CloudFormation template to launch them in a coordinated fashion.
The following steps launch and migrate an application:
- Create a new application (i.e. SharePoint)
- Add servers to the application (i.e. database and application servers)
- Configure replication settings (i.e. license type)
- Configure launch settings: this defines how the application will be launched on EC2 instances. Some of the configurations you can specify: order of launch (launch database server first and then app server), instance type, VPC, subnet, security group, whether the application is publicly accessible or not and, optionally, you could add post launch scripts.
Figure 6 – AWS SMS Target Instance Configurations
From there, you start the replication and launch the application as per the configuration above. AWS SMS generates a CloudFormation template for the application settings that you can download, edit, and use for later launches. See also Migrating a SharePoint application using the AWS Server Migration Service on the AWS Compute Blog.
If you have a requirement to import your discovery and right-sizing data and use them to launch your target machines, then CloudEndure Migration is the right tool. If you have a requirement for generating a reusable CloudFormation template of migrated environment, then AWS SMS is the right tool.
Cost
You can use both CloudEndure Migration and AWS SMS at no charge. You are only charged for the resources each tool creates in your account to complete the migration.
Additional charges for each tool are as follows:
CloudEndure Migration uses a staging area to receive replication data. The staging area contains low-cost resources to keep your cost minimum. By default, CloudEndure Migration uses the t3.small EC2 instance type for replication server and EBS magnetic for its attached volumes as long as the volume is less than 500 GB in size. There is a one-to-one mapping between each EBS volume attached to the replication server, and a disk attached to a server in your source environment. You should also consider that depending on the number of disks you replicate from your source, you may need to have more than one replication server. The typical ratio of volumes to replication server is 15:1.
CloudEndure Migration also incurs charges for storing the snapshot it takes for each volume. There are 5-7 internal snapshots needed to replicate each disk. Frequency and exact number depend on various factors, such as change rate on the Source machine and network stability. Snapshots that are no longer needed, for example, after a source server has been migrated, are deleted automatically.
The AWS SMS is similar in incurring charges for storing snapshots it takes for every replication. The number of snapshots depends on frequency of the replication job (from 1 to 24 hours). To prevent additional charges, delete snapshot copies you no longer need. AWS SMS also replicates server volumes from your on-premises environment to Amazon S3 temporarily and purges them from Amazon S3 right after creating Amazon EBS snapshots, incurring a transient charge for S3.
Encryption
CloudEndure Migration supports encryption at rest and in transit. For encryption in transit, CloudEndure Migration uses Advanced Encryption Standard (AES-256) to encrypt replication traffic between agents in the source environment and the replication server. The traffic is passed using TCP port 1500. When the CloudEndure agent is installed on each source server, CloudEndure Migration generates a separate encryption key and passes it along to the designated replication using Transport Layer Security (TLS). The key is kept there in memory (confirm) for decryption.
For encryption at rest, CloudEndure Migration gives you the option to store your volumes encrypted in the staging area. You do that by checking Enable volume encryption in the project’s replication settings. From there, you choose either the default encryption key, or a custom KMS CMK key. Note that CloudEndure Migration doesn’t support OS-based disk encryption features such as BitLocker. These should be disabled before using any CloudEndure services.
Figure 7- CloudEndure Volume Encryption
AWS SMS creates an Amazon S3 bucket in the Region on your behalf, with server-side encryption enabled and a bucket policy to delete any items in the bucket after seven days. When you configure a replication job you have the option to enable AMI encryption. If you do so, AWS SMS encrypts the generated AMIs. Your default CMK is used unless you specify a non-default CMK. Replicated server volumes are encrypted in transit by Transport Layer Security TLS 1.2.
Disaster Recovery (DR)
In many cases, I work with customers who start their experience on AWS by moving or building a disaster recovery site on AWS. Sometime later, they realize the benefits of AWS and they decide to migrate their entire workload.
Although AWS SMS and CloudEndure Migration products are not designed for disaster recovery purposes, in this scenario, it is easier for customers to use CloudEndure Disaster Recovery for the DR project, and then leverage their familiarity with the product by using CloudEndure Migration for the migration portion. CloudEndure Disaster Recovery also supports region-to-region disaster recovery within AWS. To learn more, see CloudEndure Disaster Recovery.
Conclusion
Both CloudEndure Migration and AWS SMS are good options for a lift-and-shift migration. In this post, I reviewed a few use cases where one service is better fit over the other. For example, CloudEndure Migration is a better option when you migrate physical servers to AWS, when you want to deploy a block-level replication solution, or when you have near real-time replication requirements. In contrast, AWS SMS is a better option when you want to leverage an agentless solution, when you can tolerate one hour between replication jobs, or when you want a solution that’s managed from one place in the AWS Management Console.
You can get started here with CloudEndure Migration and AWS SMS. If you have comments about this blog post, please submit them in the comments section. I look forward to hearing from you.