AWS Feed
Use Kubernetes Pod Templates with EMR on EKS to simplify running Spark workloads and control costs
Today, we are launching EMR on EKS support for Pod Templates to make it simple to run Spark jobs on shared EKS clusters. A Pod is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers. Pod Templates are specifications which determine how each Pod runs. Customers often consolidate multiple applications on a shared EKS cluster to improve utilization and save costs. However, each application may have different requirements. For example, you may want to run performance intensive workloads such as ML model training jobs on SSD-backed instances for better performance, or ad-hoc workloads on Spot instances for lower cost. You can also schedule a separate logging container to forward logs to your existing monitoring application. With this release, you can use Pod Templates with EMR on EKS to configure how to run Spark jobs on shared EKS clusters.