AWS re:Invent 2016

As customers build and run production microservices architectures based on containers, having powerful tools to manage the placement and scheduling of these workloads is critical. Amazon ECS allows customers to focus on building their application and removes the need for managing the cluster management software entirely. This session will focus on the capabilities of the new ECS task placement engine, the built-in ECS schedulers, and explore how customers can build custom schedulers using the ECS event stream and Blox, a newly announced open source scheduling framework that gives developers additional choice and control in how they manage containers in production.

Through a combination of Amazon ECS and open source technologies, customers are able to build portable CI/CD pipelines on AWS. As container based deployments become more complex, they require additional rigging for integration. In this session, we show how popular Apache products like Kakfa, Storm, and Zookeeper are being deployed on top of Amazon ECS. We hear from HERE, a provider of mapping data, technologies, and services to the automotive, consumer, and enterprise sectors about an approach that leverages Consul from Hashicorp and Amazon ECS clusters for short-cycle deployments and tag-based environment promotion.

Running and managing large scale applications with microservices architectures is hard and often requires operating complex container management infrastructure. Amazon EC2 Container Service (ECS) is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances. In this session, we will walk through a number of patterns used by our customers to run their microservices platforms. We will dive deep into some of the challenges of running microservices, such as load balancing, service discovery, and secrets management, and we’ll see how Amazon ECS can help address them. We'll also hear from Instacart how they use a blue/green deployment process to deploy services to ECS and how they manage configuration with a RDS-based metadata service.

Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.

At Capital One, we are using Docker and container technologies to advance microservices adoption, increase efficiencies of cloud resources, and decouple the application layer from the underlying infrastructure. Capital One is a federated organization with a “you build it, you own it” culture that provides autonomy and speed for delivery teams. Each federated team runs and operates their container management stack. In order for the federated teams to accelerate their cloud and container-based apps adoption, we created self-service automation tools for creation and operations management of container management stack.

Rent-A-Center’s challenge was to architect, deploy, and manage a mission-critical SAP Hybris ecommerce platform that could scale to 2 million users a month. Together with Flux7, an AWS Advanced Consulting Partner, Rent-A-Center created an AWS-based approach that would help deliver the solution to market faster, in a secure, highly available, PCI-compliant fashion. In this session, we walk through the implementation details of this solution and its challenges, and explore how Rent-A-Center is now able to achieve ROI through agility, scalability, security, and cost savings.

Customers from over all over the world streamed forty-two billion hours of Netflix content last year. Various Netflix batch jobs and an increasing number of service applications use containers for their processing. In this session, Netflix presents a deep dive on the motivations and the technology powering container deployment on top of Amazon Web Services. The session covers our approach to resource management and scheduling with the open source Fenzo library, along with details of how we integrate Docker and Netflix container scheduling running on AWS. We cover the approach we have taken to deliver AWS platform features to containers such as IAM roles, VPCs, security groups, metadata proxies, and user data. We want to take advantage of native AWS container resource management using Amazon ECS to reduce operational responsibilities. We are delivering these integrations in collaboration with the Amazon ECS engineering team. The session also shares some of the results so far, and lessons learned throughout our implementation and operations.

Join us to learn about the latest developments from Amazon ECS and the container ecosystem. Deepak Singh, General Manager of AWS Container Services, discusses the evolution of containers on AWS and shares our vision for continued innovation in this space. You also hear about how other companies are using the AWS container platform to innovate and build new businesses.

“Are you struggling with bulky images or slow push and pull times? In this session we will walk through the anatomy of a Docker image and provide techniques you can use to optimize images for faster pushes and pulls and reduce your overall storage footprint. We will discuss Docker image building (build containers versus runtime containers to remove unnecessary software), Docker image composition (minimizing the number of layers), the Docker Remote API (optimizing how images are pushed and pulled), and CI/CD Integration (automate building, versioning, and deploying images to production). We’ll also examine the tools that ECR provides to make Docker image management easier so that you can focus on building your application. Finally, we'll hear from Pinterest about how they use ECR and Docker, some valuable experiences gained along the way, and best practices for using ECR with Apache Mesos.”

Containers have had an incredibly large adoption rate since Docker was launched, especially from the developer community, as it provides an easy way to package, ship, and run applications. Securing your container-based application is now becoming a critical issue as applications move from development into production. In this session, you learn ways to implement storing secrets, distributing AWS privileges using IAM roles, protecting your container-based applications with vulnerability scans of container images, and incorporating automated checks into your continuous delivery workflow.