AWS re:Invent 2017

Cloud is the new normal, and organizations are deploying different types workloads on AWS. Understanding the performance efficiency and overall application performance is critical to ensuring that you can scale your workload to meet the demands of your customers. Understanding how well your application performs over time helps you to continuously improve and innovate your software to get the most out of the AWS platform. If you aren't measuring custom application metrics, you are operating your software blindly and cannot pinpoint areas of improvement. Learn how to use Amazon CloudWatch custom metrics, alerts, dashboards and AWS X-Ray to architect an application monitoring service to provide insight to your workload's performance.

In this session, we first look at common approaches to refactoring common legacy .NET applications to microservices and AWS serverless architectures. We also look at modern approaches to .NET-based architectures on AWS. We then elaborate on running .NET Core microservices in Docker containers natively on Linux in AWS while examining the use of AWS SDK and .NET Core platform. We also look at the use of the various AWS services such as Amazon SNS, Amazon SQS, Amazon Kinesis, and Amazon DynamoDB, which provide the backbone of the platform. For example, Experian Consumer Services runs a large ecommerce platform that is now cloud based in the AWS. We look at how they went from monolithic platform to microservices, primarily in .NET Core. With a heavy push to move to Java and open source, we look at the development process, which started in the beta days of .NET Core, and how the direction Microsoft was going allowed them to use existing C# skills while pushing themselves to innovate in AWS. The large, single team of Windows based developers was broken down into several small teams to allow for rapid development into an all Linux environment.

Many customers want a disaster recovery environment, and they want to use this environment daily and know that it's in sync with and can support a production workload. This leads them to an active-active architecture. In other cases, users like Netflix and Lyft are distributed over large geographies. In these cases, multi-region active-active deployments are not optional. Designing these architectures is more complicated than it appears, as data being generated at one end needs to be synced with data at the other end. There are also consistency issues to consider. One needs to make trade-off decisions on cost, performance, and consistency. Further complicating matters is the variety of data stores used in the architecture results in a variety replication methods. In this session, we explore how to design an active-active multi-region architecture using AWS services, including Amazon Route 53, Amazon RDS multi-region replication, AWS DMS, and Amazon DynamoDB Streams. We discuss the challenges, trade-offs, and solutions.

Reinforcement Learning (RL) can be used to solve real-world problems in robotics and conversational engines without supervision. AI algorithms that observe their surroundings and learn are considered to be the ultimate forms of AI. The RL use cases shines in multi-agent scenarios where each agent reacts in real-time to the changing situation. In this session, we explain RL, the theory, and the algorithms used. We show an MXNet-based demo that will automatically learn to play a game. We use a game and show how an agent powered by MXNet takes actions to win. Initially, you notice that the agent making very little progress, but after a few dozen iterations, it can play the game better than any human being. You can generalize this to real world problems. RL is currently used today in robotics, gaming, autonomous vehicle control, spoken language systems and many more. In this talk, I will be using Amazon EC2 P2 instances, AWS deep learning AMI, MXnet deep learning framework, Amazon EBS, and Amazon S3.

When engineering teams take on a new project, they often optimize for performance, availability, or fault tolerance. More experienced teams can optimize for these variables simultaneously. Netflix adds an additional variable: feature velocity. Most companies try to optimize for feature velocity through process improvements and engineering hierarchy, but Netflix optimizes for feature velocity through explicit architectural decisions. Mental models of approaching availability help us understand the tension between these engineering variables. For example, understanding the distinction between accidental complexity and essential complexity can help you decide whether to invest engineering effort into simplifying your stack or expanding the surface area of functional output. The Chaos team and the Traffic team interact with other teams at Netflix under an assumption of Essential Complexity. Incident remediation, approaches to automation, and diversity of engineering can all be understood through the perspective of these mental models. With insight and diligence, these models can be applied to improve availability over time and drift into success.

Every day, systems architects and cloud architects have to size cloud workloads for performance and efficiency. Do you choose T2, C3, C4, M3, or something else for your Amazon Elastic Compute Cloud (Amazon EC2) instance type? Do you need more CPUs, memory, or both? What about distributed applications across regions and Availability Zones? How do IT teams determine the right instance family and size for AWS workloads? Turbonomic solves these challenges with you. Their real-time hybrid cloud management platform can ensure that your workloads get the right resources in real time to assure performance across the compute, storage, network, application, and database layers of AWS, and across your hybrid cloud infrastructure. Get a crash course in understanding workload performance characteristics, and how Turbonomic matches to AWS resources to assure real-time, efficient performance for your AWS environment, with the ability to fully automate these processes. Whether you're new to the platform or regular users of Amazon EC2, learn to take the guesswork out of what makes each Amazon EC2 instance family unique and appropriate for your business and technical requirements. Session sponsored by Turbonomic, Inc.

The BBC iPlayer is the biggest audio and video-on-demand service in the UK. Over one-third of the country submits 10 million video playback requests every day, and the service publishes over 10,000 hours of media every week. Moving iPlayer to the cloud has enabled the BBC to shorten the time-to-market of content from 10 hours to 15 minutes. In this session, the BBC's lead architect describes the approach behind creating iPlayer architecture, which uses Amazon SQS and Amazon SNS in several ways to improve elasticity, reliability, and maintainability. You see how BBC uses AWS messaging to choreograph the 200 microservices in the iPlayer pipeline, maintain data consistency as media traverses the pipeline, and refresh caches to ensure timely delivery of media to users. This is a rare opportunity to see the internal workings and best practices of one of the largest on-demand content delivery systems operating today.

This talk includes a story and a recipe. The story is about a nerd who bought his first motorbike, got a license for it, and started hacking to make it interact and talk, all in two months. The recipe is a technical one that explains how to use Amazon Lex and Amazon Lambda to quickly prototype and deploy a serverless chatbot connected with an embedded device in order to realize an Internet of Things (IoT) application. We discuss how you can integrate your IoT application with Amazon Lex using AWS Lambda and the Amazon API Gateway, how to exchange session data to have a contextual conversation, and how to provide a successful bot experience. Expect to leave this session knowing how to build, deploy, and publish a bot, and how to attach it to an IoT device—with the potential to bringing to life any object that surrounds you.

As serverless architectures become more popular, customers need a framework of patterns to help them identify how they can leverage AWS to deploy their workloads without managing servers or operating systems. This session describes reusable serverless patterns while considering costs. For each pattern, we provide operational and security best practices and discuss potential pitfalls and nuances. We also discuss the considerations for moving an existing server-based workload to a serverless architecture. The patterns use services like AWS Lambda, Amazon API Gateway, Amazon Kinesis Streams, Amazon Kinesis Analytics, Amazon DynamoDB, Amazon S3, AWS Step Functions, AWS Config, AWS X-Ray, and Amazon Athena. This session can help you recognize candidates for serverless architectures in your own organizations and understand areas of potential savings and increased agility. What's new in 2017: using X-Ray in Lambda for tracing and operational insight; a pattern on high performance computing (HPC) using Lambda at scale; how a query can be achieved using Athena; Step Functions as a way to handle orchestration for both the Automation and Batch patterns; a pattern for Security Automation using AWS Config rules to detect and automatically remediate violations of security standards; how to validate API parameters in API Gateway to protect your API back-ends; and a solid focus on CI/CD development pipelines for serverless, which includes testing, deploying, and versioning (SAM tools).

The recent launch of VMware Cloud on AWS gives customers new options for addressing several use cases, including cloud migration, hybrid deployments, and disaster recovery. We introduce and describe design patterns for incorporating VMware Cloud on AWS into existing architecture and detail how the service's capabilities can influence future architectural plans. We explore design considerations and nuances for integrating VMware Cloud on AWS Software Defined Data Centers (SDDCs) with native AWS services, enabling you to use each platform's benefits. Architects, system operators, and anyone looking to understand VMware Cloud on AWS will walk away with examples and options for solving challenging use cases with this new, exciting service.