AWS re:Invent 2015

An end-to-end, over-the-top (OTT) video system is built of many interdependent architectural tiers, ranging from content preparation, content delivery, and subscriber and entitlement management, to analytics and recommendations. This talk will provide a detailed exploration of how to architect a media platform that allows for growth, scalability, security, and business changes at each tier, based on real-world experiences delivering over 100 Gbps of concurrent video traffic with 24/7/365 linear TV requirements. Finally, learn how Verizon uses AWS, including Amazon Redshift and Amazon Elastic MapReduce, to power its recently launched mobile video application Go90.

Using a mixture of AWS services and native applications, we address the following scaling challenges:

Content ingest, preparation, and distribution

Operation of a 24x7x365 Linear OTT Playout Platform

Common pitfalls with transcode and content preperation

Multi-DRM and packaging to allow cross platform playback

Efficient delivery and multi-CDN methodology to allow for a perfect experience globally

Kinesis as a dual purpose system for both analytics and concurrency access management

Integration with Machine Learning for an adaptive recommendation system, with real time integration between content history and advertising data

User, entitlement, and content management

General best practices for Cloud Architectures and their integration with Amazon Web Services; Infrastructure as Code, Disposable and immutable infrastructure, code deployment & release management, DevOps and Microservices Architectures

This session is great for architects, engineers, and CTOs within media and entertainment or others simply interested in decoupled architectures.

It's difficult to find off-the-shelf, open-source solutions for creating lean, simple, and language-agnostic data-processing pipelines for machine learning (ML). This session shows you how to use Amazon S3, Docker, Amazon EC2, Auto Scaling, and a number of open source libraries as cornerstones to build one. We also share our experience creating elastically scalable and robust ML infrastructure leveraging the Spot instance market.

Dive deep into specific, common use cases for enterprise customers while stepping through the process of building a cloud and IT transformation strategy leveraging the AWS Cloud Adoption Framework. We will build a prescriptive roadmap for a cloud journey leveraging best practices, common techniques, and real-world examples from other AWS successes.

Customer demand and competitive landscapes mean its not okay to move fast and break things. Whether you are following Agile, Continuous Integration (CI) or Continuous Deployment (CD), you cant afford to limit your releases, skip quality assurance (QA) testing, or let your customers beta-test your products. You need an on-demand QA platform that moves at the speed of development. Come learn how Rainforest leverages Mechanical Turk to build its network of 50,000 QA testers around the world and deliver on-demand QA through its API. We are able to deliver QA tests in 30 minutes that dont slow down development.

We are collecting tons of sensor data from billions of devices. How do you get the value from your IoT data sources? In this session, we will explore different strategies for collecting and ingesting data, understanding its frequency, and leveraging the potential of the cloud to analyze and predict trends and behavior to get most out of your deployed devices.

In this session, we show you how to use Amazon Route 53 to consolidate your DNS data and manage it centrally. Learn how to use Amazon Route 53 for public DNS and for private DNS in VPC, and also learn how to combine Amazon Route 53 private DNS with your own DNS infrastructure.

As data volumes grow, managing and scaling data pipelines for ETL and batch processing can be daunting. With more than 13.5 million learners worldwide, hundreds of courses, and thousands of instructors, Coursera manages over a hundred data pipelines for ETL, batch processing, and new product development.

In this session, we dive deep into AWS Data Pipeline and Dataduct, an open source framework built at Coursera to manage pipelines and create reusable patterns to expedite developer productivity. We share the lessons learned during our journey: from basic ETL processes, such as loading data from Amazon RDS to Amazon Redshift, to more sophisticated pipelines to power recommendation engines and search services.

Attendees learn:

Do's and donts of Data Pipeline

Using Dataduct to streamline your data pipelines

How to use Data Pipeline to power other data products, such as recommendation systems

Whats next for Dataduct

Adapting the capacity of your compute infrastructure to the demands of your applications is the domain of Auto Scaling. Adding and removing Amazon EC2 instances is only part of the story, though - there is more to it than first meets the eye. This session introduces the basics of how to use Auto Scaling before moving on to more advanced topics such as mixing Spot and On-Demand instances to optimize cost or strategies for blue/green deployments. If you have used Auto Scaling before, you can learn about useful new features like lifecycle hooks and step scaling policies that make Auto Scaling even more widely applicable.

This session explores building graph databases on AWS, examining common use cases, design patterns, and best practices. We then discuss the main options for running graph databases on AWS and go deeper into the Amazon DynamoDB storage backend plugin for Titan launched earlier this year. The Amazon Fulfillment team will share their story of running the Titan graph database on DynamoDB to track inventory going in and out of the company's fulfillment network. They provide best practices on running an efficient graph database at massive scale.

Learn how to monitor your database performance closely and troubleshoot database issues quickly using a variety of features provided by Amazon RDS and MySQL including database events, logs, and engine-specific features. You will also learn about the security best practices to use with Amazon RDS for MySQL as well as how to effectively move data between Amazon RDS and on-premises instances. Hear from Amazon RDS customer Airbnb about the best practices they have implemented in their RDS for MySQL architectures.