AWS re:Invent 2017

Expedia uses Amazon Elasticsearch Service (Amazon ES) for a variety of mission-critical use cases, ranging from log aggregation to application monitoring and pricing optimization. In this session, the Expedia team reviews how they use Amazon ES and Kibana to analyze and visualize Docker startup logs, AWS CloudTrail data, and application metrics. They share best practices for architecting a scalable, secure log analytics solution using Amazon ES, so you can add new data sources almost effortlessly and get insights quickly.

Amazon Kinesis Analytics offers a built-in machine learning algorithm that you can use to easily detect anomalies in your VPC network traffic and improve security monitoring. Join us for an interactive discussion on how to stream your VPC flow Logs to Amazon Kinesis Streams and identify anomalies using Kinesis Analytics.

Agility is the cornerstone of the DevOps movement. Developers are working to continuously integrate and deploy (CI/CD) code to the cloud, to ensure applications are seamlessly updated and current. But what about secure? Security best practices and compliance are now the responsibility of everyone in the development lifecycle, and continuous security is a critical component of the ongoing deployment process. Discover how to incorporate security best practices into your current DevOps operations, gain visibility into compliance posture, and identify potential risks and threats in your AWS environment. We demonstrate how to leverage the CIS AWS Foundation Benchmarks within Sumo to trigger alerts from your AWS CloudTrail and Amazon CloudWatch log when risks or violations occur, such as unauthorized API calls, IAM policy changes, AWS Config configuration changes, and many more.   Session sponsored by Sumo Logic

MirrorWeb offers automated website and social media archiving services with full text search capability for all content. The UK government hired MirrorWeb to provide search services across 20 years of archived data from over 4,800 websites. In this session, MirrorWeb discusses the technology stack they built using Amazon Elasticsearch Service (Amazon ES) to search across the 333 million unique documents (over 120 TB) that they indexed within a 10-hour period. They discuss how they moved data from on-premises to Amazon S3 using AWS Snowball and then processed that data using Amazon EC2 Spot Instances, reducing costs by over 90%. They also talk about how they used AWS Lambda to ingest data into Amazon ES. Finally, they share best practices for building a large-scale document search architecture.

Thousands of services work in concert to deliver millions of hours of video streams to Netflix customers every day. These applications vary in size, function, and technology, but they all make use of the Netflix network to communicate. Understanding the interactions between these services is a daunting challenge both because of the sheer volume of traffic and the dynamic nature of deployments. In this session, we first discuss why Netflix chose Kinesis Streams to address these challenges at scale. We then dive deep into how Netflix uses Kinesis Streams to enrich network traffic logs and identify usage patterns in real time. Lastly, we cover how Netflix uses this system to build comprehensive dependency maps, increase network efficiency, and improve failure resiliency. From this session, youl learn how to build a real-time application monitoring system using network traffic logs and get real-time, actionable insights.

Petabyte scale archives of satellites, planes, and drones imagery continue to grow exponentially. They mostly exist as semi-structured data, but they are only valuable when accessed and processed by a wide range of products for both visualization and analysis. This session provides an overview of how ArcGIS indexes and structures data so that any part of it can be quickly accessed, processed, and analyzed by reading only the minimum amount of data needed for the task. In this session, we share best practices for structuring and compressing massive datasets in Amazon S3, so it can be analyzed efficiently. We also review a number of different image formats, including GeoTIFF (used for the Public Datasets on AWS program, Landsat on AWS), cloud optimized GeoTIFF, MRF, and CRF as well as different compression approaches to show the effect on processing performance. Finally, we provide examples of how this technology has been used to help image processing and analysis for the response to Hurricane Harvey.

This session, we focus on common use cases and design patterns for predictive analytics using Amazon EMR. We address accessing data from a data lake, extraction and preprocessing with Apache Spark, analytics and machine learning code development with notebooks (Jupyter, Zeppelin), and data visualization using Amazon QuickSight. We cover other operational topics, such as deployment patterns for ad hoc exploration and batch workloads using Spot and multi-user notebooks. The intended audience for this session includes technical users who are building statistical and data analytics models for the business using tools, such as Python, R, Spark, Presto, Amazon EMR, Notebooks.

In this session, we will teach you the technology behind Alexa Gadgets – a new category of connected products and developer tools that enable you to create your own Alexa-connected product or game skill that work with Echo Buttons. You will hear from the GM of Alexa Gadgets, as well as early Alexa Gadget developers, Musicplode Media (the makers of Beat the Intro) and Gemmy Industries (the makers of Big Mouth Billy Bass).

In this session, we'll teach you how to use the Alexa Voice Service (AVS) and its suite of development tools to bring your first Alexa-enabled product to market. You'll learn how commercial device manufacturers are getting to market faster using the new AVS Device SDK. To ensure your customers have the best voice experience, we'll teach you how to choose an Audio Front End and client-side hardware from a range of commercial-grade Development Kits. You'll walk out of this session with the knowledge required to design products with optimized Alexa-enabled voice experiences around your unique design requirements.

In this presentation, hear from John Rome, Arizona State University's Deputy CIO, and Jared Stein, Instructure's VP of Higher Ed Strategy, on how voice technology is bringing higher education to a new era. Come learn how institutions are adopting Alexa on campus and in their curriculum to serve students in new, innovative ways and how Instructure is rethinking the delivery of education for millions of customers through their Canvas skill for Alexa.