AWS re:Invent 2016

In this popular session, you will learn about the latest features and use cases for Amazon EBS, including best practices, an overview of newly introduced features, and brand-new re:Invent announcements. In particular we will cover the expanded portoflio of volume types, including provisioned IOPS, cold storage, and throughput-optimized. This session will help database admins and application architects understand how to blend performance and cost with applicaitns for big data analytics, data warehousing, and transactional and NoSQL databases.

Not just for archiving or compliance use cases, Amazon Glacier accommodates customers simply looking to replace their on-premises long term storage with a cost efficient, durable, cloud option, from which they can easily and quickly access their data when they need to. This session will introduce newly launched features for Amazon Glacier, review the current service feature set, and share the global data center shut down and storage strategy for Sony DADC New Media Solutions (NMS). NMS is Sony’s digital servicing division providing global digital distribution, linear playout and white label OTT/Commerce solutions for clients such as BBC Worldwide, NBCUniversal, Sony Playstation, and Funimation Entertainment.

Come learn about new and existing Amazon S3 features that can help you better protect your data, save on cost, and improve usability, security, and performance. We will cover a wide variety of Amazon S3 features and go into depth on several newer features with configuration and code snippets, so you can apply the learnings on your object storage workloads.

These days, EVERY workload is considered critical by someone in the organization. As a result, SLAs are shrinking. IT is challenged to meet these SLAs, but there isn’t enough budget to provide services like disaster recovery (DR) using traditional methods and infrastructure. The good news is that public cloud platforms, like AWS, are becoming the de facto infrastructure choice for DR. However, workload portability solutions that simplify cross-platform or cloud recovery are required to meet most RTO & RPO SLAs in the cloud. AWS provides the infrastructure we need to bring DR to tier 2 and tier 3 workloads that have never been able to afford it before. Now, we need orchestration and automation to make it scalable and reliable.

You have billions of events in your fact table, all of it waiting to be visualized. Enter Tableau… but wait: how can you ensure scalability and speed with your data in Amazon S3, Spark, Amazon Redshift, or Presto? In this talk, you’ll hear how Albert Wong and Srikanth Devidi at Netflix use Tableau on top of their big data stack. Albert and Srikanth also show how you can get the most out of a massive dataset using Tableau, and help guide you through the problems you may encounter along the way.

Prezi has over 75 million registered users generating 900 GB of new data each day. They store XML describing over 260 million public presentations that have been viewed over 2 billion times. This created a scale problem that on-premises storage couldn't solve. In this session, you'll learn how Prezi leveraged the power and flexibility of Amazon S3 to turn their storage problem into an analytics opportunity and scale storage to meet the demands of their business without overspending. Prezi provides insight on how a small team of engineers accepted the challenge and succeeded using AWS with managed Hadoop (Amazon EMR) and Amazon Redshift. You'll see how Prezi runs and improves its data platform in a self-service data culture with this small team, without drowning in maintenance and support. Additionally, Prezi shows how the infrastructure team architected their services for storing and serving crucial data, user-generated and otherwise, using S3 and Amazon CloudFront—including pitfalls, best practices, cost considerations, and learnings along the way.

FINRA partnered with AWS product teams to leverage Amazon EMR and Amazon S3 extensively to build an advanced analytics solution. In this session, you'll hear how FINRA implemented a data lake on S3 to provide a single source for their big data analytics platform. FINRA ingests 75 billion records each day of stock market transactions, with an AWS storage footprint of 20 petabytes across S3 and Amazon Glacier. To deal with this workload, FINRA has architected a platform that separates storage from compute to manage capacity for each independently, leading to improved performance and cost effectiveness. You'll also learn how FINRA was able to leverage Hbase on Amazon EMR to achieve significant benefits over running Hbase on a fixed capacity cluster. FINRA was able to implement a system that seamlessly scales in response to data growth and can scale quickly in response to user traffic. By working with multiple clusters, FINRA can now isolate ETL and user query workloads and has achieved rapid, built-in disaster recovery capability by leveraging data storage on S3 to run from multiple AZs and across regions.

Startups around the world use AWS services to access the power of the cloud to grow faster and more cost effectively. In this session, Smartsheet talks about how they were able to cost-effectively build their prototype for scale and avoid replatforming at different points in the adoption curve, and Quantcast discusses how they are running a high-performance analytics solution on AWS. They provide several tips and tricks for S3, and show how they removed a traditional MySQL data store from a distributed-image hosting application so that the only required data store is S3. They also show how to avoid common, cumbersome database practices by working with the eventually consistent nature of S3 objects and the fact that objects and directories share the same namespace.

The companies Videology and Zendesk both had the same problem—how to rearchitect their big data processing platforms to scale to meet growing demand, while at the same time improving performance, availability, and cost structure? Videology provides a converged advertising solution that is screen-agnostic, ensuring unduplicated reach with the right frequency cadence to achieve guaranteed results. To achieve this, their big data platform ingests, processes, and analyzes a variety of logs. In this talk, Videology discusses how they migrated our platform to use Cloudera on a mix of m4 and r3 instances using the Amazon EBS Streaming Optimized HDD (st1) volume type. Zendesk provides a cloud-based customer support platform that allows quicker and easier interaction between businesses and customers. To deliver this experience, Zendesk runs a large Elasticsearch, Logstash, Kibana stack. The talk also discusses how Zendesk rearchitected their deployment to use m4s and also leverage the EBS Streaming Optimized HDD (st1) volume type. Tips for success will be shared throughout.

AWS Step Functions is a new, fully-managed service that makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Step Functions is a reliable way to connect and step through a series of AWS Lambda functions so that you can build and run multi-step applications in a matter of minutes. This session shows how to use AWS Step Functions to create, run, and debug cloud state machines to execute parallel, sequential, and branching steps of your application, with automatic catch and retry conditions. We share how customers are using AWS Step Functions to reliably scale multi-step applications such as order processing, report generation, and data transformation–all without managing any infrastructure.