Storage Developer Conference

PCIe devices continue to get faster and more powerful. At this point building systems where all DMA traffic must pass through the system memory of the host CPU is becoming problematic. For this reason, there has been considerable work done on both hardware standardization and software frameworks to enable Peer-2-Peer (P2P) DMAs between PCIe End Points (EPs). In this talk we will present an update on the P2PDMA ecosystem and include performance results gathered from systems that have been designed to utilize P2PDMA. We will show how such systems can outperform their conventional counterparts and lead to lower-cost and lower-power designs that still obtain optimal performance. We will also discuss the future for P2PDMA and what work is ongoing to extend the framework.

Learning Objectives: 1) Learn new types of data flow within PCIe based storage systems; 2) NVMe CMBs and how they can be used in interesting ways; 3) How software frameworks like SPDK and Linux are being updated to enable these new data flows; 4) The progress of software enablement for new types of data movement in PCIe/NVMe based storage systems.

Building on the concepts presented in the Introduction to Swordfish sessions, this session will go into more detail on the new Swordfish Scalable Storage Management API specification, including details of the Swordfish Class of Service concepts, structure and usage. It will also cover details of constructing file vs block systems. The deep-dive will also provide a look at the schema, and the ability to support both RESTful and OData clients.

The SNIA’s Scalable Storage Management Technical Work Group (SSM TWG) has developed an open industry standard specification that provides a unified approach for the management of storage systems and data services. This specification extends the DMTF’s Redfish specification using RESTful methods and JSON formatting. This session will present an overview of the specification. This session will also describe the positioning of the specification developed by the SSM TWG vs SMI-S as well as the base Redfish specification.

Many software infrastructures used in modern data centers and storage appliances rely on key value abstraction to access underlying storage and memory systems. However, the existing devices only provide a block interface via storage protocols. As a result, an S/W translation layer such as memcached, LevelDB, RocksDB, extent layer, etc is used to bridge this gap. However, the high overhead of such software layers impact on performance, reliability, resource utilization, and scalability of infrastructure. In this special session of key value SSD, we introduce a comprehensive introduction to key value SSD, covering the concept, device, software ecosystem, system proof point, performance, and standard activity.

In this tutorial, we will introduce the audience to the lunatic fringe of extreme high-performance computing and its storage systems. The most difficult challenge in HPC storage is caused by millions (soon to be billions) of simultaneously writing threads. Although cloud providers handle workloads of comparable, or larger, aggregate scale, the HPC challenge is unique because the concurrent writers are modifying shared data.

We will begin with a brief history of HPC computing covering the previous few decades, bringing us into the petaflop era which started in 2009. Then we will discuss the unique computational science in HPC so that the audience can understand the unavoidability of its unique storage challenges. We will then move into a discussion of archival storage and the hardware and software technologies needed to store today’s exabytes of data forever. From archive we will move into the parallel file systems of today and will end the lecture portion of the tutorial with a discussion of anticipated HPC storage systems of tomorrow. Of particular focus will be namespaces handling concurrent modifications to billions of entries as this is what we believe will be the largest challenge in the exascale era.

Caches in modern storage systems lack the ability to adapt automatically and optimize for dynamic workload mixes. Despite the potential for huge improvements in cost, performance, and predictability, such adaptability is extremely challenging, due to inherently complex, non-linear, and workload-dependent behavior. Even when manually-tunable controls are provided to support dynamic cache sizing, partitioning, and parameter tuning, administrators simply don’t have the information required to make good decisions.

In this talk, we will present an overview of the significant opportunity for self-optimizing caches by examining several examples from production systems. We will review recently-published research in this area, including robust, general methods for efficient cache modeling. Optimizations that leverage these models promise to improve the performance of most workloads and cache policies automatically.

The Storage Performance Development Kit (SPDK) is an open source set of tools and libraries for writing high performance, scalable, user-mode storage applications. It achieves high performance by moving all of the necessary drivers into userspace and operating in a polled mode instead of relying on interrupts. The Blobstore is a relative newcomer to SPDK and provides local, persistent, power-fail safe block allocator designed to replace filesystem usage in many popular databases. Most importantly, the Blobstore has been designed for the properties of flash and next-generation media from the start and directly leverages NVMe features. The team has already ported a popular embedded key/value database, RocksDB, to use the Blobstore which demonstrated a significant improvement for database queries under common workloads. In this session we will explore the basics of the Blobstore and review some of the latest exciting performance data!

The Windows Server 2016 release contains support for Persistent Memory including the introduction of DAX volumes. This presentation will discuss the PM improvemenetns in Windows since this release. We will also review Windows support of the NVML library.

Open Channel describes a new interface to Solid State Drives (SSDs) which promises to increase usage of SSDs’ raw bandwidth from 40% to 95%, increase user-visible flash capacity from 50%-70% to 99%, increase I/O bandwidth by 3x and reduce per-GB hardware cost by 50%. Despite many proposals and implementations proving these benefits, industry has seen limited adoption and no standards body has integrated the concept. One of the largest hurdles to adoption is that the proposed changes permeate every layer in the storage stack, from device firmware to application. To reap the benefits, we need not only an end goal, but a pragmatic approach to introducing these changes to one or two layers at a time.

We present relevant information about host and drive architecture, the expected use cases for Open Channel, and a general-purpose, maintainable end target for Open Channel. The final architecture refactors Flash Translation Layer into Log-Management, handled in the host, and NAND Management, handled in the drive. This consolidates repeated software algorithms, and retains media-specific functionality in the drive.

The results of this design set up the storage community to innovate in traditionally independent areas. With a new interface in place, host-side software developers can apply a wide variety of software and technologies to further optimize their storage logs, and firmware and drive designers can focus on improvements in media management and density improvements.

PCIe devices such as GPGPUs, FPGA accerlators, RDMA enabled NICs and NVM Express SSDs are placing a huge strain on the IO subsystem of the CPUs they are connected too. With the emergence of NVMe over Fabrics and heterogeneous compute there is often a desire to move large stream of data between these endpoints without CPU intervention.

In this presentation we give an overview of the latest work we have done to enable this PCIe Peer-2-Peer (P2P) communication in the Linux kernel. We also cover a detailed performance comparison between normal data flows and those that avail of P2P. We show how the P2P data-flows offload the IO subsystem of the CPU and lead to better throughput, latency and Quality of Service.