Storage Developer Conference

One of the most common benchmarks in the storage industry is 4KiB random read I/O per second. Over the years, the industry first saw the publication of 1M I/Ops on a single box, then 1M I/Ops on a single thread (by SPDK). More recently, there have been publications outlining 10M I/Ops on a single box using high performance NVMe devices and more than 100 CPU cores.

This talk will present a benchmark of SPDK performing more than 10 million random 4KiB read operations per second from a single thread to 20 NVMe devices, a large advance compared to the state of the art of the industry. SPDK has developed a number of novel techniques to reach this level of performance, which will be outlined in detail here. These techniques include polling, advanced MMIO doorbell batching strategies, PCIe and DDIO considerations, careful management of the CPU cache, and the use of non-temporal CPU instructions. This will be a low level talk with real examples of eliminating data dependent loads, profiling last level cache misses, pre-fetching, and more. Additionally, there remains a number of techniques that have not yet been employed that warrant future research. These techniques often push devices outside of their original intended operating mode, while remaining within the bounds of the specification, and so often require collaboration between NVMe controller and device designers, the NVMe specification body, and software developers such as the SPDK team.

Learning Objectives: 1) Optimal use of NVMe devices; 2) Optimal use of PCIe and MMIO in a storage stack; 3) Leveraging advanced x86-64 CPU instructions and making best use of the CPU cache.

The applications using NVMe, SAS, SATA, USB based storage devices find a new use and one of them is mining for open source cryptocurrency such as Burst Coin. Using low power or solar power HDD’s, SSD and most importantly NVMe technology can improve turnaround latency and build blocks on a faster scale. Utilization of security protocols allows anonymization as well as protection of the users and vendors. Burst coin has an extensive developer’s community and can run on the cloud, has dApps, its own ATM and more. More importantly, Burst is based on Proof of Capacity protocol and utilizes storage drives, arrays and enables users to build the mesh net of miners and secure blockchain protocol. Using NVMe devices we can accelerate transactions. We will show how using performance analytics tools we can create predictions on building blockchain blocks and provide insights into date usage efficiency. Additional value benefits are saving energy costs, address new markets and create adoption in the larger markets. The usage of storage devices and blockchain will enable HW secure banking transactions (via smart contracts) and much more.

Learning Objectives: 1) Learn how Proof of Capacity works with storage devices; 2) Find new applications for Storage applications; 3) Understand Data Science perception with Blockchain.

A growing trend in the market is capacity of data. This data growth is creating challenges within modern storage infrastructures and a new way to think of data is needed. The SNIA Computational Storage TWG was formed in October of 2018 to address this opportunity for the Storage industry to use innovative technologies that bring computational capabilities closer to or within the storage device. The goal of the TWG is to develop an architecture and set of definitions that allow for common communication about the problem set as well as a standardized interface between the Computational Storage device and host or peer devices. Ultimately the TWG will drive standardization of the necessary Computational Storage interfaces across the industry, contribute to and drive the development of software necessary to enable the usages, and promote the education of the industry regarding Computational Storage.

This session will provide an overview of Computational Storage, the focus areas of the TWG, and the opportunities for engagement with the rest of the industry in this space.

Learning Objectives: 1) Overview of the Computational Storage TWG goals; 2) Walk through the current architectural paradigms defined in the TWG; 3) Give the industry a common language to speak regarding computational storage.

Last year at SDC we reviewed the integration of crypto which made use of DPDK’s existing variety of drivers to usher in the capability. This year we are expanding our use of DPDK with the addition of a compression! This talk will outline the overall architecture of the compression module and explain in detail how we are managing the layout of the device and leveraging the Persistent Memory Development Kit to store metadata in super-fast persistent memory.

Learning Objectives: 1) Understand new SPDK compression feature; 2) Understand the value of SPDK in general; 3) Learn about the SPDK Community.

This talks explains the exciting new features in the Linux NVMe driver and software target in the last two years, as well as the relevant block layer changes to support these features.

Learning Objectives: 1) Learn about new Linux features; 2) Learn about new NVMe features; 3) Have fun!

LTO (Linear Tape Open) is an industry standard format for tape drives and media. To begin this talk we will give a brief overview of LTO tape: how data is recorded and accessed on tape, and some characteristics that make it very different from earlier forms of computer tape storage. We will then discuss the current state of LTO tape, including “feeds and speeds”, backwards/forwards drive and media compatibility, and the LTO Consortium’s roadmap for the future. Finally, we’ll discuss the Linear Tape File System (LTFS): what it is, how it works, and what benefits it provides. We’ll end with a detailed description of the latest important feature in LTFS version 2.5, Incremental Indexes, including a discussion of the impetus for Incremental Indexes, the benefits they provide, and an explanation of how they work. This talk is suitable for newcomers to tape storage as well as those who are already taking advantage of LTO tape and LTFS.

Learning Objectives: 1) Understand the operation and current state of LTO tape storage, and its roadmap for the future; 2) Understand the use and benefits of the Linear Tape File System (LTFS); 3) Learn how adding Incremental Indexes to LTFS improves performance and storage efficiency while preserving backwards compatibility.

Introducing pmemkv, an open-source local key/value store for persistent memory based on PMDK. Written in C/C++, pmemkv provides optimized language bindings for Java, JavaScript, and Ruby. Pmemkv includes multiple storage engines that are tailored for different use-cases. Fast, flexible and bulletproof, pmemkv is an easy way to modify applications to use persistent memory.

Learning Objectives: 1) Learn about a local/embedded key-value data store optimized for persistent memory; 2) Learn how cloud applications can easily manage key/value data on persistent platforms; 3) Code samples that demonstrate the ease of use of different language bindings.

RocksDB, a high performance key-value database developed by Facebook, has proven effective in using the high data speeds made possible by Solid State Drives (SSDs). By leveraging the NVMe standard, Eideticom’s NoLoad® presents FPGA computational storage processors as NVMe namespaces to the operating system and enables efficient data transfer between the NoLoad® Computational Storage Processors (CSPs), host memory and other NVMe/PCIe devices in the system. Presenting Computational Storage Processors as NVMe namespaces has the significant benefit of minimal software effort to integrate computational resources.

In this presentation we use Eideticom’s NoLoad® to speed up RocksDB. Compared to software compaction running on a Dell R7425 PowerEdge server, our NoLoad®, running on Xilinx’s Avleo U280, resulted in 6x improvement in database transactions and 2.5x reduction is CPU usage while reducing worst case latency by 2.7x.

Learning Objectives: 1) Computational storage with NVMe; 2) Presenting computational storage processors as NVMe namespaces; 3) Accelerating database access with NVMe computational storage processors.

Since the first release of NVMe 1.0 in 2011, the NVMe family of specifications continue to expand to support current and future storage markets, increasing the amount of new features and functionality. With that natural, organic growth, however, comes additional complexity.

In order to refocus on simplicity and ease-of-development, the NVM Express group has undertaken a massive effort to refactor the specification. The upcoming refactored specification - NVMe 2.0 - integrates the scalable and flexible NVMe over Fabrics architecture within the NVMe base specification, meeting the needs of platform designers, device vendors and developers.

But how can developers optimally design their products using the new NVMe 2.0 specification?

This session will provide attendees with the following insights:

• An overview of the existing specification structure, its logic and limitations

• Highlights on how developers use the current specification before refactoring

• Information showing how the refactored specification enables companies to architect their products with better awareness of future areas of innovation

• Details on how new features and functionalities will be included in the refactored specification

• Descriptions of how developers can leverage the refactored NVMe 2.0 specification to simply and efficiently bring new products to market

• Examination of the current projects and how to contribute

Learning Objectives: 1) Overview of the current NVMe specification structure; 2) Introduction to NVMe 2.0: the refactored specification enables companies and developers to simply and efficiently bring new products to market; 3) The new features and functionalities that will be included in NVMe 2.0 and how to get involved in current projects.

Over the years, SSD QoS has become more important to a variety of storage market segments. Traditional latency reporting methods do not always accurately depict QoS behaviors. This is problematic when attempting to understand what events lead to a specific QoS level and how to mitigate latency events that lead to levels of QoS. Defining correct statistical techniques for large populations of latencies deepens our understanding of what drives levels of QoS. Advanced statistical techniques, such a machine learning and utilizing AI, allows for deeper understanding of what drives QoS and how to correctly manage large quantities of latencies. New visualization techniques enhance capabilities to understand latency behavior and define critical scenarios that drive latency.

Learning Objectives: 1) Identify shortcomings of current QoS reporting; 2) Generate more reliable QoS values; 3) Techniques to broaden understanding of groups of latencies; 4) Identification of critical transitions in latency; 5) Identify inaccuracies that inhibit understanding QoS.