Storage Developer Conference

The Skyhook Data Management project (skyhookdm.com) at UC Santa Cruz brings together two very successful open source projects, the Ceph object storage system, and the Apache Arrow cross-language development platform for in-memory analytics. It introduces a new class of storage objects to provide an Apache Arrow-native data management and storage system for columnar data, inheriting the scale-out and availability properties of Ceph. SkyhookDM enables single-process applications to push relational processing methods into Ceph and thereby scale out across all nodes of a Ceph cluster in terms of both IO and CPU. To highlight the benefits, we will present performance for various physical layouts and query workloads over example tables of 1 billion rows, as we scale out the number of nodes in a Ceph cluster. In this talk, we first describe how we partition Apache Arrow columnar data into Ceph objects. We consider both horizontal and vertical partitioning (rows vs. columns) of tables. In contrast to objects storing opaque byte streams where the meaning of the data must be interpreted by a higher level application, Apache Arrow data can be partitioned along semantic boundaries such as columns and rows so that relational operators like selection and projection can be performed in objects storing semantically complete data partitions. Next we introduce our SkyhookDM extensions that utilize Ceph’s “CLS” plugin infrastructure to execute our methods directly on objects, within the local OSD context. These access methods use the Apache Arrow access library to operate on Arrow data within the context of an individual object and implement relational processing methods, physical data layout changes, and localized indexing of data. Relational processing methods include SELECT, PROJECT, ORDER BY, and GROUP BY with partial aggregations (e.g., local min, max, sum, count, etc.). Physical data layout operations currently supported include transforming objects between row and column layouts, which we plan to extend to co-group columns on objects. Localized indexing is performed as a new object write method and supports index lookups that are beneficial to both point queries and range queries. SkyhookDM is accessed via a user-level C++ library on top of librados. The SkyhookDM library comes with Python bindings and is used in a PostgreSQL Foreign Data Wrapper. The source code is available at github.com/uccross/skyhookdm-ceph-cls under LGPLv2. SkyhookDM is an open source incubator project at the Center for Research in Open Source Software at UC Santa Cruz (cross.ucsc.edu). This work was in part supported by the National Science Foundation under Cooperative Agreement OAC-1836650.

Learning Objectives: Using Ceph object classes to scale out relational data storage and access; Formatting, storing, and processing relational data directly in Ceph objects using librados; Indexing and physical data layout of relational data within Ceph objects.

DMTF’s Redfish® is a standard designed to deliver simple and secure management for converged, hybrid IT and the Software Defined Data Center (SDDC). This presentation will provide an overview of DMTF’s Redfish standard. It will also provide an overview HPE’s implementation of Redfish, focusing on their storage implementation and needs. HPE will provide insights into the benefits and challenges of the Redfish Storage model, including areas where functionality added to SNIA™ Swordfish is of interest for future releases.

Learning Objectives: Introduce the DMTF Redfish AP; Provide an overview of HPE’s shipping Redfish Storage implementation; Understand where SNIA Swordfish is of interest for HPE’s server-attach storage implementations

Serial Attached SCSI (SAS) is the only storage interface that embraces both high performance and high reliability, as well as providing native compatibility with low-cost per gigabyte SATA drives. This capability allows SAS to span a variety of storage environments, including tiered storage solutions. Large-scale data infrastructures utilize tens of thousands of HDDs and SSDs. Hyperscale companies need to be able to carefully manage them from a global perspective in a cost-effective way. During this presentation, the speaker will review the benefits of tiered storage and how the latest features standardized in 24G SAS storage interface technology is helping enterprises store and move data across a range of storage media with different characteristics, such as performance, cost and capacity.

Learning Objectives: The latest features of SAS,The benefits of tiered storage,How developers and users can take advantage of 24G SAS features to minimize storage costs in large scale deployments.

The SNIA has a new standard that enables SSDs to have an Ethernet interface, The Native NVMe-oF Drive specification defines pin outs for common SSD connectors. This enables these drives to plug into common platforms such as eBOFs (Ethernet Bunches of Flash). This talk will also discuss the latest Management standards for NVMe-oF drives. Developers will learn about how to program and use these new types of drives.

Learning Objectives: The developer will understand how Ethernet drives are enabled; The developer will understand how to manage Ethernet drives; Then developer will understand the future of these drives.

End to End (E2E) Data Placement or intelligent placement of data onto media requires coordination between Applications, File System, and Zoned Block devices (ZBDs). If done correctly, E2E Data Placement with ZBDs will significantly reduce storage costs and improve application performance. The talk will walk through state of the art database systems and define their data placement characteristics with the associated storage cost. Next, we discuss how E2E data placement can use the concept of a file to determine data associativity and efficiently store the file as zones on ZBDs. We will cover crucial ZBD metrics and present examples of how applications and file systems can be modified to be ZBD friendly. Methods to estimate the gains in throughput and storage cost reduction using E2E data placement and Zone Block devices will also be shown. The attendees should leave the talk understanding how E2E data placement changes the role of Zoned Block Devices from storing LBAs to storing files. And how, by strategically mapping files, and it's data, as zones, one gain device capacity and reduces storage costs while improving both throughput and latency of your storage solution.

Learning Objectives: The attendees should leave the talk understanding how E2E data placement changes the role of Zoned Block Devices,And how, by strategically mapping files, and it's data, as zones, one gain device capacity,and reduces storage costs while improving both throughput and latency of your storage solution.

High performance storage applications running on Intel® Xeon® processors actively utilize I/O capabilities and I/O accelerating features of platform by interfacing with NVMe devices. Such I/O-intensive applications may suffer from performance issues, which in a big picture can be categorized into three domains: I/O device bound – performance is limited by device capabilities core bound – performance is limited by algorithmic or microarchitectural code issues uncore bound – performance is limited by non-optimal interactions between devices and CPU. This talk focuses on the latter case. In Intel architectures the term “core” covers execution units and private caches, and all the rest of the processor is referred as “uncore”, which includes on-die interconnect, shared cache, cross-socket links, integrated memory and I/O controllers, etc. Activities happening on IO path in uncore cannot be monitored with traditional core-centric analyses, but there are pitfalls that require uncore-centric view. Intel servers provide such view by incorporating thousands of uncore performance monitoring events that can be collected in performance monitoring units (PMUs) associated with uncore IP blocks. However, using raw counters for performance analysis requires deep knowledge of hardware and appears incredibly challenging. In this talk we will discuss platform-level activities induced by I/O traffic on Intel® Xeon® Scalable processors and summarize practices for best performance of storage applications. We will overview telemetry points staying on the IO traffic path and eventually present developing uncore-specific performance analysis methodology, that reveals platform-level inefficiencies, including poor utilization of Intel® Data Direct I/O Technology (Intel® DDIO).

Learning Objectives: Uncore-centric performance analysis methodology for I/O-intensive applications running on Intel server architectures,HW operations induced by PCIe traffic and HW-level observability for them; Practices to gain best IO performance on Intel server architectures.

Compute Express Link™ (CXL™) is an industry-supported cache-coherent interconnect for processors, memory expansion, and accelerators. Datacenter architectures are evolving to support the workloads of emerging applications in Artificial Intelligence and Machine Learning that require a high-speed, low latency, cache-coherent interconnect. The CXL specification delivers breakthrough performance, while leveraging PCI Express® technology to support rapid adoption. It addresses resource sharing and cache coherency to improve performance, reduce software stack complexity, and lower overall systems costs, allowing users to focus on target workloads. Attendees will learn how CXL technology maintains a unified, coherent memory space between the CPU (host processor) and CXL devices allowing the device to expose its memory as coherent in the platform and allowing the device to directly cache coherent memory. This allows both the CPU and device to share resources for higher performance and reduced software stack complexity. In CXL, the CPU host is primarily responsible for coherency management abstracting peer device caches and CPU caches. The resulting simplified coherence model reduces the device cost, complexity and overhead traditionally associated with coherency across an I/O link.

Learning Objectives: Learn how CXL supports dynamic multiplexing between a rich set of protocols that includes I/O (CLX.io, based on PCIe®), caching (CXL.cache), and memory (CXL.mem) semantics.,Understand how CXL maintains a unified, coherent memory space between the CPU and any memory on the attached CXL device,Gain insight into the features introduced in the CXL specification

Improvements to the SMB3.1.1 client on Linux have continued at a rapid pace over the past year. These allow Linux to better access Samba server, as well as the Cloud (Azure), NAS appliances, Windows systems, Macs and an ever increasing number of embedded Linux devices including those using the new smb3 kernel server Linux (ksmbd). The SMB3.1.1 client for Linux (cifs.ko) continues to be one of the most actively developed file systems on Linux and these improvements have made it possible to run additional workloads remotely. The exciting recent addition of the new kernel server also allows more rapid development and testing of optimizations for Linux. Over the past year, performance has dramatically improved with features like multichannel (allowing better parallelization of i/o and also utilization of multiple network devices simultaneously), with much faster encryption and signing, with better use of compounding and improved support for RDMA. Security has improved and alternative security models are now possible with the addition of modefromsid and idsfromsid and also better integration with Kerberos security tooling. New features have been added include the ability to swap over SMB3 and boot over SMB3. Quality continues to improve with more work on 'xfstests' and test automation - tooling (cifs-utils) continue to be extended to make use of SMB3.1.1 mounts easier. This presentation will describe and demonstrate the progress that has been made over the past year in the Linux kernel client in accessing servers using the SMB3.1.1 family of protocols. In addition recommendations on common configuration choices, and troubleshooting techniques will be discussed.

Learning Objectives: What new features are now possible when accessing servers from Linux?,What new tools have been added to make it easier to use SMB3.1.1 mounts from Linux?,What new features are nearing completion that you should you expect to see in the near future?,How can I configure the security settings I need to use SMB3.1.1 for my workload?,How can I configure the client for optimal performance for my workload?

The NVMe Key Value (NVMe-KV) Command Set has been standardized as one of the new I/O Command Sets that NVMe Supports. Additionally, SNIA has standardized a Key Value API that works with the NVMe Key Value allows access to data on a storage device using a key rather than a block address. The NVMe-KV Command Set provides the key to store a corresponding value on non-volatile media, then retrieves that value from the media by specifying the corresponding key. Key Value allows users to access key-value data without the costly and time-consuming overhead of additional translation tables between keys and logical blocks. This presentation will discuss the benefits of Key Value storage, present the major features of the NVMe-KV Command Set and how it interacts with the NVMe standards, and present open source work that is available to take advantage of Key Value storage.

Learning Objectives: Present the standardization of SNIA KV API,Present the standardization of NVMe Key Value Command Set,Present the benefits of Key Valeu in computational storage,Present open source work on Key Value Storage.

The rapid improvement of overall $/Gbyte has driven the high performance All-Flash Array to be increasingly adopted in both enterprises and cloud datacenters. Besides the raw NAND density scaling with continued semiconductor process improvement, data reduction techniques have and will play a crucial role in further reducing the overall effective cost of All-Flash Array. One of the key data reduction techniques is compression. Compression can be performed both inline and offline. In fact, the best All-Flash Arrays often do both: fast inline compression at a lower compression ratio, and slower, opportunistic offline deep compression at significantly higher compression ratio. However, with the rapid growth of both capacity and sustained throughput due to the consolidation of workloads on a shared All-Flash Array platform, a growing percentage of the data never gets the opportunity for deep compression. There is a deceptively simple solution: Inline Deep Compression with the additional benefits of reduced flash wear and networking load. The challenge, however, is the prohibitive amount of CPU cycles required. Deep compression often requires 10x or more CPU cycles than typical fast inline compression. Even worse, the challenge will continue to grow: CPU performance scaling has slowed down significantly (breakdown of Dennard scaling), but the performance of All-Flash Array has been growing at a far greater pace. In this talk, I will explain how we can meet this challenge with a domain-specific hardware design. The hardware platform is a FPGA-based PCIe card that is programmable. It can sustain 5+Gbyte/s of deep compression throughput with low latency for even small data block sizes (TByte/s BW and less than 10ns of latency) and the almost unlimited parallelism available on a modern mid-range FPGA device. The hardware compression algorithm is trained with a vast amount of data available to our systems. Our benchmarks show it can match or outperform some of the best software compressors available in the market without taxing the CPU.

Learning Objectives: Hardware Architecture for Inline Deep Compression,Design of Hardware Deep Compression Engine,Inline and offline compression of All-Flash Array.