Storage Unpacked Podcast

Storage Unpacked 253 – Building Virtual SANs in the Cloud (Revisited)


Listen Later

In this episode, Chris chats with Abel Gordon, Chief System Architect at Lightbits Labs, discussing the challenges and benefits of building a virtual storage area network (SAN) on public cloud infrastructure. Lightbits originally developed the NVMe/TCP protocol and uses this feature to build virtual SANs using public cloud instances. This is a topic we first looked at in episode #210, so it’s good to get a practitioner’s experience.

Modern public cloud now features fast networking, low-latency NVMe and high-performance virtual and physical instances. Unfortunately, NVMe devices are ephemeral and any provisioned storage in the cloud is charged at full capacity. For users of on-premises SANs, the lack of thin provisioning may be an unwelcome surprise.

Why build a virtual SAN, other than to save storage costs? There’s a lot more involved, including delivering resiliency, scalability, targeted performance and capacity. Abel discusses the benefits, then goes on to enumerate the challenges involved when building on vendor-owned infrastructure. Finally, the discussion moves on to how Lightbits’ software is deployed and operated, including the managed application capability in Microsoft Azure.

For more information on Lightbits Labs, visit the company website at https://www.lightbitslabs.com/

As Abel, suggests you can contact him on LinkedIn or email him directly at [email protected].

Elapsed Time: 00:51:15

Timeline
  • 00:00:00 – Intros
  • 00:02:07 – Why build a virtual SAN in the public cloud
  • 00:04:30 – SANs balance out and fully exploit available performance resources
  • 00:06:36 – Public cloud charges for performance and capacity
  • 00:08:12 – On-premises SANs offered full flexibility to manage all metrics
  • 00:09:25 – Cloud autoscaling combined with software gives much more flexible storage
  • 00:13:27 – The on-demand nature of cloud works well for scaling SANs
  • 00:14:40 – New cloud features – NVMe, fast networking and NVMe/TCP have enabled solutions
  • 00:17:19 – What is NVMe/TCP?
  • 00:20:50 – What challenges are there in delivering a SAN on public cloud instances?
  • 00:24:03 – Cloud providers optimise for their system, not for your application
  • 00:25:02 – What operating system issues exist when building a virtual SAN?
  • 00:28:27 – Userspace operation requires a different programming strategy
  • 00:31:00 – NUMA awareness is essential, even in the public cloud
  • 00:32:46 – Each new instance type requires retesting and validation
  • 00:35:53 – What is the Lightbits solution and how is it deployed?
  • 00:37:00 – NVMe cloud drives are ephemeral
  • 00:41:35 – Snapshots work differently in the public cloud
  • 00:44:12 – Is Lightbits dedicated or HCI?
  • 00:46:32 – How is the solution consumed?
  • 00:47:29 – Azure offers management application capability
  • 00:50:36 – Wrap up
  • Related Podcasts & Blogs
    • Is the Public Cloud Becoming More Reliable?
    • Zesty Optimises AWS EC2 EBS Storage
    • Storage QoS In The Cloud
    • #97 – Building Storage Using NVMe/TCP with Kam Eshghi from Lightbits Labs
    • #121 – NVMe 1.4 Deep Dive Part II with J Metz
    • #210 – Building SANs in the Cloud
    • Copyright (c) 2016-2023 Unpacked Network. No reproduction or re-use without permission. Podcast episode #cv54.

      ...more
      View all episodesView all episodes
      Download on the App Store

      Storage Unpacked PodcastBy Storage Unpacked Podcast

      • 4.8
      • 4.8
      • 4.8
      • 4.8
      • 4.8

      4.8

      21 ratings


      More shows like Storage Unpacked Podcast

      View all
      Security Now (Audio) by TWiT

      Security Now (Audio)

      1,960 Listeners

      Marketplace by Marketplace

      Marketplace

      8,587 Listeners

      Risky Business by Patrick Gray

      Risky Business

      363 Listeners

      TechONTAPPodcast by NetApp

      TechONTAPPodcast

      33 Listeners

      The Ben Shapiro Show by The Daily Wire

      The Ben Shapiro Show

      151,428 Listeners

      Pivot by New York Magazine

      Pivot

      8,814 Listeners

      The Daily by The New York Times

      The Daily

      111,470 Listeners

      The Pure Report by Pure Storage

      The Pure Report

      33 Listeners

      Practical AI by Practical AI LLC

      Practical AI

      195 Listeners

      Hard Fork by The New York Times

      Hard Fork

      5,338 Listeners

      The Ringer F1 Show by The Ringer

      The Ringer F1 Show

      591 Listeners

      House of R by The Ringer

      House of R

      3,000 Listeners