SmartNICs Today

SmartNICs Today

By Scott SchweitzerTechnology
Download on the App Store

SmartNICs Today episodes

  • Optimizing IDS Performance Using FPGA SmartNICs
    Description: In this episode of SmartNICs Today, host Scott Schweitzer discusses high-speed IDS performance with Lukas Sismis and Jan Korenek of DYNANIC. They explore how FPGA-based SmartNICs and hardware-software co-design accelerate Suricata and overcome CPU scaling bottlenecks at 100G/400G+ rates. Table of Contents
    20 min
  • Ultra-low Latency Inference at the Network Edge with Xelera

    05-07-26: Today, I caught up with an old friend, Ron Renwick, and the team from Xelera Felix Winterstein, the CEO, and Andrea Suardi, Head of Acceleration, to discuss their new Silva offering and how they are bringing AI inference to the network edge. Recently, the Xelera team completed the STAC-ML™ Markets (Inference) benchmark audit on a stack that includes a STAC-ML™ Pack for Xelera Silva with AMD Alveo™ V80 on an HPE Proliant DL385 Gen10 Plus v2 server. For context, here are some highlights from this report:

    • For the small (GBT_A) and medium (GBT_B) models, 99th percentile latencies were <= 1.95µs for all Numbers of Model Instances (NMI) tested, with worst-case instance throughput > 560K inferences per second at the highest NMIs tested 
    • For the large (GBT_C) model, the 99th percentile latency was 2.88µs, with worst-case instance throughput of 379K inferences per second  
    • The maximum latency was <= 12.3µs across all models and NMI tested

      • Table of Contents

      • 00:00 Hello & Welcome
      • 00:09 Ron Background
      • 01:04 Felix Background
      • 01:40 Andrea Background
      • 02:30 Xelera Origin Story
      • 06:00 FPGAs are very sticky once you start working with them
      • 07:33 What is Xelera, what do they do?
      • 07:55 It’s not about FPGAs
      • 08:30 It’s network acceleration for the data center
      • 08:40 We are seeing a massive buildout in data center capacity
      • 09:05 Number one KPI (Key Performance Indicator) is compute capacity
      • 10:31 All that processing is starting to reach its limits
      • 10:50 In cybersecurity and networking, we see this lack of computing becoming painful
      • 11:10 The thing every infrastructure buyer or architect needs to consider
      • 11:24 What are the products that Xelera offers, and who do they address?
      • 11:45 First is a classic DPU softNIC
      • 12:05 Strong footprint in Cyber Security vertical
      • 12:26 Second product is AI Acceleration
      • 12:48 We bring Andrea back in to talk about his STAC Research Event Talk
      • 13:40 When you attend STAC events, you’re in a room with really deeply technical people
      • 14:00 All these people are here to solve one problem: optimize their execution stack
      • 14:57 Tail latency spikes are discussed, and the impact on trading
      • 15:17 Ron drills into this tail latency issue a bit more
      • 16:12 The cost of latency varies from firm to firm
      • 16:45 Too slow, and you are a price taker, not a price maker
      • 17:13 It’s important to win more than 50.001% of the time
      • 17:24 What we’re selling today is the ability to trade both faster and smarter
      • 19:40 Gradient boosting 50 us running in the CPU, went down to 5us running on the FPGA
      • 20:10 This became Silvia, an agent who runs fast.
      • 20:30 What other markets can Silva be applied to, for example, security
      • 22:00 Silva can look at each packet at the edge, for Ransomware or DDoS, pattern detection
      • 23:00 Silva is the engine, the model can vary depending on the use case, time
      • 23:30 What numbers can you provide? 
      • 24:00 Two main Silva modes, the first is Offload, inference via API with one 2M nodes
      • 25:10 Now 30us to run gradient boosting on CPU, if you use Silva, it’s 1us on FPGA
      • 25:40 LSTN with a 1M parameter model is 1ms. If you offload to an FPGA, you get 3us
      • 26:10 There is a cost using the PCIe bus, 500-700ns depending on packet size
      • 26:40 The second Silva mode is Inline mode, and this runs entirely on the FPGA
      • 27:10 Are there specific use cases or environments driving deployment environment
      • 28:15 The lowest latency requires optimize the stack; cloud won’t work
      • 29:00 We rewrote the CPU kernel to tune the cache and have it work with Gradient Boosting
      • 30:00 You can go from 500us for a standard framework 15us if you rewrite the kernel
      • 30:20 Silvia makes using an FPGA easier 
      • 30:30 SmartNICs can remove the 30% CPU overhead for processing packets
      • 31:20 Offload allows you to get your hands on the data before any packet touches the CPU
      • 32:25 In HFT, Silva can save a tremendous amount of latency by avoiding the PCIe bus
      • 33:10 When you are talking to potential customers about Silva, who is expressing interest?
      • 33:30 All three, C-level, architects, and engineers
      • 33:40 The C-Suite is often the starting point.
      • 34:35 From a Silva perspective, who do you talk with initially?
      • 34:50 The quant team initially approaches us from trading side, sometimes, infrastructure
      • 35:30 Where do you see Xelera going over the next few years?
      • 36:00 Cybersecurity, network security
      • 36:10 We are announcing on the podcast today support for 400 Gbps
      • 36:40 There is an extension of the Silvia product into network security
      • 37:40 On Silvia, we have just launched the CPU-only version for not only HFT, but also others
      • 38:00 Embedded AI use case running on the FPGA, like Gradient boosting tree
      • 39:00 Moving to LLM in general, the market for Inference is moving
      • 39:15 Cost of executing LLMs is exploding
      • 39:40 What is the one thing people should remember about Xelera?
      • 39:35 Key points is that there are plenty of SmartNIC/DPU use cases, gaps in architectures
      • 40:00 SmartNIC and DPU use cases are diverse, focusing on specifics has proven successful
      • 40:30 One thing to remember about Xelera is that we are offering a full solution
      • 40:40 You don’t have to become an FPGA expert or developer to use this technology 
      • 41:00 Closing statements
        • 42 min
      • Inference at the Network Edge with Xelera
        Description: Today, I caught up with an old friend, Ron Renwick, and the team from Xelera, Felix Winterstein, the CEO, and Andrea Suardi, Head of Acceleration, to discuss their new Silva offering and how they are bringing AI inference to the network edge. Recently, the Xelera team completed the STAC-ML™ Markets (Inference) benchmark audit on a stack that includes a STAC-ML™ Pack for Xelera Silva with AMD Alveo™ V80 on an HPE Proliant DL385 Gen10 Plus v2 server. For context, here are some highlights from this report: Table of Contents
        42 min
      • Alex Stein of Liquid Market Solutions Talks about Network Attached Compute


        We spent 30 minutes talking with Alex Stein, the CEO of Liquid Market Solutions, about SmartNICs, Network Attached Compute, AI, White Rabbit, and a whole host of other topics. See the Table of Contents below for more details.

        Table of Contents
        00:00 Hello & Welcome.
        00:14 Introduction to Alex Stein and Liquid Market Solutions (LMS).
        02:00 History, four startups, one DE Shaw-backed, and the path to 2Sigma.
        02:40 2Sigma and building Alpha Capture.
        05:20 Meeting Seth at Morgan Stanley.
        06:00 Seth is pushing them towards producing an FPGA-based solution.
        06:40 When Scott and Alex may have first met in 2009 while Alex was at 2Sigma.
        08:00 How does LMS fit into the larger SmartNIC market?
        10:20 A TOE, no session set up, as well as the other protocols.
        11:25 Strength of partnerships, while remaining focused on their core differentiation.
        12:40 Working with BittWare as a board design partner.
        13:30 They have 19 patents around their core technology; they don’t license their IP.
        14:30 Packaged their technology to directly address customer problems.
        15:10 Network Attached Compute (NAC) from NIC to NAC.
        17:10 Taking UberNIC beyond AI. Timestamping, White Rabbit, streaming, telco…
        19:00 Working with CERN to synchronize time with a resolution of 20 picoseconds.
        19:40 AI as a force multiplier.
        22:10 News sentiment portal that leveraged AI, and the value of AI moving forward.
        22:40 Challenges of SmartNICs in Financial, the complexity of FPGA programming.
        23:40 Activation Energy and what is required to move to a programmable platform.
        25:40 Customers can add their own logic on top of UberNIC.
        26:50 Power, heat, and the challenges they pose.
        28:40 Future timeline, roadmap, and the use of ARM.
        29:15 Alex hinted at a possible future ARM-based offering.
        29:25 UALink and silicon photonics for chip-to-chip.
        30:40 Looking out five years.
        32:40 Composable I/O, we’ve heard it before, it will be interesting where it goes.
        33:00 CXL and how that fits moving forward. 
        34:00 Wrap up and thank you for listening.

        35 min
      • Alex Stein of Liquid Market Solutions Talks about Network Attached Compute
        We spent over 30 minutes talking with Alex Stein, the CEO of Liquid Market Solutions, talking about SmartNICs, Network Attached Compute, AI, White Rabbit, and a whole host of issues. See the Table of Contents below for more details. Table of Contents 00:00 Hello & Welcome.00:14 Introduction to Alex Stein and Liquid Market Solutions (LMS).02:00 History, four startups, one DE Shaw-backed, and the path to 2Sigma.02:40 2Sigma and building Alpha Capture.05:20 Meeting Seth at Morgan Stanley.06:00 Seth is pushing them towards producing an FPGA-based solution.06:40 When Scott and Alex may have first met in 2009 while Alex was at 2Sigma.08:00 How […]
        35 min
      • 3-20-26 DDoS Defense with the DYNANIC Team
        03-19-26 We chatted with Jan Korenek, Ph.D., CSO and Co-Founder of DYNANIC, and Lukáš Kekely, Ph.D., CTO of DYNANIC, about DDoS attacks and how to defend against them using various NIC technologies. We talked about Cloudflare and their use of generic ASIC-based NICs in their scrubbing centers, as well as ASIC-based SmartNICs with many ARM cores and FPGA-based SmartNICs.
        25 min
      • DDoS Defense with the Team from DYNANIC

        03-19-26 We chatted with Jan Korenek, Ph.D., CSO and Co-Founder of DYNANIC, and Lukáš Kekely, Ph.D., CTO of DYNANIC, about DDoS attacks and how to defend against them using various NIC technologies. We talked about Cloudflare and their use of generic ASIC-based NICs in their scrubbing centers, as well as ASIC-based SmartNICs with many ARM cores and FPGA-based SmartNICs.

        25 min

      About SmartNICs Today

      From the publisher's feed

      Discussions with industry experts in the field of SmartNICs and DPUs.