As practical applications of AI are rolled out, they are increasingly being deployed on-premises at scale. We are wrapping up this season of Utilizing Tech with Solidigm focused on AI Data Infrastructure by discussing practical deployment considerations with Ariel Pisetzky, VP of Information Technology and Cyber at Taboola in a discussion with Jeniece Wnorowski and Stephen Foskett. Companies like Taboola are built on data and have been deploying AI-driven applications for years. Generative AI brings new capabilities but is part of a spectrum of solutions that leverage data to produce results for customers. As applications mature, many companies are looking to bring them back on-premises, and this trend will likely accelerate given the cost of AI infrastructure as-a-service offerings. Owned infrastructure can also deliver beyond expected lifespans, representing a potential windfall for businesses that can continue to use deprerciated hardware. This is especially true of large flash drives, which have proven much more reliable than initially predicted. Although it is tempting to buy the biggest, fastest infrastructure to extend the lifespan of equipment, Pisetzky recommends focusing on equipment that is flexible and can be re-purposed in other ways in the future. Server storage is unique in that it is easy to upgrade and replace it in place, even hot-swapping drives, and large lives have a very long lifespan.

Hosts:

Guest: Ariel Pisetzky, VP of Information Technology and Cyber, Taboola: https://www.linkedin.com/in/ariel-pisetzky/

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Modern AI infrastructure has exposed the importance of reliability and predictability of storage in addition to performance. This episode of Utilizing Tech, presented by Solidigm, features Kelley Osburn of Graid Technology discussing the challenges of maximizing performance and resiliency of storage for AI with Jeniece Wnorowski and Stephen Foskett. AI servers are optimized for machine learning processing, and Graid Technology SupremeRAID offloads processing to GPUs similarly to the way these massively-parallel processors offload ML processing. They also have a peer-to-peer DMA feature to direct the data directly to the processor rather than forcing all data to pass through a single processor or channel. There is a need for RAID software at many spots in the data pipeline, from ingestion and preparation to processing and consolidation, and each requires performance and availability. There are many applications that require maximum performance and capacity without impacting the host CPU, including military, medical research and diagnostics, and financial, in addition to AI processing.

Hosts:

Guest: Kelley Osburn, Senior Director at Graid Technology: https://www.linkedin.com/in/kelleyosburn/

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Many of the largest-scale data storage environments use Ceph, an open source storage system, and are now connecting this to AI. This episode of Utilizing Tech, sponsored by Solidigm, features Dan van der Ster, CTO of Clyso, discussing Ceph for AI Data with Jeniece Wnorowski and Stephen Foskett. Ceph began in research and education but today is widely used as well in finance, entertainment, and commerce. All of these use cases require massive scalability and extreme reliability despite using commodity storage components, but Ceph is increasingly able to deliver high performance as well. AI workloads require scalable metadata performance as well, which is an area that Ceph developers are making great strides. The software has also proved itself adaptable to advanced hardware, including today’s large NVMe SSDs. As data infrastructure development has expanded from academia to HPC to the cloud and now AI, it’s important to see how the community is embracing and improving the software that underpins today’s compute stack.

Hosts:

Guest: Dan van der Ster, CTO at CLYSO and Ceph Executive Council Member: https://www.linkedin.com/in/dan-vanderster/

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

As the volume of data supporting AI applications grows ever larger, it's critical to deliver scalable performance without overlooking power efficiency. This episode of Utilizing Tech, sponsored by Solidigm, brings Chris Gladwin, CEO and co-founder of Ocient, to talk about scalable and efficient data platforms for AI with Jeniece Wnorowski and Stephen Foskett. Ocient has developed a new data analytics stack focused on scalability with energy efficiency for ultra-large data analytics applications. At scale, applications need to incorporate trillions of data points, and it is not just desirable but necessary to enable this without losing sight of energy consumption. Ocient leverages flash storage to reduce power consumption and increase performance but also moves data processing closer to the storage to reduce power consumption further. This type of integrated storage and compute would not be possible without flash, and reflects the architecture of modern processors, which locate memory on-package with compute. Ocient is already popular in telco, e-commerce, and automotive, and the scale of data required by AI applications is similar, especially as concepts like retrieval-augmented generation are implemented. The conversation around datacenter, cloud, and AI energy usage is coming to the fore, and companies must address the environmental impact of everything we do.

Hosts:

Guest:

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Cutting-edge AI infrastructure needs all the performance it can get, but these environments must also be efficient and reliable. This episode of Utilizing Tech, brought to you by Solidigm, features Davide Villa of Xinnor discussing the value of modern software RAID and NVMe SSDs with Ace Stryker and Stephen Foskett. Xinnor xiRAID leverages the resources of the server, including the AVX instruction set found on modern CPUs, to combine NVMe SSDs, providing high performance and reliability inside the box. Modern servers have multiple internal drive slots, and all of these drives must be managed and protected in the event of failure. This is especially important in AI servers, since an ML training run can take weeks, amplifying the risk of failure. Software RAID can be used in many different implementations, with various file systems, including NFS and high-performance networks like InfiniBand. And it can be tuned to maximize performance for each workload. Xinnor can help customers to tune the software to maximize reliability of SSDs, especially with QLC flash, by adapting the chunk size and minimizing write amplification. Xinnor also produces a storage platform solution called xiSTORE that combines xiRAID with the Lustre FS clustered file system, which is already popular in HPC environments. Although many environments can benefit from a full-featured storage platform, others need a software RAID solution to combine NVMe SSDs for performance and reliability.

Hosts:

Davide Villa, Chief Revenue Officer at Xinnor: https://www.linkedin.com/in/davide-villa-b1256a2/

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Organizations seeking to build an infrastructure stack for AI training need to know how the data platform is going to perform. This episode of Utilizing Tech, presented by Solidigm, includes Curtis Anderson, Co-Chair of the Storage Working Group at MLCommons, discussing storage benchmarking with Ace Stryker and Stephen Foskett. MLCommons is an industry consortium seeking to improve AI solutions through joint engineering. The organization publishes the well-known MLPerf benchmark, which now includes practical metrics for storage solutions. The goal of MLPerf Storage is to answer the key question: Will a given data infrastructure support AI training of a given scale. The organization encourages storage vendors to run the benchmarks against their solutions to prove the suitability to support specific workloads. The AI industry is already shifting its focus from maximum scale and performance to more-balances infrastructure using alternative GPUs, accelerators, and even CPUs, and is increasingly concerned about price and environmental impact. The question of data preparation is also rising, and this generally uses a different CPU-focused solution. MLPerf Storage is focused on training today and will soon address data preparation, though this can be quite different for each data set. The next MLPerf Storage benchmark opens soon, and we encourage all data infrastructure companies to get involved and submit their own performance numbers.

Hosts:

Guest: Curtis Anderson, Co-Chair MLCommons Storage Working Group: https://www.linkedin.com/in/curtis-anderson-174aa/

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Model training seriously stresses data infrastructure, but preparing that data to be used is a much more difficult challenge. This episode of Utilizing Tech features Subramanian Kartik of VAST Data discussing the broad data pipeline with Jeniece Wnorowski of Solidigm and Stephen Foskett. The first step in building an AI model is collecting, organizing, tagging, and transforming data. Yet this data is spread around the organization in databases, data lakes, and unstructured repositories. The challenge of building a data pipeline is familiar to most businesses, since a similar process is required in analytics, business intelligence, observability, and simulation, but generative AI applications have an insatiable appetite for data. These applications also demand extreme levels of storage performance, and only flash SSDs can meet this demand. A side benefit is the improvements in power consumption and cooling versus hard disk drives, and this is especially true as massive SSDs come to market. Ultimately the success of generative AI will drive greater collection and processing of data on the inferencing side, perhaps at the edge, and this will drive AI data infrastructure further.

Hosts:

Guest:

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Analysts and press spend a lot of time talking about specs and performance numbers, so it's always a treat when we get to talk to people who are testing and using these products. This episode of Utilizing Tech is focused on AI Data Infrastructure and features Jordan Ranous from StorageReview and is co-hosted by Stephen Foskett and Ace Stryker from our sponsor, Solidigm. StorageReview has constricted an experimental environment focused on astrophotography as a way to demonstrate AI applications in challenging edge environments. Their setup included a ruggedized Dell server, NVIDIA GPU, and Solidigm SSDs. This is the same sort of setup found at edge compute environments in retail, manufacturing, and remote use cases. StorageReview benchmarks storage devices by profiling real-world applications and building representative infrastructure to test. When it comes to GPUs, the goal is to keep these expensive processors operating at maximum capacity through optimal network and storage throughput.

Hosts:

Guest:

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #Sponsored, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

Great AI needs excellent data infrastructure, in terms of capacity, performance, and efficiency. This episode of Utilizing Tech serves as a preview of season 7, brought to you by Solidigm, and features co-hosts Jeniece Wnorowski and Ace Stryker along with Stephen Foskett. Solidigm's partners are discovering just how important it is to optimize every element of the Ai infrastructure stack. With ever-larger AI datacenters being built, efficient storage can make a big difference, from power and cooling to physical density to performance. As we will hear throughout season 7, different AI environments will need specialized data infrastructure, from the edge to the cloud. With retrieval-augmented generation (RAG) emerging as a new trend in AI, it makes high-performance even more important at run-time.

Hosts:

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingTech, #AIDataInfrastructure, #AI, @SFoskett, @TechFieldDay, @UtilizingTech, @Solidigm,

We knew that 2024 would be the year of AI right from the start, but this season of the podcast has seen incredible development and change. This final episode of Utilizing Tech Season 6 features hosts Frederic Van Haren, Allyson Klein, and Stephen Foskett discussing the current state of AI infrastructure half-way through 2024. In addition to AI Field Day, we experienced NVIDIA GTC and numerous product introductions over the last few months. It's truly an ecosystem play now, with every company showing how well they can partner to build AI infrastructure. At the same time, a few superusers of AI are responsible for the basic models, including Google, Amazon, and Microsoft, and of course OpenAI and the other dedicated generative AI firms. The key to bringing this to the enterprise market is transfer learning, which will see a few base models tuned and trained for specific use cases. This season saw a range of guests discussing storage, data platforms, connectivity, and application development, and every one is focused on delivering practical AI solutions in the enterprise.

Hosts:

Follow Utilizing Tech

Tech Field Day

Tags: #UtilizingAI, #UtilizingTech, #YearofAI, @UtilizingTech, @TechFieldDay, @TheFuturumGroup, @SFoskett, @FredericVHaren, @TechAllyson,