AI Inference Storage Powered by CSAL on Bluefield-3 DPUs
The rapid development of Large Language Models (LLMs) has posed an urgent demand for efficient Key-Value cache (KV cache) management to optimize AI inference performance in multi-turn conversations and long-context processing. This White Paper explores the integration of the Cloud Storage Acceleration Layer1 (CSAL) with the BlueField-3 Data Processing Unit 2 (DPU), addressing the storage challenges of KV cache in high-concurrency AI workloads through a flexible storage architecture.
This architecture combines high-capacity SSDs and high-performance SSDs to deliver system level configuration flexibility. Pairing CSAL with one or more DPU, it dynamically allocates data between high-capacity SSD, high-performance SSD, or cache-layer storage resources based on the specific needs of different AI workload stages such as data preparation, training, inference, and retrieval-augmented generation (RAG), thereby significantly improving throughput and substantially reducing Time-to-First-Token (TTFT).
About the Authors
Wayne Gao is a Principal Engineer and Solution Storage Architect at Solidigm. He has worked on Solidigm’s Cloud Storage Acceleration Layer (CSAL) from pathfinding to commercial release. Wayne has over 20 years of storage developer experience, has four U.S. patent filings/grants, and is a published EuroSys paper author.
Bo Li serves as a senior storage solutions architect at Solidigm. With over two decades of experience in system design and development across multiple organizations, he specializes in optimizing the performance of networked and storage solutions. In recent years, Bo has concentrated his efforts on advancing the industry-wide adoption of non-volatile storage technologies.
Mariusz Barczak is a Principal Engineer at Solidigm. He has over 13 years of experience finding innovations in storage software and storage solutions. His particular expertise is caching solutions, software defined storage, virtualization, and storage analytics. Mariusz holds numerous patents and is active in the open-source community. He is currently focused on leading the Solidigm team for Cloud Storage Acceleration Layer (CSAL) which delivers mixed media solutions combining Solidigm SLC SSDs with other storage components, such as Solidigm QLC SSDs, to deliver efficient and durable storage.
Sarika Mehta is a Senior Storage Solutions Architect at Solidigm, bringing over 16 years of experience from her tenure at Intel’s storage division and Solidigm. Her focus is to work closely with Solidigm customers and partners to optimize their storage solutions for cost and performance. She is responsible for tuning and optimizing Solidigm’s SSDs for various storage use cases in a variety of storage deployments that range from direct-attached storage to tiered and non-tiered disaggregated storage solutions. She has diverse storage background in validation, performance benchmarking, pathfinding, technical marketing, and solutions architecture.
Scott Werntz is a Solutions Architect at Solidigm. He brings over 30 years of industry experience in data center design and cloud computing to his role. With the advent of AI, IoT, virtualized workloads, and software defined storage, Scott has broadened his expertise into these emerging technologies to help clients understand how to best approach their changing storage solution needs. Scott holds numerous industry certifications along with his hands-on data center experience.
Kapil Karkra is a Sr. Principal Engineer at Solidigm responsible for software and solutions pathfinding for next-generation storage solutions supporting AI infrastructure. His work focuses on evolving Cloud Storage Acceleration Layer (CSAL), a host-based FTL with RAID and Caching, bringing technologies such as Mixed Media (MM) and Flexible Data Placement (FDP) to market, and defining turnkey reference architectures that integrate hardware and software to accelerate adoption of high-density NAND SSDs (QLC, PLC, and HLC) for AI and cloud workloads. Kapil holds a bachelor's degree in electrical engineering from the National Institute of Technology (NIT), India, and an MBA from Arizona State University.
