Solidigm Redefines AI Storage at AI Infrastructure Field Day

Density Without Compromise

Section 1: Storage density meets cluster-scale performance

AI Infrastructure Field Day 5 opened with a reality check. Allyn Malventano pulled up storage diagrams from two years ago. KV caching did not even appear on those slides. In fact, that tier did not exist in the conversation then. Now it is central to AI inference performance. Things change fast in this space. Solidigm is thrilled to show what we bring to this new tier. But enough on the backstory. Let's get into the numbers.

Solidigm covers the full stack of the AI storage pyramid. HBM and DRAM face tight supply and rising prices. The G3.5 context tier and G4 shared storage tier are where capacity matters. The Solidigm D5-P5336 anchors that high-capacity tier. The D7-PS1010 delivers flagship performance for local and context storage.

Items discussed in this section

• AI storage flows across training, fine-tuning, and inference
• The G1 through G4 memory hierarchy plus the G3.5 context tier
• KV cache math: 13.1 GB per user request at 42,000 tokens (Llama 3 70B)
• Why context reuse eliminates the recompute tax

Storage is no longer just persistence. It becomes memory when vector indexes and KV cache offload to SSDs.

Section 2: The 24 PB cluster — TCP baseline results

First up, the cluster-scale exercise with MinIO. The setup was straightforward. 8x8 configuration with eight client systems hitting eight storage servers. Each storage node packed with 24x Solidigm D5-P5336 122TB drives. Total raw capacity: 24 petabytes (for those counting, that is about 3PB per chassis).

MinIO Warp emulated GPU storage workloads. Client-to-storage traffic ran over single 400 Gigabit Ethernet NICs per client. Storage nodes had dual 400GbE for both client-facing and inter-node traffic.

Items discussed in this section

• 8-node cluster delivering ~24PB raw capacity
• 24x Solidigm D5-P5336 122TB QLC SSDs per storage node (~3PB per chassis)
• Solidigm D7-PS1010 TLC drives for load generation (client systems)
• MinIO Warp benchmark with 256 MiB objects
• Peak GET: 268 GiB/s at 8 clients
• Peak PUT: 120 GiB/s at 7 to 8 clients
• Zero errors across all 16 test runs

The baseline performance scaled almost linearly to 8 nodes. First-byte latency stayed in the 3 ms to 5 ms median range. Over 250 gigabytes per second (GB/sec) over TCP (Transmission control Protocol) is extreme given TCP's inherent overhead. Good stuff. These numbers are worth highlighting. You would expect TCP to bog down well before hitting these numbers. Don't be alarmed by the scale — this is what happens when you throw enough parallelism at the problem.

This cluster design is the building block of the MinIO ExaPOD reference architecture. Same node configuration scales from petabytes to exabytes.

Section 3: The 3x performance journey — Stack tuning

The initial numbers were good. The tuned numbers were better. Roughly 3x improvement over stock configuration.

Phase 1 established the TCP baseline with default Linux and MinIO settings. Phase 2 removed scheduling jitter, packet overhead, and congestion bottlenecks through OS and fabric tuning. Phase 3 co-tuned CPU, NIC, TCP, switch fabric, and MinIO layout for maximum throughput. The catch? Each phase added tuning complexity.

Items discussed in this section

• Three-phase tuning methodology
• SMT disabled, NIC interrupts pinned
• MinIO erasure set tuning: 24 erasure sets, 8 drives each, EC:4
• Switch and NIC buffer adjustments based on cable lengths
• CPU cost per GiB/s tracked across phases

The tuning work matters because it sets the comparison point for what comes next. You cannot measure RDMA (Remote Direct Memory Access) gains without a clean TCP baseline first.

Section 4: Beyond TCP — Dual-pathing and RDMA

Solidigm is exploring further performance enhancements. Two paths stand out.

First, dual-pathing to double initiator bandwidth. This becomes especially relevant for future integration with compute nodes such as the NVIDIA B200 or B300. 

Second, RDMA to bypass CPU overhead during data transfers. RDMA enables direct memory-to-memory communication between network adapters. Zero-copy, kernel bypass, microsecond latency. TCP overhead falls off hard at these scales. Expect more on this front soon.

Items discussed in this section

• TCP overhead becomes the bottleneck before NVMe
• RDMA benefits: zero-copy, kernel bypass, microsecond latency
• Dual-pathing + RDMA combination for maximum throughput
• Tuning complexity increases with each layer

The current exercise simulated GPU memory with DRAM. RDMA would still reduce CPU bottlenecks and enable higher throughput. Dual-pathing and RDMA together add tuning complexity. The performance payoff makes it worth pursuing.

MinIO MemKV: The G3.5 tier in action

The presentation moved to MinIO MemKV after the TCP baseline. This is the G3.5 concept in practice.

KV cache frequently exceeds HBM capacity on GPUs. In agentic workloads, context memory constantly expands. When HBM fills, GPUs evict and recompute the KV cache. That wastes cycles during pre-fill and increases Time to First Token (TTFT).

Items discussed in this section

• MinIO MemKV as distributed shared context memory
• Direct NVMe access, microsecond latency
• No file system or kernel overhead
• 43x more concurrent requests per GPU node
• 97 GB/s aggregated throughput
• Scales to 16,000 concurrent requests per second in superpods

MemKV bypasses traditional file systems and kernel overhead by leveraging direct NVMe access. It is engineered as an extension of memory, without durability features unnecessary for transient KV cache data.

The speakers called this a software problem best addressed through optimized software, not complex hardware like CXL. MemKV serves large KV cache blocks (2MB to 64MB tensors) directly from NVMe, avoiding file system metadata overhead.

Cost impact: over $2 million per year saved for a single H200 node in public cloud. Better GPU utilization drives the savings.

Solidigm and Metrum: Storage as AI memory

Section 1: RAG retrieval

The final section covered the Solidigm collaboration with Metrum. The evaluation tested SSDs as AI memory for RAG and KV-cache operations.

Metrum ingests vast amounts of video, generates detailed metadata, and creates a searchable database for AI inference. The evaluation compared HNSW (memory-based vector database) with DiskANN (storage-optimized) using Solidigm D7-PS1010 SSDs with real-world video pipeline data, not synthetic benchmarks.

Items discussed in this section

• Metrum AI white paper: "Breaking Memory Barriers in Video Analytics"
• DiskANN vs HNSW at 1M, 10M, 100M vectors
• 65% DRAM reduction: 331GB down to 116GB at 100M
• 27.4x faster time to first token: 39.26s to 1.43s
• 14% higher query throughput at 1,024 threads
• 99%+ recall maintained up to 100M vectors

HNSW showed a slight edge at lower concurrencies with smaller datasets. DiskANN cleaned house as concurrency increased, especially with real-world video pipeline data. Note that as we scale to 100M vectors, the DiskANN curve pulls away from HNSW. That is where the storage-optimized design pays off.

The ability to move substantial datasets from expensive DRAM to economical SSDs offers a powerful advantage, offering significant cost savings without sacrificing performance. DiskANN trounces HNSW at scale.

Section 2: KV-cache offload — Eliminating redundant compute

Beyond vector database lookups, SSDs play a critical role in KV-cache offload. By caching recurrent queries on Solidigm drives, the system drastically improved time to first token and eliminated redundant GPU recomputation.

A 180,000-token document pushed GPU memory to 95% utilization. Without cache reuse, every new question pays the full 39-second prefill cost. With SSD-backed KV offload, revisited documents hit the cache in about 1.4 seconds: No recomputation penalty.

Items discussed in this section

• KV-cache offload for repeated long-context inference
• 27.4x faster context switch for cache hits
• GPU cycles freed for other tasks
• Applies to any large-scale AI system with persistent context

This efficiency matters when serving hundreds or thousands of concurrent users. Faster, more responsive AI interactions. Valuable GPU compute cycles freed for other tasks.

Section 3: How it all integrates

The presentation closed by mapping the AI data pipeline to the infrastructure tiers that support it.

Raw data ingest and data preparation feed training and fine-tuning. Inference sits at the end, with KV offload and archive completing the loop. The infrastructure tiers run from G1 (GPU HBM) through G2 (host DRAM), G3 (local SSD), G3.5 (context memory storage), and G4 (shared storage).

Items discussed in this section

• AI data pipeline: ingest, prep, training, inference, archive
• G1 through G4 + G3.5 tier mapping
• D7-PS1010 for local and context performance
• D5-P5336 for shared capacity tier
• Telemetry and lab validation in Solidigm AI Central Lab

The product placement is straightforward. The Solidigm D7-PS1010 handles local boot, checkpointing, and the G3.5 context tier where low latency matters. The D5-P5336 anchors the G4 shared storage tier where capacity and cost per TB dominate the decision. No surprises there.

Wrap-up

The event wrapped with delegate discussions that dug into deployment details, cooling requirements, and the operational complexity of RDMA vs TCP paths. The presentations live on through Tech Field Day replays.