Performance and Data Management at Scale for the AI Data Center - Jean Thomas Acquaviva, DDN
The unfolding of the AI market had powerful implications in terms of the organization and architecture of the data centers. Regarding data management, the first impact was related to a push for performance. Here, taking the existing HPC parallel file systems as a groundwork, with additional features related to data management, checkpointing, and unified namespaces, DDN met the extreme performance requirements. Data solutions delivering TB/s of read and write bandwidth is now something DDN is routinely installing around the world. Winning this challenge has made DDN a partner of choice for NVIDIA, as illustrated by their recently announced EOS system.
The second impact on data management is not related to extreme performance to feed GPUs but more about the ability to structure the data lake and interoperate across sites and edge devices. Implementing the concept of a data outer ring, with a data lakehouse and powerful metadata engine, is the next brick in the DDN vision for an AI-oriented data center.
In this talk, we will describe the added values of this new system and how it unified the notion of data governance and performance.