Vaultaire: a data vault for system metrics, backed onto Ceph
Andrew Cowie
Before you can do serious analytics you need to have serious data. You can't go averaging it away just to save space; you need the raw data.
Vaultaire is a distributed metrics store. It is fault tolerant, performs well, is space efficient, and — since it stores data directly in Ceph — scales horizontally as your cluster grows.
Most systems crumble when faced with the thousands of metrics per second typically being gathered in a production environment. Ceph is good at absorbing lots of writes, and is especially good at handling randomly distributed reads.
You may have experienced Ceph via it's S3-compatible HTTP gateway or via the RBD block device service. You may not know these are built atop a simple library that handles talking to the storage cluster and locating your data, librados, and that's how Vaultaire stores its time-series data.
Using librados directly opens some pretty cool possibilities. It's powerful, but low-level, and as you'd expect there are gotchas. We'll talk about building a librados-based application, lessons learned relying on Ceph in production, and the challenges of building predictive modelling tools on top of a metrics store that doesn't throw data away.