Systems @Scale 2019 - Accordion Better Memory Organization for LSM Key Value Stores
Eshcar Hillel, Senior Research Scientist, Verizon Media
https://code.fb.com/core-data/systems...
Log-structured merge (LSM) stores have emerged as the technology of choice for building scalable write-intensive key-value storage systems. Though inherent to the LSM design, frequent compactions are a major pain point because they slow down data store operations, primarily writes, and increase disk wear. Another performance bottleneck in today’s state-of-the-art LSM stores, in particular ones that use managed languages like Java, is the fragmented memory layout of their dynamic memory store. In this talk, Eshcar shows that these pain points may be mitigated via better organization of the memory store. She also presents Accordion — an algorithm that addresses these problems by reapplying the LSM design principles to memory management. Accordion is implemented in the production code of Apache HBase, where it was extensively evaluated. Eshcar demonstrates Accordion’s double-digit performance gains versus the baseline HBase implementation and discuss some unexpected lessons learned in the process.