A Year of Observability Across the Toolstack: Breakthroughs, Challenges, Improveme... Marcus Granado
A Year of Observability Across the Toolstack: Breakthroughs, Challenges, Improvements - Marcus Granado, XenServer, Cloud Software Group
We have been using Distributed Tracing as a form of Observability in the XAPI toolstack for over a year. DT increased our ability to introspect the real-time behaviour of the toolstack code and make comparable measurements across development, testing, commissioning and production. We are starting to go beyond the control plane towards the Storage Management layer using a novel observer wrapper, and designing our instrumentation of the I/O data path. This talk will present the challenges we faced implementing DT, breakthroughs in code and design we had to achieve to go over these challenges, and the improvements and opportunities we detected. We'll conclude by suggesting design patterns that the Xen Ecosystem could use to speed up adoption of DT across components that were initially designed without support for DT.