Testing Self-Adaptive Software with Probabilistic Guarantees on Performanc... (Video, ESEC/FSE 2020)
"Testing Self-Adaptive Software with Probabilistic Guarantees on Performance Metrics (Video, ESEC/FSE 2020)
Claudio Mandrioli and Martina Maggio
(Lund University, Sweden; Saarland University, Germany / Lund University, Sweden)
Abstract: This paper discusses the problem of testing the performance of the adaptation layer in a self-adaptive system. The problem is notoriously hard, due to the high degree of uncertainty and variability inherent in an adaptive software application. In particular, providing any type of formal guarantee for this problem is extremely difficult. In this paper we propose the use of a rigorous probabilistic approach to overcome the mentioned difficulties and provide probabilistic guarantees on the software performance. We describe the set up needed for the application of a probabilistic approach. We then discuss the traditional tools from statistics that could be applied to analyse the results, highlighting their limitations and motivating why they are unsuitable for the given problem. We propose the use of a novel tool – the scenario theory – to overcome said limitations. We conclude the paper with a thorough empirical evaluation of the proposed approach, using two adaptive software applications: the Tele-Assistance Service and the Self-Adaptive Video Encoder. With the first, we empirically expose the trade-off between data collection and confidence in the testing campaign. With the second, we demonstrate how to compare different adaptation strategies.
Article: https://doi.org/10.1145/3368089.3409685
Supplementary archive: https://doi.org/10.5281/zenodo.3896795 (Badges: Artifacts Available, Artifacts Evaluated — Reusable, Artifacts Evaluated — Functional)
Supplementary web page: https://github.com/ManCla/ESEC-FSE-2020
ORCID: https://orcid.org/0000-0002-7013-1191, https://orcid.org/0000-0002-1143-1127
Submitted to the conference by Claudio Mandrioli on 2020-10-29
Video Tags: Testing, Self-Adaptive Software, Autonomous Systems, fse20main-p171-p, DOI: 10.1145/3368089.3409685, DOI: 10.5281/zenodo.3896795, ORCID: 0000-0002-7013-1191, ORCID: 0000-0002-1143-1127, Artifacts Available, Artifacts Evaluated — Reusable, Artifacts Evaluated — Functional
Presentation at the ESEC/FSE 2020 conference, November 8–13, 2020, https://2020.esec-fse.org/
Sponsored by ACM SIGSOFT, https://www.sigsoft.org/
Twitter: / fseconf
Reddit: / esecfse "
ISSTA 2021: Panels Celebrating 30 editions of ISSTA (statements from panel in timeband 1)
ISSTA 2021: Panels Celebrating 30 editions of ISSTA (statements from panel in timeband 3)