How Empirical Research Supports Tool Development: A Retrospective Analysis and new Horizons
Shared ESEM and SSBSE Keynote by Prof. Massimiliano di Penta (University of Sannio, Italy)
Abstract:
Empirical research provides two-fold support to the development of approaches and tools aimed at supporting software engineers. On the one hand, empirical studies help to understand a phenomenon or a context of interest. On the other hand, studies compare approaches and evaluate how software engineers benefit from them. Over the past decades, there has been a tangible evolution in how empirical evaluation is conducted in software engineering. This is due to multiple reasons. On the one hand, the research community has matured a lot thanks also to guidelines developed by several researchers. On the other hand, the large availability of data and artifacts, mainly from the open-source, has made it possible to conduct larger evaluations, and in some cases to reach study participants. In this keynote, I will first overview how empirical research has been used over the past decades to evaluate tools, and how this is changing over the years. I will also emphasize the importance of combining quantitative and qualitative evaluations, and how sometimes depth turns out to be more useful than just breadth. I will also emphasize how research is not a straightforward path, and negative results are often an essential component for future advances. Last, but not least, I will emphasize how the role of empirical evaluation is changing with the pervasiveness of artificial intelligence methods in software engineering research.