Session 3A: Simulation Insights on Power Analysis with Binary Responses: From SNR Methods to skprJMP
Dr. Tyler Morgan-Wall is a Research Staff Member at the Institute for Defense Analyses, and is the developer of the software library skpr: a package developed at IDA for optimal design generation and power evaluation in R. He is also the author of several other R packages for data visualization, mapping, and cartography. He has a PhD in Physics from Johns Hopkins University and lives in Silver Spring, MD.
Logistic regression is a commonly-used method for analyzing tests with probabilistic responses in the test community, yet calculating power for these tests has historically been challenging. This difficulty prompted the development of methods based on signal-to-noise ratio (SNR) approximations over the last decade, tailored to address the intricacies of logistic regression's binary outcomes and complex probability distributions. Originally conceived as a solution to the limitations of then-available statistical software, these approximations provided a necessary, albeit imperfect, means of power analysis. However, advancements and improvements in statistical software and computational power have reduced the need for such approximate methods. Our research presents a detailed simulation study that compares SNR-based power estimates with those derived from exact Monte Carlo simulations, highlighting the inadequacies of SNR approximations. To address these shortcomings, we will discuss improvements in the open-source R package "skpr" as well as present "skprJMP," a new plug-in that offers more accurate and reliable power calculations for logistic regression analyses for organizations that prefer to work in JMP. Our presentation will outline the challenges initially encountered in calculating power for logistic regression, discuss the findings from our simulation study, and demonstrate the capabilities and benefits "skpr" and "skprJMP" provide to an analyst.
Session Materials: https://dataworks.testscience.org/wp-...