Relative goodness-of-fit tests for Models with Latent Variables
I describe a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable.
Given the premise that “all models are wrong,” the goal of the test is to determine whether one model significantly outperforms the other in respect of a reference data sample.
The test generalises earlier kernel Stein discrepancy (KSD) tests to the case of latent variable models, a much more general class than the fully observed models treated previously. The new test, with a properly calibrated threshold, has a well-controlled type-I error. In the case of models with low-dimensional latent structure and high-dimensional observations, our test significantly outperforms the relative maximum mean discrepancy test, which is based on samples from the models, and does not exploit the latent structure.
We illustrate the test on probabilistic topic models of arXiv articles.
This recording was made for the ICSA 22 conference: https://www.icsa.org/icsa-2022-china-...
It was also presented here:
https://ensai.fr/en/event/gofcp-2022/
and here:
https://ims.nus.edu.sg/events/steins-...
The talk describes this paper:
A Kernel Stein Test for Comparing Latent Variable Models
Heishiro Kanagawa, Wittawat Jitkrittum, Lester Mackey, Kenji Fukumizu, Arthur Gretton
https://arxiv.org/abs/1907.00586
which extends the earlier paper:
A Kernel Test of Goodness of Fit
Kacper Chwialkowski, Heiko Strathmann, Arthur Gretton
https://arxiv.org/abs/1602.02964