Key note - Principles and challenges of model validation - John Helliwell
Si ligandum requires, circumspice. *
Crystallography and cryoEM are both, data-rich and knowledge-rich. Validation of molecular structure models thus takes place against two components: examination of direct experimental evidence, and assessment of plausibility in view of prior knowledge. This essentially Bayesian approach also allows to consider the scientific context – does the item to be validated bear any relevance to the claims made? Far from postmodern relativism, context-based validation allows to focus on parts of the model relevant for the scientific narrative or brought-forward hypothesis. Nonetheless, as obvious from the fact that every atom contributes to each reflection of a data set, every part of the model should be built and refined as accurately as possible, not in the least for the purpose of contributing to a valid structure data base forming the prior knowledge for computational modelling and machine learning.
Metrics used for validation of ligands are reasonably robust given good data and high ligand occupancies. Including solvent contributions has allowed ligand modelling at progressively lower fractional occupancies, which in turn requires increasing robustness of validation metrics. Valid probabilistic metrics require a clearly defined null hypothesis, which at very low ligand occupancies combined with partially ordered solvent density becomes nontrivial. If multiple models (ligands) yield similar probabilities when tested against a valid null hypothesis, the presumed presence of a specific ligand then is purely the result of prior knowledge, with no direct support by evidence - evidence of a ligand present is not the same as evidence of a particular ligand present.
Apologies to Christopher Wren