Productivity and reproducibility: conflicting pressures on scientists?
Dorothy V M Bishop (University of Oxford)
Recorded: Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig - 10 May 2021
N.B. There is considerable overlap between this 35 mins talk and a longer 55 min talk: "How current research practices are damaging science, and how open science offers some solutions" - • How current research practices are damagin...
Summary
I first introduce the idea of quasi-cumulative science: where we think we are building on foundations of prior work, only to find that the earlier work is not solid. There is no single cause for this, but rather an unfortunate combination of factors that, taken together, have a detrimental effect on science. I discuss five of these, together with possible solutions, namely:
1. Publication bias
2. Citation bias.
3. P-hacking
4. Low statistical power
5. Obsession of funders/journals with novelty.
Other resources:
https://www2.slideshare.net/deevybishop
http://deevybee.blogspot.com/2012/11/...
References
Bishop, D. (2019). Rein in the four horsemen of irreproducibility. Nature, 568(7753), 435–435. https://doi.org/10.1038/d41586-019-01...
Bishop, D. (2020). How scientists can stop fooling themselves over statistics. Nature, 584(7819), 9–9. https://doi.org/10.1038/d41586-020-02...
Border, R., Johnson, E. C., Evans, L. M., Smolen, A., Berley, N., Sullivan, P. F., & Keller, M. C. (2019). No support for historical candidate gene or candidate gene-by-interaction hypotheses for major depression across multiple large samples. American Journal of Psychiatry, 176(5), 376–387. https://doi.org/10.1176/appi.ajp.2018...
Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P., & Willmes, K. (2015). Registered Reports: Realigning incentives in scientific publishing. Cortex, 66, A1–A2. https://doi.org/10.1016/j.cortex.2015...
De Vries, Y. A., et al (2018). The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychological Medicine 48, 2453-2455. doi:10.1017/S0033291718001873
Greenberg, S. A. (2009). How citation distortions create unfounded authority: Analysis of a citation network. BMJ, 339, b2680. https://doi.org/10.1136/bmj.b2680
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82(1), 1–20. https://doi.org/10.1037/h0076157
Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., Coriat, A.-M., Foeger, N., & Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biology, 18(7), e3000737. https://doi.org/10.1371/journal.pbio....
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86....