E.4 Further resources
Incentives
More information on Registered Reports:
Assessing the effectiveness of Registered Reports:
Chambers, C. D., Tzavella, L. (2020). Registered Reports: Past, Present and Future. https://doi.org/10.31222/osf.io/43298
Hardwicke, T. E., Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793-796. https://doi.org/10.1038/s41562-018-0444-y
Workflow of Registered Replication Reports (example AMPPS):
Brian Nosek on the importance of replication:
Statistics
Researcher degrees of freedom and QRP’s
Two papers on the reasons behind false discoveries:
Simmons, J. P., Nelson, L. D., Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366. https://doi.org/10.1177/0956797611417632
Ioannidis, J. P. A. (2005) Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
More on the \(\alpha\)-debate:
Morey, R. D. (2018, Jan 1). Redefining statistical significance: the statistical arguments [blog post]. Retrieved from https://medium.com/@richarddmorey/redefining-statistical-significance-the-statistical-arguments-ae9007bc1f91
de Ruiter, J. (2019). Redefine or justify? Comments on the alpha debate. Psychonomic Bulletin & Review, 26, 430-433. https://doi.org/10.3758/s13423-018-1523-9
Overview of methods to adjust the family-wise error rate:
- Chen, S.-Y., Feng, Z., Yi, X. (2017). A general introduction to adjustment for multiple comparisons. Journal Of Thoracic Disease, 9(6), 1725-1729. https://doi.org/10.21037/jtd.2017.05.34
Statistical power
An introductory video on statistical power by the OSF:
A video on consequences of low statistical power by the OSF:
An R package to assess Type S and Type M error rates:
Correcting for inflated effect sizes fueled by publication bias:
- Simonsohn, U., Nelson, L. D., Simmons, J. P. (2014). \(p\)-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results. Perspectives on Psychological Science, 9(6), 666-681. https://doi.org/10.1177/1745691614553988
Frequentist power analysis in R:
G*Power - A Software for frequentist power analysis:
A blogpost on Bayesian power analysis:
Transparency
A survey that aimed to gain insight into why authors keep their data private:
- Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E., Wagenmakers, E.-J. (2018). Data Sharing in Psychology: A Survey on Barriers and Preconditions. Advances in Methods and Practices in Psychological Science, 1(1), 70-85. https://doi.org/10.1177/2515245917751886
More information on open science badges:
More information on TOP Guidelines:
A platform for disclosure statements:
Guidelines to write README files and Metadata:
More on the hows and whys of sharing:
- Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Hoeflich Mohr, A., IJzerman, H., Nilsonne, G., Vanpaemel, W., Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4(1), 20. https://doi.org/10.1525/collabra.158
Miscellaneous
A comic about the replication crisis:
A youtube-playlist of 5-10 minute videos on open science:
A video on norms in science:
Highly recommended resources that provide a broad overview of the replication crisis, contributing factors, and solution attempts:
Chambers, C. (2017). The Seven Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice. Princeton University Press. https://doi.org/10.1515/9781400884940
Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C. D., Du Sert, N. P., … Ioannidis, J. P. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 0021. https://doi.org/10.1038/s41562-016-0021