Workshop description:

In the first part of this workshop, a practical introduction is provided to recently developed statistical tools that can be used to deal with the inherent uncertainties in an inductive science. The goal is to allow researchers to improve the ways in which they design and evaluate research. The benefits of meta-analytic techniques such as p-curve analysis and meta-regression will be highlighted, with a focus on using these techniques to control for the effects of publication bias when evaluating studies. In addition, sequential analyses will be discussed, which will allow researchers to design well-powered studies by collecting data, analyzing it, collecting more data, and analyzing it, without p-hacking, even when effect sizes are uncertain. These techniques allow researchers to improve their statistical inferences from a Frequentist approach to statistics.

In the second part of the workshop, a gentle introduction is presented to some Bayesian perspectives in applied data analysis which constitute natural alternatives to standard null-hypothesis significance testing. Here the main objective is to provide some simple and manageable examples which highlight basic differences with respect to the frequentist perspective and stress the importance of shifting towards a more modeling oriented approach in data analysis. However, we will also see that there are some relevant differences in the way researchers adopt the Bayesian methodologies in applied data analysis and that they do not necessarily converge to the same substantive interpretations and goals.

Slides:

Part 1 (pdf), Part 2 (pdf)

References - part one

Lakens D. & Evers E.R.K. (2014). Sailing from the seas of chaos into the corridor of stability: Practical recommendations to increase the informational value of studies. Perspectives on Psychological Science, 9, 278-292. (publisher web site)..

Simonsohn U., Nelson L.D., & Simmons J.P. (2014). P-Curve: A Key to the File-Drawer. Journal of Experimental Psychology: General, 143, 534-547. (pdf copy | publisher web site).

Nickerson R.S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301. (pdf copy | publisher web site).

Lakens D. (2014). Performing high-powered studies efficiently with sequential analyses. European Journal of Social Psychology. Special issue article: Methods and statistics in social psychology: Refinements and new developments, 44, 701-710. (pdf copy | publisher web site).

References - part two

Gelman A. & Carlin J. (2014). Beyond power calculations: assessing Type S (Sign) and Type M (Magnitude) errors. Perspectives on Psychological Science, 9, 641-651. (pdf copy | publisher web site).

Masson M.E.J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679-690. (pdf copy | publisher web site).

Gelman A. & Cosma R.S. (2013). Philosophy and the practice of Bayesian statistics. British Journal of Mathematical and Statistical Psychology, 66, 8-38. (pdf copy | publisher web site).

Wagenmakers E-J., Verhagen J., Ly A., Bakker M., Lee M.D., Matzke D., Rouder J.N., & Morey R.D. (2014). A power fallacy. Behavior Research Methods, DOI: 10.3758/s13428-014-0517-4. (pdf copy| publisher web site).