Bayesian data analysis & cognitive modeling

01: course overview

Michael Franke

At a glance

 

  • BDA is about what we should believe given:
    • some observable data, and
    • our model of how this data was generated.

 

  • Our best friend will be Bayes rule: \[\underbrace{P(\theta \, | \, D)}_{posterior} \propto \underbrace{P(\theta)}_{prior} \times \underbrace{P(D \, | \, \theta)}_{likelihood}\]

 

  • If \(P(\theta \, | \, D)\) is hard to compute, we resort to magic some clever stuff.

Example: coin flips

  • \(\theta \in [0;1]\) is the bias of a coin:
    • if we throw a coin, the outcome will be heads with probability \(\theta\)
  • we have no clue about \(\theta\) at the outset:
    • a priori we consider every possible value of \(\theta\) equally likely
  • we observe that of 24 flips 7 were heads
  • what shall we believe about \(\theta\) now?

“Classic statistics”

 

 

  • relies on sampling distributions & \(p\)-values
    • standard “tests” can have rigid built-in assumptions
    • implicitly rely on experimenter’s intentions

 

  • looks at point estimates only

 

  • bag of magic tricks

tunnel vision

Any thoughts?

   

abelson quote

from Abelson (1995) “Statistics as a Principled Argument”; quote tweeted by Richard D. Morey, April 24 2017

Pros & Cons of BDA

Pro

  • well-founded & totally general
  • easily extensible / customizable
  • more informative / insightful
  • stimulates view: “models as tools”

Drawing

Con

  • less ready-made, more hands-on
  • not yet fully digested by community
  • possibly computationally complex
  • requires thinking (wait, that’s a pro!)

Drawing2

3 times Bayes

 

  1. Bayesian statistics
    • Bayesian alternatives to “classic” statistical analyses (e.g., Kruschke 2015)  
  2. Bayesian cognitive modeling
    • custom models of the data-generating process (e.g., Lee & Wagenmakers 2014)  
  3. Bayesian models of cognition
    • model (human) cognition as Bayesian inference (e.g., Goodman & Tenenbaum 2016)

Goals of this course

 

  • to understand basic ideas of BDA (contrast with NHST)

 

  • to see how BDA blends seamlessly into cognitive modeling

 

  • to be able to read current literature on BDA

 

  • to implement and evaluate Bayesian analyses

What this course will cover

  • primer on frequentist statistics
  • Bayesian data analysis
    • theory:
      • estimation, comparison criticism
    • applications:
      • generalized linear models (mixed effects)
      • selected cognitive models
  • Bayesian computing
    • MCMC techniques (focus on Hamiltonian MC)
    • Bayes factor approximations
  • programming languages
    • R
      • tidyverse
    • Stan

Credits

 

course entry requirement:

  • pass 1st homework set (due November 6)
    • simple programming exercises in R
    • maybe some easy probability basics

for 8 credits you must …

  • hand in all homework sets (5), and
  • finish with a final project, e.g.:
    • a term paper (theory, concepts, literature, …)
    • computational simulation study
    • an analysis of your own data set
    • a replication/extension of some other analysis

 

grade will be a (non-arbitrary) function of homework grades and final project

Etiquette

  • slides are not self-explanatory -> you must attend class
  • one-minded attention during class -> no social media, news or whatever
    • 5 minute half-time break
  • come prepared, read around, think along, ask questions
    • we will not trouble-shoot your software installation process
  • recap lesson later the same day:
    • rethink
    • reread
    • take notes
    • prepare questions
  • nobody is master; all are fallible; knowledge gain is community work

Preparation for next class