Reporting experimental results (and more)

Michael Franke

road map for today

 

  • project suggestions

 

  • Canvas functions

 

  • Rmarkdown

 

  • APA guidelines

 

  • practical exercises with R

Canvas

Job: extend your experiment

  • instead of pictures (of cats etc.), use automatically generated pictures of colored shapes, like these:

    binary

  • do not generate pictures manually beforehand, but generate a random picture on each trial

Job description

use the CanvasTemplate, available here: https://github.com/babe-project/CanvasTemplate

  • adapt it to do the following, on each trial:
    • sample total_set_size from {10,25,50} and a non-zero focal_set_size of at most total_set_size
    • choose a random shape from {circle, square, triangle}
    • display focal_set_size of shape in the focal_color, the rest in the other_color (choose colors from {blue, red, yellow, green})
    • arrange colored shapes arbitrarily on the screen
    • dynamically construct the sentence to be judged/rated as: “Some of the shapes are focal_color.”
    • record all relevant random choices for each trial’s picture

Canvas template

  • in index.html:
  • in helpers.js:
  • in views.js:

Rmarkdown

Rmarkdown: overview

  • lean markup language which allows execution of R code
  • exports to multiple formats
    • simplifies a reproducible work flow

Rmarkdown

How to report an experiment

APA guidelines to structure experiment reports (roughly)

  • (optional) motivation/design
    • what kind of experiment (e.g., between vs. within-subject design) and why
  • participants
    • how many participants were recruited how/from where; gender distribution; age; payment etc.
  • materials & apparatus
    • which pictures/sentences/instructions/machinery was used (e.g., which eye-tracker, …)
  • procedure
    • how the experiment was structured, how many trials of which kind in which order etc.
  • (optional) analysis (also sometimes referred to as ‘design’)
    • what are the dependent and independent variables used for statistical analysis
  • (optional) data preparation
    • data manipulation before further analyses, e.g., data cleaning, variable transformation
  • results
    • summary statistics (e.g., mean error rates), plots & analyses (e.g., \(p\)-values)
  • discussion
    • interpret the results