example implementation

an example implementation is available here:

QuantifierNaturalness

deployment in _babe

deployment := handling results & recruiting participants

1. debug mode:
   • execute experiment locally in your browser
   • show results of in HTML-table in browser

2. live experiment:
   • experiment is hosted on a web-server
   • anyone can follow a link to do the experiment
     • possibly: participants are recruited and reimbursed via crowd-sourcing platform
   • experimental results are stored in and retrieved from a database using the _babe server app

going live with _babe

content of MinimalTemplate/config/config_deploy.js

var config_deploy = {
    // obligatory fields
    
    // the experimentID is needed to recover data from the _babe server app
      // you receive the experimentID when you create the experiment using the _babe server app
      "experimentID": "8",

    // set deployment method; use one of:
    //'debug', 'localServer', 'MTurk', 
    // 'MTurkSandbox', 'Prolific', 'directLink'
    "deployMethod" : "directLink", 
    
    // optional fields
    
    // who to contact in case of trouble
    "contact_email": "YOUREMAIL@wherelifeisgreat.you", 
};

relevant for us are:

1. localServer &
2. directLink

using the directLink method

use directLink for experiments where participants receive a link and work from wherever they are; results are submitted to and retrieved from a database via the internet

• use our installation of the _babe backend on Heroku
  • you will receive a password for this when the time comes
• set deployMethod: "directLink"
• visit the _babe server app at https://babe-backend.herokuapp.com/ & create a new experiment
  • note the experimentID that is automatically asigned & enter it in config_deploy.js
  • the experiment must be set to live mode in the server app (otherwise it refuses data submissions)
• retrieve your results in the browser at address https://babe-backend.herokuapp.com/
• alternatively: install your own instance of the server app as explained here: _babe backend Heroku installation
  • in this case, change the address to your server app instance in scripts/main.js

using the localServer method

use localServer for experiments where participants are physically present and use your computer (e.g., in the lab or in the field); results are stored on a local database (no internet connection required)

• install the _babe backend locally on the computer where participants do the experiment
  • follow installation instructions here: _babe backend local installation
• set deployMethod: "localServer"
• visit the _babe server app at localhost:4000 & create a new experiment
  • note the experimentID that is automatically asigned & enter it in config_deploy.js
  • the experiment must be set to live mode in the server app (otherwise it refuses data submissions)
• retrieve your results in the browser at address localhost:4000

preregistration reports

irrevocable public commitment to experimental design/materials/procedure and statistical analysis

two kinds of analyses (to be distinguished clearly in your paper/report):

1. confirmatory:
   • the experiment was conducted with the intention to test a hypothesis
     • ideally preregister this!

2. exploratory:
   • explore your data to generate new hypotheses
     • if anything interesting comes up: preregister and run a new study to test the new hypothesis

two kinds of preregistration reports

1. peer-reviewed preregistration:
   • journal reviewers decide whether your paper will be published based on your preregistration report

2. simple preregistration:
   • just commit in public using a suitable platform, e.g., from the Open Science Foundation
   • this is what we will use
   • concretely, use the OSF preregistration challenge format for your preregistration report
     • template can be downloaded here: GoogleDoc PreReg challenge template

plan of action for your group project

1. team up, find topic, divide roles
2. visit us with a preliminary plan, which we will make more concrete together
   • which study, which materials, how to realize (lab or online, …)
3. make a schedule and communicate it with us
4. plan & implement experiment
5. pilot your experiment for debugging (2-3 participants (this could be you!))
6. implement statistical analyses based on debug-pilot data
7. write preregistration report
   • include analyses scripts
   • wait for feedback from us before continuing
8. run experiment
9. write paper