Background: the Simon task

The Simon task is pretty cool. The task is designed to see if responses are faster and/or more accurate when the stimulus to respond to occurs in the same relative location as the response, even if the stimulus location is irrelevant to the task. For example, it is faster to respond to a stimulus presented on the left of the screen with a key that is on the left of the keyboard (e.g. q), than with a key that is on the right of the keyboard (e.g. p).

Experiment

Participants

A total of 213 participants took part in an online version of a Simon task. Participants were students enrolled in either “Introduction to Cognitive (Neuro-)Psychology” (N = 166), or “Experimental Psychology Lab Practice” (N = 39) or both (N = 4).

Materials & Design

Each trial started by showing a fixation cross for 200 ms in the center of the screen. Then, one of two geometrical shapes was shown for 500 ms. The target shape was either a blue square or a blue circle. The target shape appeared either on the left or right of the screen. Each trial determined uniformly at random which shape (square or circle) to show as target and where on the screen to display it (left or right). Participants where instructed to press keys q (left of keyboard) or p (right of keyboard) to identify the kind of shape on the screen. The shape-key allocation happened experiment initially, uniformly at random once for each participant and remained constant throughout the experiment. For example, a participant may have been asked to press q for square and p for circle.

Trials were categorized as either ‘congruent’ or ‘incongruent’. They were congruent if the location of the stimulus was the same relative location as the response key (e.g. square on the right of the screen, and p key to be pressed for square) and incongruent if the stimulus was not in the same relative location as the response key (e.g. square on the right and q key to be pressed for square).

In each trial, if no key was pressed within 3 seconds after the appearance of the target shape, a message to please respond faster was displayed on screen.

Procedure

Participants were first welcomed and made familiar with the experiment. They were told to optimize both speed and accuracy. They then practiced the task for 20 trials before starting the main task, which consisted of 100 trials. Finally, the experiment ended with a post-test survey in which participants were asked for their student IDs and the class they were enrolled in. They were also able to leave any optional comments.

Results

Loading and inspecting the data

We load the data into R and show a summary of the variables stored in the tibble:

It is often useful to check general properties, such as the mean time participants spent on the experiment:

About 21.62 minutes is quite long, but we know that the mean is very susceptible to outliers, so we may want to look at a more informative set of summary statistics:

HW1: Make a histogram of the timeSpent.

Summarizing & cleaning the data

We look at outlier-y behavior at the level of individual participants first, then at the level of individual trials.

Individual-level error rates & reaction times

It is conceivable that some participants did not take the task seriously. They may have just fooled around. We will therefore inspect each individual’s response patterns and reaction times. If participants appear to have “misbehaved” we discard all of their data. (CAVEAT: Notice the researcher degrees of freedom in the decision of what counts as “misbehavior”! It is therefore that choices like these are best committed to in advance, e.g. via pre-registration!)

We can calculate the mean reaction times and the error rates for each participant.

Let’s plot this summary information:

Here’s a crude way of branding outlier-participants:

We then clean the data set in a first step by removing all participants identified as outlier-y:

Trial-level reaction times

It is also conceivable that inidividual trials resulted in early accidental key presses or were interrupted in some way or another. We therefore look at the overall distribution of RTs and determine (similarly arbitrarily, but once again this should be planned in advance) what to exclude.

Let’s decide to exclude all trials that lasted longer than 1 second and also all trials with reaction times under 100 ms.

Exploring the (main) data

We are mostly interested in the influence of congruency on the reactions times in the trials where participants gave a correct answer. But here we also look at, for comparison, the reaction times for the incongruent trials.

Here is a summary of the means and standard deviations for each condition:

Here’s a plot of the reaction times split up by whether the answer was correct and whether the trial was congruent or incongruent.

To be continued (for homework)

The analysis above is very preliminary. We could do many more things (which we should pre-register or be explicit that they are exploratory). For example, the responses made by students enrolled in the different classes could differ. In the dataframe, there is a column class, which we could use to split the data into two groups to compare them (we already did this once when we described the sample).

For example:

d %>% filter(class == "Experimental Psych Lab") # filters the students only enrolled in Experimental Psych Lab
## # A tibble: 4,391 x 18
##    submission_id    RT condition correctness class experiment_id
##            <dbl> <dbl> <chr>     <chr>       <chr>         <dbl>
##  1          7439   991 incongru… correct     Expe…            52
##  2          7439   341 incongru… correct     Expe…            52
##  3          7439   393 congruent correct     Expe…            52
##  4          7439   344 incongru… incorrect   Expe…            52
##  5          7439   709 incongru… correct     Expe…            52
##  6          7439   591 incongru… correct     Expe…            52
##  7          7439   516 congruent correct     Expe…            52
##  8          7439   518 incongru… correct     Expe…            52
##  9          7439   499 congruent correct     Expe…            52
## 10          7439   375 congruent correct     Expe…            52
## # … with 4,381 more rows, and 12 more variables: key_pressed <chr>,
## #   p <chr>, pause <dbl>, q <chr>, target_object <chr>,
## #   target_position <chr>, timeSpent <dbl>, trial_number <dbl>,
## #   trial_type <chr>, mean_RT <dbl>, error_rate <dbl>, outlier <lgl>

d %>% filter(class == "Intro Cogn. Neuro-Psychology") # filters the students only enrolled in Experimental Psych Lab
## # A tibble: 19,314 x 18
##    submission_id    RT condition correctness class experiment_id
##            <dbl> <dbl> <chr>     <chr>       <chr>         <dbl>
##  1          7432   938 incongru… correct     Intr…            52
##  2          7432   744 incongru… correct     Intr…            52
##  3          7432   528 incongru… correct     Intr…            52
##  4          7432   706 incongru… correct     Intr…            52
##  5          7432   547 congruent correct     Intr…            52
##  6          7432   591 incongru… correct     Intr…            52
##  7          7432   652 incongru… correct     Intr…            52
##  8          7432   627 incongru… correct     Intr…            52
##  9          7432   485 incongru… incorrect   Intr…            52
## 10          7432   515 incongru… correct     Intr…            52
## # … with 19,304 more rows, and 12 more variables: key_pressed <chr>,
## #   p <chr>, pause <dbl>, q <chr>, target_object <chr>,
## #   target_position <chr>, timeSpent <dbl>, trial_number <dbl>,
## #   trial_type <chr>, mean_RT <dbl>, error_rate <dbl>, outlier <lgl>

d %>% filter(class == "both") # filters the students enrolled in both classes
## # A tibble: 477 x 18
##    submission_id    RT condition correctness class experiment_id
##            <dbl> <dbl> <chr>     <chr>       <chr>         <dbl>
##  1          7462   641 incongru… correct     both             52
##  2          7462   460 congruent correct     both             52
##  3          7462   518 incongru… correct     both             52
##  4          7462   586 congruent correct     both             52
##  5          7462   612 incongru… correct     both             52
##  6          7462   445 congruent correct     both             52
##  7          7462   445 congruent correct     both             52
##  8          7462   452 congruent correct     both             52
##  9          7462   355 congruent correct     both             52
## 10          7462   478 incongru… correct     both             52
## # … with 467 more rows, and 12 more variables: key_pressed <chr>, p <chr>,
## #   pause <dbl>, q <chr>, target_object <chr>, target_position <chr>,
## #   timeSpent <dbl>, trial_number <dbl>, trial_type <chr>, mean_RT <dbl>,
## #   error_rate <dbl>, outlier <lgl>

Analysis

We are interested in comparing the RTs of correct answers in the congruent and incongruent conditions. We saw a difference in mean reaction times, but we’d like to know if this difference is meaningful. One way of testing this is by running a regression model, which tries to predict RTs as a function of conguency. In the simplest case we would therefore do this:

According to this analysis, there is reason to believe in a difference in RTs between congruent and incongruent groups. The coefficient estimated for the incongruent group is on average ca. 25 ms higher than that of the congruent group.

However, we can also look at the interaction between correctness and condition. As shown in the above graph, there are four different cells in a 2x2 grid.

In the below model, this is coded with ‘dummy coding’ such that the top-left cell (congruent-correct) is the intercept, and each other cell is calculated by the addition of offsets.

We may want to ask the question: are reaction times to correct-congruent responses shorter than reaction times to incorrect-incongruent responses?

To do this, we first need to extract the posterior samples from our model.

Then we need to determine the correct offsets to match the correct-congruent and incorrect-incongruent cells in the design matrix.

Once we know these, we can calculate the probability that the comparison is in the correct direction.

Homework: What does this value mean? Answer on Stud.IP VIPS before Thursday May 9, 09:00