Post by pzelchenko on Dec 25, 2021 7:31:59 GMT
Merry Christmas! Our experiments duplicate a protocol frequently used in studies of contextual cueing (CC) (see Chun & Jiang, 1998). We run each subject through 720 trials, broken into 30 blocks of 24 trials each. Each one of these 24 trials will present what CC experiments call an "old" or "new" display. The "old" condition picks from only 12 repeating display layouts, while the "new" condition picks from among 12 unique new layouts chosen for that task set from a set of 360 random layouts. Hence in each task block, there will be 24 trials: the 12 "old" displays and the 12 "new" displays, all interspersed randomly (e.g., ONNONONONNNOONONOONNONOO). There are a total of 372 possible layout arrangements in the experiment (12old + 30*12new = 372); these are all pre-generated as bitmaps. The task is to select the target among distractors and measure RT.
We're generating trials using the following task and block code for each of the 30 blocks (task 2 shown; all sets from 1-30 are otherwise identical):
Table "all" is from the file "all_tables.txt" (61 columns; attached). The first 30 columns are one for each of the 30 task sets, each column containing only the 24 stimulus image names (e.g., -show bitmap @2- in the above example). Note that for each block, the "new" are simply the following 12 in the set of bitmaps: these 12 should be shuffled with the 12 "old" to make up the 24 trials for that block. For the next block, the same 12 "old" will be shuffled with a subsequent range of 12 "new" layouts. Columns 31-60 hold the response keys (1-2) for each corresponding bitmap, and column 61 simply says "old" in rows 1-12 and "new" in rows 13-24 and is used only for data collection.
It appears that most subjects are not showing any randomization artifacts. However, subjects 1001 and 1008 are definitely skewed and wildly skewing the SD. Two samples of the same number of 7200 random coin tosses show a mean run length of 2 +/- 0.1, across each of 720 runs in 10 fictitious subjects (this means that HH or TT will be as common as HT or TH, but longer runs will be increasingly rare; the runs of 10 to 20 that we are regularly seeing should be vanishingly uncommon):
Nearly all subjects fall within the given normal range, with a couple of slight outliers. None of these subjects seems to exhibit unusually long runs or the within-condition forward and reverse runs of 2-3 consecutive objects. Subjects 1001 and 1008 tend to show much longer runs as well as frequent within-condition runs.
Unless we're doing something wrong in our code, I'm suspecting an implementation-specific issue with PsyToolkit's random-number generation routines in a specific browser. I'm trying to track down which browsers were used by which subjects. The two possibilities are Chrome or 360, probably both using Windows. See also three files attached.
We're generating trials using the following task and block code for each of the 30 blocks (task 2 shown; all sets from 1-30 are otherwise identical):
task T2
table all
part fixation
show bitmap @2
readkey @32 5000
part feedback
save RT STATUS KEY @2 @61 BLOCKNAME &trialblock B2
message info3
set &trial 0
tasklist
T2 24 all_before_repeat
endTable "all" is from the file "all_tables.txt" (61 columns; attached). The first 30 columns are one for each of the 30 task sets, each column containing only the 24 stimulus image names (e.g., -show bitmap @2- in the above example). Note that for each block, the "new" are simply the following 12 in the set of bitmaps: these 12 should be shuffled with the 12 "old" to make up the 24 trials for that block. For the next block, the same 12 "old" will be shuffled with a subsequent range of 12 "new" layouts. Columns 31-60 hold the response keys (1-2) for each corresponding bitmap, and column 61 simply says "old" in rows 1-12 and "new" in rows 13-24 and is used only for data collection.
This code all seems to work fine. The problem comes with the randomized presentation of these columns. Whether to present the "old" (e001-e012) or "new" (e025-e036) at any trial N should hover near chance. At most, we might normally see two or perhaps three consecutive "old" or "new" trials ("head/tail" coins) in a row. And yet, for many blocks, there are some highly improbable runs of presentation. Too frequently we are seeing runs of 10 and higher.
From our 10 subjects collected so far, there are some patterns that offer clues: We see the skew only in 2 of the 10 subjects. The pattern within these subjects is very long consecutive runs of "old" or "new" screens (despite the fact that they are taken from a single table column!). Furthermore, within these subjects' long runs of new or old, we seem to see runs of items of two and sometimes three, either in forward or reverse order:
It appears that most subjects are not showing any randomization artifacts. However, subjects 1001 and 1008 are definitely skewed and wildly skewing the SD. Two samples of the same number of 7200 random coin tosses show a mean run length of 2 +/- 0.1, across each of 720 runs in 10 fictitious subjects (this means that HH or TT will be as common as HT or TH, but longer runs will be increasingly rare; the runs of 10 to 20 that we are regularly seeing should be vanishingly uncommon):
Nearly all subjects fall within the given normal range, with a couple of slight outliers. None of these subjects seems to exhibit unusually long runs or the within-condition forward and reverse runs of 2-3 consecutive objects. Subjects 1001 and 1008 tend to show much longer runs as well as frequent within-condition runs.
Unless we're doing something wrong in our code, I'm suspecting an implementation-specific issue with PsyToolkit's random-number generation routines in a specific browser. I'm trying to track down which browsers were used by which subjects. The two possibilities are Chrome or 360, probably both using Windows. See also three files attached.