ohkrl
New Member
Posts: 1
|
Post by ohkrl on Apr 18, 2020 19:09:38 GMT
Dear all,
I have 4 countarbalnced lists in my experiment and am using a "jump" statement to assign participants randomly to one of the four lists. I was wondering, however, if there is a way to present each list (experiment) a roughly equal number of times across all respondens (similar to Evenly Present Elements in Qualtrics option).
Thanks much!
Oleksandr.
|
|
|
|
Post by PsyToolkit on Apr 18, 2020 21:32:15 GMT
It is not possible, but ultimately, you need to think about what you actually want theoretically.
If you want to assign different conditions randomly to participants, the way it is implemented makes a lot of sense, because it is random (as good as computer randomness is, but that is a different discussion).
For example, if you have 4 different counterbalanced experiments, and you get, say, 98, 102, 97 and 101 participants, then you are fine. It does not need to be exactly equal from any theoretical point of view.
|
|
|
|
Post by jochen on May 1, 2020 15:26:29 GMT
But wouldn't Oleksandr's approach implement proper counterbalancing? For example, if you expect some confounding serial order effects that will be roughly similar between Ss, wouldn't it be best to have exactly (rather than roughly) the same number of Ss in the "Start with task 1" block as in the "Start with task 2" block?
|
|
|
|
Post by PsyToolkit on May 1, 2020 19:26:28 GMT
The way the computer works is that it chooses a random number and will give the participant one of the 4 conditions.
The computer does not take into account how many other people have been assigned to that condition.
If you really want exactly the same number of participants, you have the following options:
1) Keep monitoring which condition people are in in each condition, and once you have enough participants for a certain condition, change your survey so that that condition won't occur anymore and then recompile.
2) After you have collected data, just randomly choose the same number of participants of each counterbalancing condition and do your analysis.
Neither is probably ideal, but it is difficult to implement this. I know people would like this and I might implement this at some point.
|
|
|
|
Post by jochen on May 1, 2020 23:37:54 GMT
Okay, I haven't yet delved into the survey part. Currently I'm just playing with the experiment coding part, and the more I play with Psytoolkit the more I like it.
But: Surely a computer can do a permutation? The functions sample (in R) or randperm (in Matlab) can easily do that. So, as long as the number of Ss is known in advance (which is usually a requirement, or isn't it?), it should be easy to implement (I volunteer to help in doing so). Anyhow, for the experimental psychologist typically using a smaller N this might be more important than for the people using questionnaires who need large N anyhow because of the larger measurement error. So maybe some such functionality could be incorporated into the blockorder command?
Best, Jochen
|
|
|
|
Post by PsyToolkit on May 2, 2020 12:20:26 GMT
Hi Jochen, I apprectiate your insights, I have your email, I will write you an email to discuss.
There are all sorts of issues. Yes, I could add a way of saying how many participants you want, but then what if people do not finish the surveys and so on. And what if two participants start the survey at the same time, but one finishes and the other does not. In the end, it is quite difficult to make sure you end up exactly with the same number of participants, depending on the timing of the start of the surveys. But I guess it can somehow be done.
Best, Gijsbert |
|
|
|
Post by jochen on May 2, 2020 12:37:10 GMT
Maybe one possibility would be to have an "exact number participants" choice that is set to "off" by default.
|
|
|
|
Post by PsyToolkit on May 2, 2020 13:05:44 GMT
Yes, that makes sense to add anyway, that is, let a survey stop when people have enough participants. And as a more detailed version, people can set how they should be distributed across conditions. The difficult is to make it work for a broad range of possible users. While we discussed a straightforward scenario here, people often wants different variations for good reasons.
Also, it cannot be done perfectly. For example, imagine you want exactly 10 users. Once you have 9 participants, 2 people just happen to start more or less at the same time, so the computer would let them both through to start, but if they both correctly finish, you will have 11 participants. This is impossible to prevent. Ultimately the only solution is to manually decide which participants you want to use.
|
|
