tyak9
New Member
Posts: 13
|
Post by tyak9 on Oct 25, 2020 23:25:36 GMT
From what I have understood, the default is that the embedded experiments will have their data saved within the experiments themselves and not with the survey data. Is there a way to have the experiment data be saved along with the rest of the survey data instead of a separate sheet?
The reason I would like to do this is because I will have 6 embedded experiments and I want to make sure that I know which participants data is what. This is a concern to me as the participants could be taking it at the same time and I would imagine that the data will be saved in the order it is received and not by participant, being the experiments are separate.
Also, I will be using SPSS to analyze my data and it would be nice to have all of the data organized by participant already so that I do not have to figure that part out and potentially mess that up.
Thanks in advance for all of your replies!
|
|
|
|
Post by joebutler on Oct 26, 2020 0:40:22 GMT
This can be done, as I've set up a survey which does not let you do a later experiment if you score below a threshold on an earlier experiment. The information I used to achieve this was below: www.psytoolkit.org/lessons/var_in_out.html (if you don't want participants to receive feedback, then you can just replace the feedback text with something like "experiment finished, click to continue". There may be another way of doing this, but that's how I've implemented it). |
|
evag
New Member
Posts: 9
|
Post by evag on Oct 26, 2020 8:55:44 GMT
In fact this is our plan as well: we have several consecutive tests as a survey and then several experiments embedded in the survey. We expected to be able to recover the data as a whole from each participant. If it is not possible then we will try to do what joe butler says. It is not possible to analyse the data of the experiments without the survey tests data and all of them are linked.
As the pandemic soars more and more complex experiments have to be translated to online versions. We expect to find challenges along the way as the current Qualtrix/Gorilla and co. software is not usable for such tasks.
|
|
|
|
Post by alexandra (AEC) on Oct 26, 2020 10:03:15 GMT
Data is always saved with the random code each participant receives at the start. Using a software like R, you can write a script using this identifier that creates tables for each participant, which you then can load into SPSS.
|
|
tyak9
New Member
Posts: 13
|
Post by tyak9 on Oct 26, 2020 19:04:35 GMT
This can be done, as I've set up a survey which does not let you do a later experiment if you score below a threshold on an earlier experiment. The information I used to achieve this was below: www.psytoolkit.org/lessons/var_in_out.html (if you don't want participants to receive feedback, then you can just replace the feedback text with something like "experiment finished, click to continue". There may be another way of doing this, but that's how I've implemented it). Would you mind putting a sample of your code where you used the var in and var out functions to accomplish this? Those were the functions I thought of using but after reading it closer, I did not feel like it would work. I may have misunderstood something though. |
|
tyak9
New Member
Posts: 13
|
Post by tyak9 on Oct 26, 2020 19:05:18 GMT
Data is always saved with the random code each participant receives at the start. Using a software like R, you can write a script using this identifier that creates tables for each participant, which you then can load into SPSS. That is a very good point! Thank you for bringing that to attention! If I cannot get this to go the way I would like, this will be a good back up option! |
|
evag
New Member
Posts: 9
|
Post by evag on Oct 28, 2020 10:47:55 GMT
I don't understand this point from Alexandra (AEC): are you sure a random code is generated for each participant and, somehow, is available inside each experiment that is embedded in the survey?
The only other option I can think of is creating a random string (or two if we are not sure the random generator is sound, there is a precedent we heard -different software- where a big study with more than 300 participants had repetitions in the subject ID!!).
Such random string then in passed around to be stored and saved inside each experiment embedded, just in case. We would like a more streamlined solution though.
|
|
|
|
Post by joebutler on Oct 28, 2020 11:00:47 GMT
Hi Evag - I need to pop out for the day in moment, but I will post my solution later on. It involves var out within the experiment; the jump command within the survey based on the var out command; and the feedback command within the experiment (which is probably the most complicated bit depending on your experiment, as it means calculating the data within the experiment which I found a bit fiddly as this is new syntax to me). .
However reference your point about duplicating the random string, there is also a server time inside the subject files, so maybe you can use that as an additional sanity check when marrying up subject experimental data to their survey data.
|
|
|
|
Post by yiannis on Oct 28, 2020 13:05:54 GMT
I don't understand this point from Alexandra (AEC): are you sure a random code is generated for each participant and, somehow, is available inside each experiment that is embedded in the survey? You can try it yourself I guess. The option for the code can be found in the survey options and it should be saved both in the surveys and the embedded exps. photos.app.goo.gl/woqLiEcXfbe3kCid6 |
|
tyak9
New Member
Posts: 13
|
Post by tyak9 on Oct 28, 2020 13:57:42 GMT
I don't understand this point from Alexandra (AEC): are you sure a random code is generated for each participant and, somehow, is available inside each experiment that is embedded in the survey? The only other option I can think of is creating a random string (or two if we are not sure the random generator is sound, there is a precedent we heard -different software- where a big study with more than 300 participants had repetitions in the subject ID!!). Such random string then in passed around to be stored and saved inside each experiment embedded, just in case. We would like a more streamlined solution though. There is a video from Psytoolkit that shows that this code is generated in each of the experiments and is identical for each participant. I believe you can find it in the videos about embedded experiments in the lessons section. |
|
|
|
Post by joebutler on Oct 30, 2020 0:21:47 GMT
Tyak9 - here is some code I used to test/learn the function:
In the survey:
# check the large survey-library for ready-to-run examples you can
# cut and paste here
l: examplequestion1
t: radio
q: Which of the following foods do you like best?
- Rice
- Noodles
- Bread
- Potatoes
l:
t: experiment
q: Next will be a short computer game...
- perceptual_task
l: myfeedback
t: info
q: Your perception score on a scale from 1 to 10 was {$percept_correct}.
Your score was {$percept_correct}
l:
t: jump
- if $percept_correct <= 7 then goto examplequestion3
l: examplequestion2
t: check
q: Select all of the following drinks did you had today?
- Water
- Orange juice
- Tea
- Coffee
l: examplequestion3
t: check
- you failed!
In the experiment:
options
fullscreen
resolution 800 600
origin topleft
set &trialnum 1
bitmapdir stimuli
set &colour_correct 0
set &shape_correct 0
var out percept_correct
End of experiment:
feedback
set &percept_correct max c7;
text align left
text -200 -200 "you have finished:"
text -200 150 "press mouse or touch screen"
wait_for_key mouse
end
Hopefully this is helpful!
|
|
tyak9
New Member
Posts: 13
|
Post by tyak9 on Oct 30, 2020 23:56:33 GMT
joebutler What purpose do the the $ serve in this script? I know what they do in excel, which I believe is also JavaScript, but it does not look like it is doing the same thing, yet it look necessary to do what you did.
|
|
|
|
Post by alexandra (AEC) on Oct 31, 2020 10:09:36 GMT
The $ serves as a variable identifier. In the example posted by joebutler you have the experiment variable &percept_correct , which is available in the survey through the function "var out". See here: www.psytoolkit.org/doc3.2.0/syntax.html#_var_out$percept_correct in the survey calls the content of this variable. |
|
ericg
New Member
Posts: 2
|
Post by ericg on Nov 5, 2020 19:12:03 GMT
Hi all,
I am trying to do the same thing with one of my research students. We have a multitasking experiment embedded within a survey. We have asked people in the survey to create a unique identifier for themselves to link to their data. This info comes through in the “data” Excel and csv files. But that is not tied to the “mean”, “median”, etc Excel and csv files; those files only have a single column of the respective reaction time values and are not in any way linked to the identifier.
Complicating matters further, there does not seem to be a way to even really compare across two files and match them.
Someone suggested possibly scripting with R, but neither of us are skilled with this enough to do that on the timeline we are working on (pending graduation timeline). Is there a way to set up something in PsyToolKit or the experimental code (standard multitasking experiment within the experiment library) to generate a variable that will be in the output file(s)?
Many thanks!
|
|
andre
Experienced
Posts: 35
|
Post by andre on Nov 24, 2020 11:49:13 GMT
This is maybe not spot on what you where looking for, but what I do when I have multiple experiments called from one single survey:
1. In the survey-data file, you get one line/row per participant. This row contains the name of each output file of the experiments in different columns. Thus, you can always link the Survey data with all the experiment data output files of one single participant.
2. I merge survey data and experiment in several steps. I analyse the experiment data myself in Excel, using Pivot tables to generate means for the different conditions (incl using 'filters' to remove erroneous trials and alike). However, to do this in one go, I would need some participant identifier within the experiment output file. For this, I use a script which adds the filename as the first column (see below). Then I can merge all experiment output files into one big output file, and create a Pivot table with one line/row per participant and all the different repeated-measure conditions as columns. This pivot table will have the filename as first column. I can then copy-paste this into the PsyToolKit Survey-data file and align the two (PsyToolKit saves the output file names as well, so I can just simply compare row by row. The PsyToolKit Survey data file may have additional rows for those participants who started the survey, but didn't finish the experiment(s). This then needs adjustment, just delete those cells and tell Excel to 'Shift up' the remaining cells. For ~100 datasets (N) this takes only 10 or 15 min or so per experiment, and needs to be done only once). Then I have all data of the Survey and all experiments in one big Excel file, which I use as basis for a straight copy-paste of the columns of interest to SPSS.
In more detail my analysis steps:
1. Create a batch file (e.g. with the name "AddFilenames.bat") with the following content:
@echo off &setlocal
set "folder=D:\Data"
set "output=D:\Output.txt"
>"%output%" (for %%a in ("%folder%\*.txt") do for /f "usebackq delims=" %%b in ("%%a") do echo.%%~nxa %%b)
Adjust the two set commands to your paths.
Copy the batch file into D:\Data (technically not necessary...)
1. Copy all result files (e.g. "PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt") into a separate directory (for the above script, into "D:\Data")
Such a result file has many lines like this:
-931848 -786425 practice_a -1 0 602 -999 1 -999 2 2 0 0 STaud_HighT 602 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999
-931848 -786425 practice_a -1 0 435 -999 1 -999 2 2 0 0 STaud_HighT 435 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999
2. In Windows, open a Command Line in that data directory (D:\Data) and run the batch file (just type "AddFilenames.bat" on the command line). Should also work without command line and just double-clicking on the filename in Windows Explorer.
3. You should now have a new file, "Output.txt" in D:\ This new file looks now like this:
PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt -931848 -786425 practice_a -1 0 602 -999 1 -999 2 2 0 0 STaud_HighT 602 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999
PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt -931848 -786425 practice_a -1 0 435 -999 1 -999 2 2 0 0 STaud_HighT 435 -999 -999 -999 -999 -999 -999 -999 -999 -999 -999
The filename "PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt" is now the first column and serves as participant identifier.
4. In the data directory (D:\Data) open a command line (for this, you can do a right-click in the Explorer while pressing Shift and select "Open Command Line here" or something like this. Note that Powershell doesn't work. If a Powershell opens, just type "cmd" in the Powershell window). On the command line, type the following command:
copy *.txt All_data.txt
Now, all your result files will be copied into one single text file (below each other).
5. Open this "All_data.txt" in Excel. Use the 'Import' dialogue to set the delimiter to the 'Space' character. Now you have all data in Excel. The create meaningful column names in the first line and create a Pivot table over all data.
6. The pivot table I'm mostly using has a setup as follows:
- "Filters": I usually create an extra column in Excel which is "0" if the trial is correct and "1" if it is incorrect. I use that variable (column usually called 'Error') to filter only trials which have the value of 0, effectively excluding error trials from the calculation of means [*]
- "Columns": My condition name, e.g. "Mental Load"
- "Rows": This is where the Filename goes
- "Values": For example "Average of RT" (assuming you named a column RT)
[*] In addition this coding allows to analyse error rates quickly:
- remove the error column from the Filters field
- Use "Average of Error" as Values
- Then you get relative error frequencies, i.e. a value of 0.023 means 2.3% error rate
This pivot table has one line/row per participant, and in the following columns the means of the DV I'm interested in (Mental Load in this example).
I copy-paste (paste 'Values only') the data-part of that table into an empty tab in the Excel sheet, and sort the whole table by the filename column (which contains the "PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt"). Thus, all data are sorted by filename.
7. I open the Survey data output file ('data.csv') in Excel, and also sort the whole file by the column contain the filename of the experiment (e.g. PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt). Then I create enough empty columns to the right of the column where the filename of the experiment (e.g. "PRP-V1_1.2020-08-06-1255.data.45163140-b658-44bb-adf9-2f7f269706a2.txt") is given (or just hide the columns to the right).
8. Now I can copy-paste the result of step 6 into the survey file from step 7. If there are no missing/partial data, both will perfectly align. If someone started the Survey (and created a 'line/row' in the data.csv), then this needs to be removed. Just select the cells to the left (careful if you hid cells/columns), right click, "Delete..." and move cells up.
To make it easy to spot inconsistencies, I usually create an additional column. Suppose, the filenames in the data.csv were stored in column X, and you pasted the pivot-table starting into column Y (so that column Y contains the filenames from the Pivot table), then I create a new column Z with the following formula:
=if(X2=Y2, "", "ERROR").
Then this column Z will be empty if the data.csv and the pivot-data align (i.e., belong to the same participant), and show "ERROR" if they come from different participants.
9. I potentially do some further stuff in Excel (e.g. calculating the mean of a couple of survey questions if I haven't done it in PsyToolKit)
9. I now can copy-paste the columns of interest to SPSS (including one of the 'filename' columns to have a reference point), they are in the format to then run ANOVAs, t-tests, etc straight away.
This may sound complicated, but if you've done it a few times, it becomes second nature. For the import of a whole study I usually don't need more than an hour in total.
Hope this is of help to some...
|
|
