Factorial vignette study
Show each participant one version of a scenario from a factorial design, then have everyone rate it on the same questions.
When to use it
Section titled “When to use it”Use this for vignette experiments: a short scenario varies on crossed factors (here, a job applicant’s gender and age), each participant reads exactly one version, and all participants answer the same rating questions. Showing one person two versions reveals the manipulation, so the factor assignment must be between subjects, and duplicating the project per version would split the sample and the results. subjectGroup keeps it in one file: one tagged instructions row per cell of the design, shared untagged questions for everyone, and the assigned version recorded per participant in the results.
Trial file
Section titled “Trial file”| type | subjectGroup | title | head | content | responseType | responseOptions | required | gender | age | button1 |
|---|---|---|---|---|---|---|---|---|---|---|
| instructions | You will read a short description of a job applicant, then answer a few questions about them. | NEXT | ||||||||
| instructions | 1 | Applicant profile | Sarah is 28 years old and has applied for the team lead role. Her references describe her as dependable and direct. | female | young | NEXT | ||||
| instructions | 2 | Applicant profile | Sarah is 58 years old and has applied for the team lead role. Her references describe her as dependable and direct. | female | older | NEXT | ||||
| instructions | 3 | Applicant profile | Mark is 28 years old and has applied for the team lead role. His references describe him as dependable and direct. | male | young | NEXT | ||||
| instructions | 4 | Applicant profile | Mark is 58 years old and has applied for the team lead role. His references describe him as dependable and direct. | male | older | NEXT | ||||
| form | How competent do you consider this applicant? | likert | Not at all;Slightly;Moderately;Very;Extremely | 1 | ||||||
| form | How likely would you be to hire this applicant? | likert | Very unlikely;Unlikely;Unsure;Likely;Very likely | 1 | ||||||
| form | What annual salary would you offer this applicant? | box | 1 |
type,subjectGroup,title,head,content,responseType,responseOptions,required,gender,age,button1 instructions,,,,"You will read a short description of a job applicant, then answer a few questions about them.",,,,,,NEXT instructions,1,Applicant profile,,Sarah is 28 years old and has applied for the team lead role. Her references describe her as dependable and direct.,,,,female,young,NEXT instructions,2,Applicant profile,,Sarah is 58 years old and has applied for the team lead role. Her references describe her as dependable and direct.,,,,female,older,NEXT instructions,3,Applicant profile,,Mark is 28 years old and has applied for the team lead role. His references describe him as dependable and direct.,,,,male,young,NEXT instructions,4,Applicant profile,,Mark is 58 years old and has applied for the team lead role. His references describe him as dependable and direct.,,,,male,older,NEXT form,,,How competent do you consider this applicant?,,likert,Not at all;Slightly;Moderately;Very;Extremely,1,,, form,,,How likely would you be to hire this applicant?,,likert,Very unlikely;Unlikely;Unsure;Likely;Very likely,1,,, form,,,What annual salary would you offer this applicant?,,box,,1,,,
How it works
Section titled “How it works”The four tagged instructions rows are the 2 x 2 design: gender (female, male) crossed with age (young, older), one row per cell, with the vignette text in content under a shared title. Each participant is assigned one subjectGroup value per session (randomly by default, or evenly across participants when the project’s even-allocation setting is on), sees only that group’s row plus every untagged row, and their group number is saved into the results as their condition label.
The form rows are untagged, so every participant rates whichever vignette they read on identical questions: two likert scales with labelled points and a free box for the salary offer. Identical questions across versions is what makes the factor comparison clean.
The gender and age columns are custom annotation columns documenting the factor levels in the trial file itself. At analysis time the condition comes from the saved subjectGroup info value (instructions rows write no results rows, so the annotations do not travel there); map the group numbers back to factor levels with the trial file at hand.
Variations
Section titled “Variations”- More factors or levels: one tagged row per cell (2 x 2 x 2 needs eight rows);
subjectGroupdoes not care how many groups there are. - Several vignettes per participant: bare group numbers all share one assignment per session, so reuse the same tags to hold one version consistent throughout, or use named group labels (
scen1:1,scen2:1, …) to give each scenario its own independent assignment. - Score or recode ratings for an index: add a
scorecell to the likert rows (see the Likert scale with score recipe). - Attach a scored comprehension check: an untagged scored question (see the Scored quiz form recipe) catches participants who did not read the vignette.