PS598 Unit 7 Research Design and Sample Paper Example

Envelopes That Look the Same: A PS598 Unit 7 Paper on Research Design, Validity Threats and Sample Size for a Pre-Exam Writing Experiment

[Student Name]

Purdue University Global

PS598: Master's Capstone in Psychology

Unit 7 Research Design and Sample Paper

[Instructor Name]

[Date]

The writer, the college, the enrollment figures and the planned study are composite and illustrative; the sources cited are real.

Design

Two groups, formed by chance and compared on one occasion, with a self-compassion score from week one as a covariate: that is the design. Students who consent will complete the Self-Compassion Scale Short Form in the first week of the term. Ten minutes before the first unit exam in Anatomy and Physiology I, each student will open a sealed envelope containing a writing prompt and write for ten minutes. Half the envelopes will contain the self-compassion prompt and half a neutral prompt asking students to describe the route they took to campus that morning. Students will then take the exam. Immediately afterward, they will complete the Cognitive Test Anxiety Scale about how they felt during the exam. Exam scores will be collected from the instructors after grading.

In Unit 2 I considered a third condition, expressive writing about worries. The power analysis below showed that adding it would require roughly 120 more students, more than the college can supply in one academic year. The third arm is left for a follow-up study.

Why Random Assignment

Shadish et al. (2002) treat internal validity as how confidently a study can say the thing it changed produced the difference it found. Random assignment is the strongest protection, because it spreads differences among participants, such as preparation, prior grades and anxiety, evenly across conditions on average, so that they cannot explain differences in outcomes. My hypotheses are causal, so random assignment is essential.

Randomization will be done at the student level. Envelopes will be identical on the outside and shuffled by a library aide who has no other role in the project, and students will receive them in the order they enter the room. Neither the instructors nor I will know which prompt a student received until after scoring.

Identical envelopes keep the instructors blind and the students unaware that their neighbors may be writing about something else.

Threats to Internal Validity

Shadish and colleagues list common threats to internal validity. Several are controlled by the design. Selection is controlled by random assignment. History, events between the start of the study and the outcome, is unlikely to differ between conditions, since both groups take the same exam at the same time. Testing effects are minimal because the anxiety scale is given only once, after the exam.

Others need specific safeguards. Diffusion of treatment, in which students in one condition learn about the other, is possible if students compare prompts. Since prompts are opened at the exam, the risk is low for the first exam, which is why the study uses only that one. Experimenter expectancy is reduced by keeping instructors blind. Differential attrition, if students in one condition skip the post-exam scale more often, will be checked by comparing completion rates.

One threat cannot be fully removed: students who engage with the prompt only superficially. I will code each writing sample for whether it addresses the prompt and report results with and without nonengaged writers.

Exam Day, Minute by Minute

The procedure on exam day has to fit a fixed fifty-minute class period. At the start of class, the research assistant hands out envelopes as students take their seats. Instructors read a single scripted sentence: "Please open your envelope and follow the instructions for the next ten minutes." The assistant times the writing, collects the sheets in a folder without reading them and the instructor distributes the exam. When students hand in their exams, they receive the anxiety scale on a half sheet with their study code and return it to a box by the door. The whole study adds about twelve minutes to the period, so instructors have agreed to schedule the first exam in their seventy-five-minute lab slot.

Designs I Did Not Choose

A within-subjects design, in which every student completes both kinds of writing before different exams, would need fewer students, but carryover is a serious problem: once students practice self-compassion, they cannot return to their earlier habits for the neutral session. A cluster design, randomizing whole sections rather than students, would reduce diffusion but would require many more sections to reach the same power, since students within a section resemble one another. Individual randomization with sealed envelopes balances these concerns. A waitlist design, delaying the exercise for half the students until the second exam, was also considered; it would give everyone the exercise eventually, but I chose to meet that fairness goal through the ethics plan instead, by sharing the prompt with all sections after the first exam.

External Validity

The sample will be community college students in one course at one college in Wyoming. Results may not generalize to universities, other courses or other regions. Shadish and colleagues emphasize that generalization should be argued, not assumed, so the proposal will describe the sample in detail and treat broader claims cautiously.

Sample Size

Cohen (1992) argued that studies should be planned with enough power to detect the effect sizes researchers expect, and provided conventions of small, medium and large effects. Brief interventions tend to produce small to medium effects. For the main comparison, an independent-samples test of cognitive test anxiety between two groups, I used G*Power (Faul et al., 2007) with a two-tailed alpha of .05, desired power of .80 and an expected effect size of d = 0.35, between Cohen's small and medium conventions. The required sample is about 130 students per group, or 260 in total.

Recruitment

The college enrolls about 240 students in Anatomy and Physiology I across eight sections each fall and a similar number each spring. Assuming that about 65 percent of students consent and that 10 percent of those do not complete the post-exam scale, the study will need both terms of one academic year to reach 260 complete cases. Running the study in two terms adds a possible history threat, so term will be included as a covariate.

Conclusion

A randomized two-group design with sealed, identical envelopes and blind instructors will let the study speak about cause, at least for these students on this exam. Shadish and colleagues' list of threats guided the safeguards, and the remaining threat of low engagement will be measured. A power analysis in G*Power, informed by Cohen's conventions, sets the sample at about 260 students across two terms.

References

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.