HD560 Unit 3 Design Selection Rationale Example

Two Hundred Sixty Applicants, One Hundred Twenty Places: A Unit 3 Design Selection Rationale for Evaluating Free Hearing Aids by Lottery

[Student Name]

Purdue University Global

HD560: Health Education Evaluation and Research

Unit 3 Design Selection Rationale

[Instructor Name]

[Date]

Within Earshot, Tamsen County and every number are composite; the design text and the insurance lottery study cited are real.

The Question and the Constraints

The primary question asks whether an offer of Within Earshot reduces loneliness at six months among eligible applicants, compared with a wait-list. Four constraints shape the choice. The program has 120 places this year and cannot expand. Applicants who are not served now will be served next cycle, beginning in six months. The evaluation budget allows two assessments per person. And the agency's board will not accept any design that denies the program to someone who would otherwise have received it.

Six Candidate Designs

Design 1, single-group pretest and posttest. Low cost. Rejected: last year's report used it, and it cannot separate the program from regression toward the mean, history or loss.

Design 2, comparison with older adults at the senior center who did not apply. Moderate cost. Rejected: people who apply for hearing aids differ from those who do not in hearing, motivation and perhaps loneliness itself.

Design 3, historical comparison with last year's participants. Low cost. Rejected: last year's group was recruited differently and measured differently, and the senior center reopened in between.

Design 4, regression discontinuity at the income cutoff. Applicants just under 200 percent of the poverty line would be compared with those just over it, who are not eligible. Rejected: too few applicants fall near the cutoff, and those just over it were not assessed at all.

Design 5, staggered rollout across the county's two senior centers. Moderate cost. Rejected: with only two sites, any difference could reflect the sites themselves.

Design 6, a lottery among eligible applicants, with those not selected placed on a six-month wait-list and assessed on the same schedule. The lottery is run separately for English and Spanish speakers so both groups are balanced by language, and married couples who apply together are drawn as a unit. Moderate cost. Chosen.

Why the Lottery

When a program has more eligible applicants than places, someone will be left out regardless. A lottery decides who goes first without favoring anyone, and it balances the two groups on everything, measured or not, at the outset (Shadish et al., 2002). Oregon's 2008 Medicaid lottery offers a precedent: the state had more applicants than places and drew names, which later allowed a rigorous comparison from an allocation it had to make anyway (Finkelstein et al., 2012).

The lottery adds no scarcity to the program; it changes only how a shortage that already exists is shared, and in exchange the agency gets the one comparison that can separate the program from everything else happening to older adults that year.

How the Lottery Will Run

The lottery is held in public at the senior center in January, with the evaluator and a member of the agency's board present. Each eligible applicant receives a number. A computer program, run on a laptop facing the audience, draws numbers separately for English and Spanish speakers in proportion to their share of applicants. Couples who applied together share one number. The list of results is posted the same day, and every applicant receives a letter within a week, in their language, stating their place and the date wait-listed applicants will be served.

Applicants are told about the lottery before they apply, on the application form and in person at intake. Anyone who would rather wait for next year's first-come allocation may withdraw without penalty, and their choice is recorded along with a brief reason, which helps the agency understand how applicants view the lottery.

The Price of the Choice

The paper names four costs.

First, the order of service changes. Last year places went to the earliest applicants. Under the lottery, some early applicants will wait six months who would otherwise have been served at once. The board accepted this after discussing it, and the agency will tell every applicant how places are allocated before they apply.

Second, the comparison ends at six months. Once wait-listed applicants receive hearing aids, there is no longer a group without them. The evaluation cannot speak to effects at one or two years.

Third, the estimate is the effect of being offered the program. Some people offered places will not take them or will stop wearing the aids. The primary result therefore describes the effect of the offer, which is what the board decides to fund, rather than the effect of wearing hearing aids.

Fourth, no one can be blinded. Participants and fitters know who has hearing aids. Assessors will be separate from the program staff, but participants' answers may still reflect knowing they received help.

What Wait-Listed Applicants Receive

Wait-listed applicants are not left with nothing. They receive the agency's standard information packet on hearing loss, the drop-in hour's schedule for general questions, and a call at three months to confirm their place and answer questions about the fitting appointment they will receive. They are not fitted with aids during the six months, and the evaluation asks at the end whether they found aids elsewhere.

Sample Size

With 120 applicants offered places and 140 on the wait-list, a standard deviation of about 1.8 points on the three-item loneliness scale (Hughes et al., 2004) and adjustment for baseline scores, the design should detect a difference of about 0.6 points with 80 percent power after allowing for 15 percent loss. A smaller true difference could be missed, a limit the report will state plainly.

What the Board Asked

Two board members raised concerns. One asked whether a lottery was fair to the oldest applicants, who might not live to see the wait-list's turn. The evaluator responded that six months is the same wait the shortage imposes now, and the board added a rule: applicants over 90 are placed in the first group outside the lottery and excluded from the analysis. The other asked whether the evaluation could be cheaper. The answer was that the lottery itself costs nothing; the cost is in the two assessments, which any credible design would need.

Conclusion

The lottery is the strongest design available, because the shortage of places allows a fair random allocation. Its costs are real but bounded, and each is stated in the evaluation plan so that the board, the applicants and the funder know what the evaluation can and cannot show.

References

Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., & Oregon Health Study Group. (2012). The Oregon Health Insurance Experiment: Evidence from the first year. Quarterly Journal of Economics, 127(3), 1057-1106. https://doi.org/10.1093/qje/qjs020

Hughes, M. E., Waite, L. J., Hawkley, L. C., & Cacioppo, J. T. (2004). A short scale for measuring loneliness in large surveys: Results from two population-based studies. Research on Aging, 26(6), 655-672. https://doi.org/10.1177/0164027504268574

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.