HD460 Unit 7 Process Evaluation Example

Ninety-Two Percent Delivered, but to Whom? A Unit 7 Process Evaluation of a Home-Based Late-Life Depression Program

[Student Name]

Purdue University Global

HD460: Health Education Program Implementation and Evaluation

Unit 7 Process Evaluation

[Instructor Name]

[Date]

The program, its participants and every figure are composite; the process evaluation guide, the RE-AIM framework, the fidelity framework and the home-based trial cited are real.

Purpose and Questions

The evaluation asks five questions drawn from a standard process evaluation guide: did the program reach the intended population, was it delivered as planned, how much did participants receive, how were they recruited, and what got in the way (Saunders et al., 2005). The RE-AIM framework adds a focus on reach and representativeness, asking not only how many took part but whether they resembled the people the program was meant for (Glasgow et al., 1999).

Methods

Data came from the referral log, the visit checklist, the supervisor reviews, the waiting list and short interviews with eight participants and both counselors. The comparison population for reach is the east side's adults aged 60 and older from census estimates.

Recruitment and Reach

Referrals: [64], of whom [52] came from the home-delivered meals program, [9] from the two senior centers and [3] from screening at a church health fair. Assessed as eligible: [49]. Enrolled: [41], or 84 percent of those eligible. Declined: [8], most saying they did not want "a counselor" in their home.

Of the [41] enrolled, [76] percent were women, compared with [58] percent of the east side's older adults. Spanish speakers were [7] percent of participants and about [22] percent of the area's older adults. Participants were older than average, with a median age of [79], reflecting the meals program's clients.

Delivery

Scheduled visits: [328], eight for each enrolled participant. Delivered: [301], or 92 percent. Delivered by phone after the week-six change: [58], or 19 percent of all visits. Weekly case reviews held: [49] of [52] weeks.

Dose Received

Median visits per participant: 7. Participants receiving at least six visits: [33] of [41], or 80 percent. Five participants left early: two moved to assisted living, one was hospitalized, two stopped answering.

Fidelity

Average visit checklist score: 17.6 out of 20, or 88 percent. Supervisor ratings on reviewed visits averaged 16.4, slightly lower than counselors' self-ratings, a gap mainly in the activity planning element. Fidelity frameworks treat content, dose and quality as separate dimensions (Carroll et al., 2007), and here content and dose were high while the quality of activity planning varied.

The program reached the people the meals program already served, older women living alone, very well, and reached men and Spanish speakers at about half their share of the community, which no delivery figure would reveal.

What Got in the Way

Counselors described three barriers. The caseload exceeded the plan by week six, leading to the waiting list and phone visits. Men referred by meal drivers were more likely to decline, often saying they did not need help. And the program had only one Spanish-speaking counselor, who left in month four; the replacement does not speak Spanish.

Participants interviewed valued the home visits and the counselors' patience. Two said phone visits felt "less real." One said she had started going to the senior center lunch for the first time in a year.

Interpretation

The program was delivered close to plan, with most visits completed, most participants receiving a full dose and fidelity above the threshold. Its main weakness is reach. Because almost all referrals came through one partner, the program served that partner's clients. The trial behind the model enrolled clients of community aging services in a similar way (Ciechanowski et al., 2004), so this pattern is common, but it is not inevitable.

Recommendations

1. Recruit through partners that see older men, such as veterans' organizations, barbershops and senior housing maintenance staff.

2. Hire or contract a Spanish-speaking counselor and translate outreach materials.

3. Keep tracking phone visits separately so the outcome evaluation can compare them.

4. Add a short quality rating for activity planning to the supervisor review.

Limits

The interviews were few and with participants who stayed. Census comparisons are approximate. Self-rated fidelity may overstate delivery.

References

Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40. https://doi.org/10.1186/1748-5908-2-40

Ciechanowski, P., Wagner, E., Schmaling, K., Schwartz, S., Williams, B., Diehr, P., Kulzer, J., Gray, S., Collier, C., & LoGerfo, J. (2004). Community-integrated home-based depression treatment in older adults: A randomized controlled trial. JAMA, 291(13), 1569-1577. https://doi.org/10.1001/jama.291.13.1569

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322-1327. https://doi.org/10.2105/AJPH.89.9.1322

Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6(2), 134-147. https://doi.org/10.1177/1524839904273387