HS840 Unit 3 Cluster Trial Methods Appraisal Example

Selected Is Not Inserted: A Unit 3 Article Methods Appraisal of a Cluster Randomized Trial of Clinic Training for Long-Acting Contraception

[Student Name]

Purdue University Global

HS840: Advanced Health Research and Analysis Methods

Unit 3 Article Methods Appraisal

[Instructor Name]

[Date]

The trial appraised is real and its figures are reported as published; the judgments are the writer's.

The Study in One Paragraph

Harper et al. (2015) conducted a cluster randomized trial in 40 Planned Parenthood health centers across the United States from 2011 to 2013. Twenty centers were randomly assigned to receive evidence-based training for clinicians and staff on counseling about and inserting IUDs and implants; twenty provided standard care. Usual costs for contraception were maintained at all sites. Women aged 18 to 25 attending family planning or abortion visits who did not want pregnancy in the next year were recruited, 1,500 in all. The primary outcome was selection of an IUD or implant at the visit; the secondary outcome was pregnancy within twelve months. Analyses used generalized estimating equations to account for clustering. More women at intervention sites reported counseling on IUDs or implants (71 percent against 39 percent) and selected one (28 percent against 17 percent). Pregnancy was lower at intervention sites after family planning visits, 7.9 against 15.4 per 100 person-years, but not after abortion visits, 26.5 against 22.3.

Why This Tool

The trial randomized clinics, not women, so the appraisal uses the RoB 2 tool with the additional domain for cluster randomized trials, which addresses bias arising from identifying and recruiting participants after clusters are randomized (Sterne et al., 2019). Reporting is checked against the CONSORT extension for cluster trials. A tool for nonrandomized studies would understate the protection randomization provides and miss the cluster-specific risk.

Domain 1a: The Randomization Process

Forty clinics were randomized in equal numbers. Randomizing whole clinics is appropriate for a training intervention, since trained and untrained clinicians in one clinic would share colleagues, devices and scripts. The appraisal looks for how the allocation sequence was generated, whether it was concealed until clinics were committed, and whether clinics were stratified by size or region. With forty clusters, chance imbalance in clinic characteristics is possible, so a table of baseline clinic features matters. Judgment: low concern, conditional on concealment being reported.

Domain 1b: Identifying and Recruiting Participants After Randomization

This is the trial's most important methodological question. Clinics were randomized first, and women were recruited afterward by clinics that knew their allocation. Staff who had just been trained in LARC counseling may have approached women differently, and women interested in LARC may have been more likely to agree to a study at a clinic where staff discussed it.

The counseling results draw on 797 women from intervention clinics and 693 from control clinics, and an imbalance of that size after cluster randomization is a signal to examine who was recruited, not a detail to pass over.

The appraisal therefore asks whether recruitment procedures were identical across arms, who did the recruiting, and whether baseline characteristics of enrolled women, such as age, prior LARC use and visit type, were balanced. If intervention clinics recruited more women already interested in LARC, part of the difference in selection would reflect recruitment, not training. Judgment: some concerns.

Domain 2: Deviations From the Intended Intervention

Blinding clinic staff was impossible, since the intervention was their training. That is acceptable for estimating the effect of assignment to training. The appraisal asks whether control clinics might have changed practice on their own during the study years, a period when professional societies were promoting LARC widely, which would shrink the measured difference between arms. Judgment: low to some concerns, and any bias here would likely make the reported effect conservative.

Domain 3: Missing Outcome Data

LARC selection was recorded at the index visit, so missing data are minimal for the primary outcome. Pregnancy within twelve months required follow-up, and loss to follow-up in young adults over a year is typically substantial. The appraisal looks for retention by arm and for an analysis that handles missing follow-up under reasonable assumptions. Judgment: low concern for the primary outcome, some concerns for pregnancy.

Domain 4: Measurement of the Outcome

Selection was recorded at visits by clinics that knew their allocation, and counseling was reported by participants. Both could be influenced by knowledge of the intervention. Selection is also not the same as insertion. A woman who selects an IUD but must return for a second visit may never receive it, which is precisely the gap two-visit protocols create in other settings (Bergin et al., 2012). The trial's primary outcome cannot tell whether training increased the number of devices actually placed. Judgment: some concerns.

Domain 5: Selection of the Reported Result

The pregnancy effect was significant after family planning visits and absent after abortion visits. The appraisal checks whether this division by visit type was prespecified in a registered protocol. If it was planned, the contrast is an informative finding about different clinical contexts; if it was not, the family planning result should be read as exploratory. Judgment: some concerns until the registration is checked.

Reporting Against the CONSORT Cluster Extension

The extension asks trials to report the rationale for cluster randomization, how clustering was handled in sample size and analysis, intracluster correlation coefficients, the flow of clusters and individuals through the trial, and baseline characteristics at both levels. The trial's use of generalized estimating equations addresses clustering in the analysis. The appraisal records whether the intracluster correlation for LARC selection was reported, since later studies, including the one this course is building toward, need it to estimate sample size.

Overall Verdict

Supported. Training clinic staff in LARC counseling and insertion, delivered to whole clinics, increased the share of young women who reported LARC counseling and who selected an IUD or implant at the visit. Randomization at the clinic level protects this claim from most confounding, and the effect is large enough to survive moderate recruitment bias.

Supported with caution. The reduction in pregnancy after family planning visits, a secondary outcome with follow-up losses and a subgroup contrast.

Not supported. Any claim about insertion, as distinct from selection, and any claim about general primary care. The trial took place in specialized reproductive health centers, where staff and patients differ from those in a primary care network, and it kept usual costs in place, so it cannot show what happens when cost barriers are removed.

What This Means for the Planned Study

Three lessons carry forward. The planned study should measure insertion within a fixed window, not selection. It should identify participants without relying on clinic staff who know the allocation, which favors an outcome drawn from records for all requesting patients. And it should report its intracluster correlation so the next study can use it.

References

Bergin, A., Tristan, S., Terplan, M., Gilliam, M. L., & Whitaker, A. K. (2012). A missed opportunity for care: Two-visit IUD insertion protocols inhibit placement. Contraception, 86(6), 694-697. https://doi.org/10.1016/j.contraception.2012.05.011

Harper, C. C., Rocca, C. H., Thompson, K. M., Morfesis, J., Goodman, S., Darney, P. D., Westhoff, C. L., & Speidel, J. J. (2015). Reductions in pregnancy rates in the USA with long-acting reversible contraception: A cluster randomised trial. The Lancet, 386(9993), 562-568. https://doi.org/10.1016/S0140-6736(14)62460-0

Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., Emberson, J. R., Hernán, M. A., Hopewell, S., Hróbjartsson, A., Junqueira, D. R., Jüni, P., Kirkham, J. J., Lasserson, T., Li, T., ... Higgins, J. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ, 366, l4898. https://doi.org/10.1136/bmj.l4898