HS305 Unit 9 Quasi-Experimental Article Critique Example

Did the Signs Do It? A Unit 9 Article Critique of a One-Hospital Before-and-After Study of Stair Prompts at the Elevator Banks

[Student Name]

Purdue University Global

HS305: Research Methods for Health Sciences

Unit 9 Article Critique

[Instructor Name]

[Date]

The hospital study critiqued here is a composite written for this sample; the appraisal checklist, the review of stair prompts and the methods papers cited are real.

The Study in Brief

The article reports a project at a [420]-bed teaching hospital. Infrared counters were fixed above the doors of four stairwells and at the entrance to each elevator lobby on the ground floor. Counts were taken for two weeks in February. In March, the hospital placed printed signs beside every elevator call button reading "Take the stairs: two floors up, three floors down, better for your heart." Counts were taken again for two weeks in April. Of [138,600] counted trips in February, [11.2] percent were by stairs; of [141,300] in April, [16.8] percent were. A chi-square test gave p < .001. The authors conclude that point-of-decision signs increased stair use and recommend them for all hospitals.

Why a Checklist

Before-and-after studies without a separate comparison group are quasi-experiments: the investigators control when the change happens but not who gets it. The JBI critical appraisal checklist for quasi-experimental studies asks nine questions about whether the cause is clear, whether the groups being compared are alike, whether there is a control, whether outcomes were measured more than once and the same way, and whether the analysis fits (Tufanaru et al., 2020). I used it because it is short, it was built for this design, and each item names a specific way a before-and-after result can mislead. The answer to each item below is yes, no or unclear, with the reason.

Item by Item

Does the presumed cause clearly come before the effect? Yes. The signs went up between the two counting periods, so the timing is right for the signs to have caused a change.

Were the participants in the before and after periods similar? Unclear. The counters record trips, not people. February and April may have had different mixes of staff, patients and visitors, and the article gives no census, visitor or staffing figures for either period. A teaching hospital also changes much of its trainee workforce on fixed dates, and the article does not say whether a rotation fell between the periods.

Were the before and after periods alike apart from the signs? No. The discussion mentions, in one sentence, that one of the three main elevators was out of service for six days in April for an inspection. A missing elevator means longer waits, and longer waits push people toward stairs. Season may matter too: the authors note a mild April but do not test whether weather changes trips between buildings.

The elevator outage is the single fact most likely to explain the result without the signs, and the article reports it in the discussion instead of removing those six days from the count.

Was there a control group? No. The hospital has a second campus with the same elevator layout that did not get signs. Counting there over the same weeks would have shown how much stair use changes from February to April without any signs at all.

Were there multiple measurements before and after? No. There is one two-week period on each side. A single before measurement cannot show whether stair use was already rising, and a single after measurement cannot show whether the change lasted once the signs became familiar.

Was follow-up complete? Not applicable in the usual sense, since there is no cohort to lose. But the article reports that counters on one stairwell failed for two days in February, and those days were left out, which slightly changes the denominator between periods.

Were outcomes measured in the same way? Yes, by the same counters in the same places.

Were outcomes measured reliably? Unclear. Infrared counters miss people walking side by side and double count people who pause in a doorway. The authors did not compare the counters with a manual count, so the size of the error, and whether it differed between busy and quiet weeks, is unknown.

Was the analysis appropriate? No. A chi-square test treats every trip as independent, but a nurse who takes the stairs twelve times a shift contributes twelve linked observations. With over a hundred thousand trips, almost any difference will reach significance, so the p value says little about whether the change is real or meaningful.

What the Wider Evidence Says

The signs are not implausible. The Community Preventive Services Task Force's updated systematic review found that point-of-decision prompts placed where people choose between stairs and elevators or escalators increase stair use, with a median relative increase that was modest and with most included studies using designs like this one (Soler et al., 2010). That evidence makes it reasonable to expect some effect. It does not tell us whether this hospital's increase came from the signs, because the review's own studies share the same weaknesses.

Designs That Would Have Answered the Question

Two changes would have strengthened the study at little cost. The first is a comparison site: counting the second campus over the same weeks. The second is more time points. Weekly counts for twelve weeks before and twelve after would allow an interrupted time series, which estimates the trend before the change and tests whether the level or slope shifted when it happened (Kontopantelis et al., 2015). That design handles a gradual rise that was already under way and shows whether the effect fades. Methods papers on quasi-experimental studies describe a ladder of designs from a single before-and-after comparison, to one with a nonequivalent control, to repeated measures with a removed or reintroduced intervention (Harris et al., 2006). This study sits on the lowest rung.

Strengths

The study has real strengths. It measured behavior directly rather than asking people whether they used the stairs, which avoids the overreporting that self-reported activity tends to produce. The intervention is cheap and clearly described, so another site could copy it exactly. The counters were in place before the signs went up, so the same instrument was used in both periods. The authors also reported the elevator outage at all, which allowed a reader to spot it.

The Sentence the Evidence Supports

The authors wrote that the signs increased stair use. The evidence supports something narrower: "In two April weeks after signs were placed, the share of counted trips by stairs was higher than in two February weeks before; an elevator outage during the second period and the absence of a comparison site mean the rise cannot be attributed to the signs." Their recommendation that all hospitals adopt the signs goes further still, since a single site's result says nothing about how a different building, staff or season would respond.

What This Critique Taught Me

Before this unit I would have read the p value and the percentages and accepted the conclusion. Working through the checklist made me look for the event that happened at the same time as the intervention, and in this article it was sitting in the discussion. For my own sit-stand desk question, it confirms why random assignment matters: without it, any change at the employer, a new manager, a move to a new floor or a wellness campaign, could take credit or blame. When I read a before-and-after study now, I look first for what else changed between the two measurements and whether anyone counted a place that did not get the intervention.

References

Harris, A. D., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J. (2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal of the American Medical Informatics Association, 13(1), 16-23. https://doi.org/10.1197/jamia.M1749

Kontopantelis, E., Doran, T., Springate, D. A., Buchan, I., & Reeves, D. (2015). Regression based quasi-experimental approach when randomisation is not an option: Interrupted time series analysis. BMJ, 350, h2750. https://doi.org/10.1136/bmj.h2750

Soler, R. E., Leeks, K. D., Buchanan, L. R., Brownson, R. C., Heath, G. W., Hopkins, D. H., & Task Force on Community Preventive Services. (2010). Point-of-decision prompts to increase stair use: A systematic review update. American Journal of Preventive Medicine, 38(2 Suppl), S292-S300. https://doi.org/10.1016/j.amepre.2009.10.028

Tufanaru, C., Munn, Z., Aromataris, E., Campbell, J., & Hopp, L. (2020). Systematic reviews of effectiveness. In E. Aromataris & Z. Munn (Eds.), JBI manual for evidence synthesis (Chapter 3). JBI. https://synthesismanual.jbi.global