NU803 Unit 8 Statistical Analysis Write-Up Example

A Small p-Value That Probably Overstates Itself: A Statistical Analysis Write-Up Comparing Hyperglycemic Monitored Days Before and After a Steroid Hyperglycemia Bundle, With the Clustering the Test Ignores

[Student Name]

Purdue University Global

NU803: Change Management and Clinical Data Analysis

Unit 8 Statistical Analysis Write-Up

[Instructor Name]

[Date]

The units and all figures are composite; numbers are bracketed and computed consistently. Design effects are illustrative.

The Question and the Data

The question for this write-up is whether the proportion of monitored steroid days with a reading above 250 mg/dL differed between the [16]-week baseline and the [12]-week implementation period. Baseline data included [612] monitored days, of which [190] were hyperglycemic, [31.0] per 100. Implementation data included [914] monitored days, of which [166] were hyperglycemic, [18.2] per 100. The run chart had already shown a shift from implementation week three; the statistical test asks a different question, whether the difference in pooled proportions is larger than chance alone would easily produce.

The Test

Testing the two-by-two table of period against hyperglycemic day gave a chi-square of [34.0] with one degree of freedom, p < .001. The absolute difference was [12.9] hyperglycemic days per 100 monitored days, with a 95 percent confidence interval of [8.4] to [17.3]. The relative risk was [0.59], with an interval of [0.49] to [0.70], meaning that a monitored day during implementation was about [41] percent less likely to include a reading above 250 than a monitored day at baseline.

Checking the Numbers by Hand

Because the result will be presented to people who do not run statistics, the key figures were checked by hand as well as by software. The baseline proportion is [190] divided by [612], or [0.310]; the implementation proportion is [166] divided by [914], or [0.182]. Their difference is [0.129], or [12.9] per 100. The standard error of that difference, from the usual formula for two independent proportions, is about [0.023], and adding and subtracting [1.96] times that value gives the interval of [8.4] to [17.3]. The relative risk is [0.182] divided by [0.310], or [0.59]. Showing these steps lets a nurse manager see where each number comes from and why the clustering correction below changes the interval but not the point estimate.

The Assumption the Test Makes

Like most simple tests of proportions, chi-square treats every observation as independent of every other. Monitored days are not. A patient on steroids for five days contributes five monitored days, and a patient whose glucose runs high on one day is likely to run high the next. Observations within the same patient are more alike than observations from different patients, which is what statisticians call clustering. The degree of similarity within clusters is measured by the intracluster correlation coefficient, and ignoring it makes standard errors too small and p-values too small (Kerry & Bland, 1998).

The test counted [1,526] days as if they came from [1,526] people, when they came from far fewer, and every day a patient stays high adds evidence that is not independent.

The effect of clustering can be estimated with the design effect, calculated as one plus the average cluster size minus one, multiplied by the intracluster correlation. In these data the average number of monitored days per patient was about [2.8], and, for illustration, an intracluster correlation of [0.5], plausible for a patient's glucose on consecutive days, gives a design effect of about [1.9]. Dividing the effective sample size by that amount widens the confidence interval for the absolute difference to roughly [6.8] to [19.0] and reduces the chi-square to about [18], still with p < .001.

What Survives

Even after a generous allowance for clustering, the difference remains large and unlikely to reflect chance alone, and that is what the committee most needs to hear. What does not survive is the precision implied by the uncorrected analysis. The uncorrected interval suggests the true reduction is between about [8] and [17] per 100; the corrected one says between about [7] and [19]. The honest summary is that hyperglycemic monitored days fell substantially, by somewhere around a third to a half in relative terms, with the exact size uncertain.

Better Analyses the Project Could Not Run

Two analyses would handle clustering directly. A patient-level analysis could compare the proportion of patients with any hyperglycemic monitored day in each period, though it would lose information about how many days each patient spent high. A model that accounts for repeated measurements within patients, such as generalized estimating equations or a mixed-effects logistic model, would use all the days while adjusting for their correlation. Neither was available within the project's software and skills, and both are recommended for any publication of the results. Guidance on appraising quality improvement studies emphasizes that analyses should account for the structure of the data, including clustering, and that pre-post designs remain vulnerable to secular trends regardless of statistical method (Fan et al., 2010).

Why Both Analyses Were Run

Some improvement teams run only a run chart and others only a test. This project ran both because they answer different questions for different audiences. The unit staff, who look at the chart every week, need to know whether their work is changing results now. The operations committee, which will decide whether to fund the diabetes resource nurse role, asked for an estimate of the size of the change with some measure of confidence. Running both, and reporting where they agree, gives each audience what it needs without pretending that either analysis settles the question of cause.

What the Test Cannot Say

The test compares two periods. Causation lies beyond what it can show, because the design has no concurrent control, and it cannot show when the change began. The run chart answers the second question, showing a shift from week three. Taken together, the two analyses give a clearer picture than either alone: the chart shows when and whether change occurred, and the test gives an estimate of its size with appropriate caution.

The Balancing Measure

Low-glucose days were [16] of [1,440] steroid days at baseline, [1.1] per 100, and [13] of [1,080] during implementation, [1.2] per 100. The difference is small and the counts too few for a meaningful test; the review threshold set in the evaluation plan was never reached.

Reporting

The report to the operations committee will present the rates, the absolute and relative differences with both uncorrected and clustering-adjusted intervals, and a plain statement of what the analysis cannot show. Run chart rules and their results will be reported alongside (Perla et al., 2011).

References

Fan, E., Laupacis, A., Pronovost, P. J., Guyatt, G. H., & Needham, D. M. (2010). How to use an article about quality improvement. JAMA, 304(20), 2279-2287. https://doi.org/10.1001/jama.2010.1692

Kerry, S. M., & Bland, J. M. (1998). The intracluster correlation coefficient in cluster randomisation. BMJ, 316(7142), 1455. https://doi.org/10.1136/bmj.316.7142.1455

Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety, 20(1), 46-51. https://doi.org/10.1136/bmjqs.2009.037895