HI595 Unit 9 Evaluation Plan Example

The Same Definition, Measured Again: A Unit 9 Capstone Evaluation Plan With One Primary, Three Process and Three Balancing Measures

[Student Name]

Purdue University Global

HI595: Master's Capstone in Health Informatics

Unit 9 Evaluation Plan

[Instructor Name]

[Date]

Carrowby Health, its baseline, measures and every target are composite; the sources cited are real.

Primary Measure

The definition is carried over from the Unit 2 proposal without alteration, and a reader can check it there: of eligible exams, the proportion whose device maker, model and MRI labeling already sit in coded fields of the record by the day the scan was booked for.

Baseline: 51 of 150 consecutive eligible exams, 34 percent, 95 percent confidence interval 27 to 42 percent, from the Unit 6 audit. Target: 80 percent. Post-period: all eligible exams in months four through nine after go-live, allowing three months for the backlog of existing patients to be worked.

Process Measures

1. Share of new implants placed at Carrowby that have a structured entry within one business day of surgery. Baseline: not applicable; the entry does not yet exist. Target: 95 percent.

2. Share of worklist exams whose status becomes Verified by the due date the specification sets. Target: 85 percent.

3. Median business days from scheduling to Verified, for outside implants. Target: five or fewer.

Balancing Measures

Each balancing measure exists to catch damage the change might do somewhere the primary measure cannot see (Provost & Murray, 2011).

1. Scheduling lead time for outpatient MRI for patients with implants, which could lengthen if verification delays booking. Alert if the median rises by more than three days.

2. Screening nurse hours per week spent on verification, from time logs. Alert if hours exceed the staffing plan by more than 20 percent for two months.

3. Exams postponed from Hold status, which could rise early as unverified cases are caught before arrival instead of at the scanner. This measure is expected to rise at first; a rise is acceptable only if day-of cancellations fall by a similar amount.

A rise in Holds is the design working, not failing, as long as the cancellations it replaces fall by a similar number.

Design

The evaluation is a before-and-after comparison using the same abstraction form and definition as the baseline audit. A monthly run chart of the primary measure will begin three months before go-live and continue for twelve months after, so that any shift can be seen against the timing of launch rather than inferred from two averages.

What the Design Cannot Rule Out

Three alternative explanations could produce a change unrelated to the specification. A new MRI scanner or schedule could change the mix of exams. Wider attention to MRI safety after a publicized incident could make staff search harder regardless of the worklist. And a change in the surgical device vendor could alter how identifiers are captured. The run chart partly addresses these: a shift that begins in the month of launch is more persuasive than one that drifts. The plan also records any such event on the chart as it happens.

Timing

The primary measure can be read within weeks of each scan, so the final reading arrives about ten months after launch, with the run chart showing direction earlier. Process measures are available monthly from the first month.

Sample Size

Thirty exams a month gives about 180 in the six-month post-period, enough to detect a rise from 34 to 80 percent with wide margin and to show a smaller rise with a usable interval.

Reporting

Each quarterly report will show the run chart, the three process measures against targets and the three balancing measures against their alert levels, followed by a short paragraph on any event that might explain a change. A one-page version goes to the sponsor; the full version goes to the safety committee.

Data Collection

The worklist itself records statuses and dates, which provides process measures automatically. The primary measure requires a monthly sample of thirty eligible exams abstracted on the baseline form by a trained reviewer, with five re-abstracted by a second reviewer each month to check agreement.

Statistical Approach

The primary measure will be compared before and after with a difference in proportions and its interval. The run chart will be read with standard rules for detecting a shift, such as six consecutive points above the baseline median. Improvement reporting guidance asks projects to describe how they will judge whether observed change is meaningful (Ogrinc et al., 2016), and the rules are written down at this stage so the results cannot pick their own test.

Learning Between Readings

The plan treats the first months as a period of learning rather than judgment. If process measures lag, the team will test small changes, such as moving the worklist review to the start of the screening nurse's shift, and record each change on the run chart, following the cycle of planning, testing, studying and acting that improvement methods recommend (Langley et al., 2009).

Who Owns the Evaluation

The MRI safety officer owns the primary measure, the health information management director owns process measure one, and the imaging services manager owns the balancing measures. Results go to the MRI safety committee quarterly.

References

Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). Jossey-Bass.

Ogrinc, G., Davies, L., Goodman, D., Batalden, P., Davidoff, F., & Stevens, D. (2016). SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Quality & Safety, 25(12), 986-992. https://doi.org/10.1136/bmjqs-2015-004411

Provost, L. P., & Murray, S. K. (2011). The health care data guide: Learning from data for improvement. Jossey-Bass.