GB892 Unit 6 Seminar Reflection Example

One Pooled Estimate and No Ranking: A Unit 6 Seminar Reflection on What Twelve Centers Can Support

[Student Name]

Purdue University Global

GB892: Doctoral Project III: Data Analysis

Unit 6 Seminar Reflection

[Instructor Name]

[Date]

The classmates and Haverly Logistics are composite; the sources cited are real.

My Plan Going In

I brought two deliverables to the seminar that I had promised our chief operating officer. The first was a ranking: an estimate of the portal's effect at each of the six adopting centers, from largest to smallest, so that he could see where it worked best. The second was a savings figure: the minutes saved per visit, multiplied by visits and by the detention rate carriers charge per hour, giving a dollar amount the portal had saved. I had the spreadsheet half built.

The First Question: How Do Minutes Become Dollars?

A classmate who manages accounts payable at a manufacturer asked how I would turn minutes into detention dollars. I explained the multiplication. She asked whether detention is charged for every minute a truck waits. It is not. Carriers charge detention only after a free period, usually two hours, and they bill it monthly by carrier, not by visit. A truck that waited ninety minutes before the portal and eighty after saved ten minutes but no detention, because it was inside the free period both times. And because our invoices carry no visit numbers, I could not see which visits generated charges. My multiplication would assign savings to minutes that never cost anything.

The Second Question: What Would the Interval Look Like?

The instructor asked the second question. With one treated center behind each site-level estimate, and twelve centers in total, what would a confidence interval around any single site look like? I said I would report the point estimates without intervals, since the ranking was descriptive. The room disagreed. A point estimate for one center compared with six comparison centers reflects that center's particular month-to-month noise as much as any effect of the portal, and without an interval there is no way to know whether the third-ranked center differs from the fifth. Bertrand et al. (2004) showed how easily difference-in-differences designs overstate precision with few groups; a design with one treated unit per estimate is the extreme case. Cameron et al. (2008) showed that bootstrap methods can repair inference with a handful of clusters, but not with one.

A ranking of six centers, each estimated from one treated unit, would have ordered noise and presented it as performance.

What I Gave Up

The ranking went first. The study can estimate one pooled effect across the six adopting centers, with an interval computed by a wild cluster bootstrap, and that is what it will report. Site medians will appear in a descriptive table, clearly labeled as descriptions rather than estimates of effect, so that the chief operating officer can see the variation without being invited to rank it.

I also gave up the savings figure as a result. It moves to the discussion section, where it will be presented as an illustration with every assumption stated: the free period, the share of visits near the two-hour threshold and the detention rate. The share of visits over two hours, which the yard data do record, becomes a secondary outcome, since among the recorded fields it comes nearest to tracking which trucks were at risk of detention.

What I Still Maintain

I still believe the chief operating officer's question about which centers benefited most is a legitimate one, and that the interviews may help answer it in a different way. If supervisors at one center describe crews scheduled around appointments and supervisors at another describe crews unchanged, that difference is evidence about mechanism, even if it cannot rank effect sizes. Angrist and Pischke (2009) caution against asking a design for more than it can deliver, and I would rather deliver one estimate I can defend than six I cannot.

What the Exchange Taught Me About Promises

The larger lesson concerns what I had promised before checking whether the data could deliver it. Both deliverables came from conversations with the chief operating officer months earlier, before I had looked closely at how detention was billed or how many treated units each estimate would rest on. I had treated his questions as my research questions. The seminar showed me that a practitioner's question needs to be translated into one the design can answer, and that the translation should happen before the promise is made.

How I Will Explain It to the Chief Operating Officer

I will tell him that the study can say with reasonable confidence how much waits changed across the six centers together, that it cannot reliably say which center did best, and that the interviews will describe what differed between centers. He asked for a ranking because he wants to know where to invest next; the pooled estimate and the interview themes will serve that purpose better than a ranking that might reverse with another month of data.

References

Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press.

Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? The Quarterly Journal of Economics, 119(1), 249-275. https://doi.org/10.1162/003355304772839588

Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008). Bootstrap-based improvements for inference with clustered errors. The Review of Economics and Statistics, 90(3), 414-427. https://doi.org/10.1162/rest.90.3.414