GB794 Unit 6 Seminar Reflection Example

The Same Data, the Opposite Answer: A Unit 6 Seminar Reflection on Assortment Evidence

[Student Name]

Purdue University Global

GB794: Advanced Topics in Marketing

Unit 6 Seminar Reflection

[Instructor Name]

[Date]

The classmates and Thornbury Market are composite; the studies discussed are real.

What I Said

Early in the session I said, more confidently than my own Unit 2 critique justified, that "the evidence shows assortment cuts raise sales, by about 11 percent on average." I was quoting Boatwright and Nunes (2001), whose study of an online grocer's large reduction in low-selling items found category sales rose across most of the categories they examined. In my head, the Unit 2 caveats had faded and the headline had stayed. I had even repeated the 11 percent figure to our merchandising team the week before, as if it were a forecast for our bar aisle rather than an average across forty-two categories at one online grocer two decades ago.

The Challenge

The reply came from a classmate with years in retail analytics, who asked whether I knew that the same reduction at the same grocer had been studied again. Borle et al. (2005) followed every household's purchases across the whole store, so their model saw total spending as well as category sales. They found that the assortment reduction lowered overall store sales, because households shopped less often and spent less in total, even though sales within many cut categories rose. Their article states plainly that this result contrasts with earlier studies, including the one I had quoted.

What struck me was that neither study was careless. Both used the grocer's own records of real purchases. They reached opposite conclusions because they asked different questions of the same event: one asked what happened inside the categories that were cut, the other what happened to customers' relationship with the store.

Two careful studies of one assortment cut disagreed because one counted sales in the aisle and the other counted customers at the door.

The Wider Problem the Instructor Raised

Our instructor used the exchange to raise a broader point. If a single event can be measured in ways that point in opposite directions, then a researcher who looks at many outcomes after the fact can usually find one that supports the conclusion they hoped for. Simmons et al. (2011) demonstrated how this happens: by making a handful of reasonable analytic choices after seeing data, such as which outcome to report, when to stop collecting observations and which covariates to include, researchers can make an effect that is not there look statistically real. They recommended that researchers commit to their choices in advance and report every outcome they measured.

The connection to my own work was uncomfortable. My Unit 3 design for Thornbury's bar test named one primary outcome, category sales, and two secondary ones. If category sales rose while basket value fell, which outcome would I report to the vice president as the result? I had not decided. That left exactly the kind of flexibility Simmons and colleagues warned about.

What I Now Believe

I no longer believe the evidence shows that assortment cuts raise sales. It shows that cuts of low-selling items can raise sales within a category, and that the same cuts can reduce a store's total sales when some customers lose items they care about. Which effect dominates depends on which items are cut, how many categories are cut at once and how attached customers are to what disappears. My concession stops there: I still think a targeted cut of near-duplicate items in one category, with shelf space held constant, is likely to do little harm, and the bar test is designed to find out.

What Changed in the Bar Test

I made two changes to the Unit 3 design, and I will register both publicly before launch. First, the decision rule now names total basket value per loyalty household, not category sales, as the outcome that determines whether the cut goes chain-wide, because that is the outcome the store-level study showed could move against the category result. Category sales remain reported but cannot by themselves justify the cut. Second, the plan names shopper type as a planned comparison: households that bought bars in at least six of the last twelve months will be analyzed separately from occasional buyers, since the customers most likely to leave over a missing item are the regular buyers of that item. Both changes are written into the registration so that I cannot choose between outcomes after I see them.

Looking back, my error in the seminar was not citing a weak study. It was citing a strong study for a claim it did not make. Boatwright and Nunes measured category sales and said so; I turned their category finding into a claim about stores, which is exactly the claim the second study tested and found wanting.

References

Boatwright, P., & Nunes, J. C. (2001). Reducing assortment: An attribute-based approach. Journal of Marketing, 65(3), 50-63. https://doi.org/10.1509/jmkg.65.3.50.18330

Borle, S., Boatwright, P., Kadane, J. B., Nunes, J. C., & Shmueli, G. (2005). The effect of product assortment changes on customer retention. Marketing Science, 24(4), 616-622. https://doi.org/10.1287/mksc.1050.0121

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366. https://doi.org/10.1177/0956797611417632