GB513 Unit 7 Two-Sample t Test Comparison Example

A $2.50 Check Increase, Tested Two Ways: A Unit 7 Two-Sample Comparison of a Menu Redesign

[Student Name]

Purdue University Global

GB513: Business Analytics

Unit 7 Two-Sample Comparison

[Instructor Name]

[Date]

The restaurant group, its locations and every figure are composite and illustrative; the methods cited are real.

The Pilot and the Decision

Copper Kettle Kitchens operates 56 casual dining locations. Last quarter it redesigned its menu, moving high-margin entrees to the top right of each page, adding photographs of three dishes and removing dollar signs. Eight locations used the new menu for eight weeks while eight comparable locations kept the old one. The other forty were not part of the test. Reprinting menus and retraining servers at those forty would cost about $3,000 a location, or $120,000. The decision is whether to reprint, and the analysis below is written for the chain's chief operating officer.

How the Checks Were Sampled

For each group, 120 checks were drawn at random from all dinner checks in the eight-week pilot period, 15 from each location. Lunch checks were excluded because the redesign changed only the dinner menu, and lunch spending follows a different pattern. Checks with discounts above 20 percent, mostly staff meals and complaint resolutions, were also excluded from both groups, so that unusual pricing would not distort the comparison.

Summary Table

Pilot group: 120 randomly sampled checks. Mean $38.70; standard deviation $9.80; 95 percent interval for the mean about $36.93 to $40.47.

Control group: 120 randomly sampled checks. Mean $36.20; standard deviation $8.90; 95 percent interval about $34.59 to $37.81.

The Main Test: Two Independent Samples

Hypotheses. H0: the pilot and control mean checks are equal. H1: they differ. A two-tailed test is used, since the redesign could in principle lower checks.

Method. A two-sample t test assuming unequal variances, because the two groups' standard deviations differ and the groups are independent (Levine et al., 2021).

Results. The pilot's checks average $2.50 more. Dividing that gap by its standard error, about $1.21, gives a t statistic near 2.07 with roughly 236 degrees of freedom; the chance of a gap this large in either direction, if the menus made no difference, is about 0.040.

At the 0.05 level, the difference is statistically significant: the pilot locations' average check is higher than the control locations'.

A Second Test: Before and After at the Pilot Locations

Because pilot and control locations differ in neighborhood and traffic, a second test compares each pilot location with itself. For each of the eight pilot locations, the average check in the eight weeks before the change was subtracted from the average in the eight weeks after.

Results. Across the eight locations, checks rose an average of $2.10, with a standard deviation of $1.90. That yields a paired t statistic near 3.13 on 7 degrees of freedom and a p-value close to 0.017.

This paired design controls for differences between locations (Anderson et al., 2020), and it points the same way: checks rose after the new menu at the same restaurants.

Two different tests, one comparing locations and one comparing each location with itself, both find a higher check after the redesign.

Statistical and Practical Significance

Passing a significance test does not by itself make a gap worth acting on. This one is worth acting on. Each Copper Kettle location serves about 60,000 checks a year. An increase of about $2.10 to $2.50 per check across forty locations would add roughly $5.0 million to $6.0 million in annual revenue, against a one-time reprint cost of $120,000. Ronald Wasserstein and Nicole Lazar warn against treating a p-value as a measure of how large or valuable an effect is (Wasserstein & Lazar, 2016), which is why the revenue conversion is part of the analysis.

What Else Could Explain the Difference

Three other explanations were considered. Pilot locations might simply be busier or wealthier; the paired test, which compares each location with itself, addresses that. A seasonal rise in checks during the pilot weeks would affect both groups; the control group's average did not rise, which argues against it. And servers at pilot locations knew they were in a test and may have sold harder. That cannot be ruled out, but it would likely fade, which is why the recommendation includes another quarter of tracking.

Caveats

The pilot ran for eight weeks, so novelty may have inflated the effect. The groups were chosen as comparable, not randomly assigned. And higher checks could come partly from price changes made during the pilot; none were recorded, but the finance team should confirm.

Recommendation

Reprint the menus at the forty remaining locations, and track average check at the original pilot locations for another quarter to confirm the effect lasts. If the pilot locations' advantage shrinks below about $1.00 a check by the end of that quarter, the group should study which menu changes drove the effect before the next redesign. The reprint can be phased over two months, starting with the busiest locations, so that most of the expected revenue gain arrives early.

References

Anderson, D. R., Sweeney, D. J., Williams, T. A., Camm, J. D., & Cochran, J. J. (2020). Statistics for business and economics (14th ed.). Cengage.

Levine, D. M., Stephan, D. F., & Szabat, K. A. (2021). Statistics for managers using Microsoft Excel (9th ed.). Pearson.

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129-133. https://doi.org/10.1080/00031305.2016.1154108