GB794 Unit 5 Measurement Validity Review Example

A Score That Counts Taps, Not Loyalty: A Unit 5 Measurement Validity Review of a Grocer's Engagement Index

[Student Name]

Purdue University Global

GB794: Advanced Topics in Marketing

Unit 5 Measurement Validity Review

[Instructor Name]

[Date]

Thornbury Market, its vendor score and every figure are composite and illustrative; the measurement frameworks cited are real.

The Score as Documented

The vendor's documentation describes the score's recipe. It combines four inputs for each loyalty member: the number of times the member opened Thornbury's app in the past month, the member's points balance, the number of store or online visits in the past month and the number of email clicks. Each input is standardized across members, and the four are weighted 30, 30, 25 and 15 percent and rescaled to run from 0 to 100. The board report shows the average across active members, which rose from 54 to 61 over the past year.

The Construct the Score Claims to Measure

Churchill (1979) proposed that developing a measure should begin by specifying the domain of the construct: what it includes and what it excludes. Only then can items be chosen and tested. The vendor calls its score an engagement score, but the board report presents it as a measure of loyalty, so the review asks whether it measures loyalty.

Dick and Basu (1994) defined loyalty as the combination of two things: a relative attitude, meaning a favorable view of the store compared with alternatives, and repeat patronage. Both are needed. A customer who shops often because a store is nearest but prefers a competitor has what they called spurious loyalty; a customer who prefers the store but rarely visits has latent loyalty. True loyalty requires both the attitude and the behavior.

Which Parts of the Definition the Inputs Touch

Of the four inputs, one, visits per month, measures repeat patronage directly. App opens and email clicks measure engagement with Thornbury's digital channels, which may accompany loyalty but are not part of it: a member might open the app often to check prices before buying elsewhere. The points balance measures accumulated spending minus redemptions, so a high balance can mean heavy past spending or simply that a member has not redeemed. None of the four inputs reaches relative attitude. The score has no way to distinguish a member who prefers Thornbury from one who shops there because it is convenient.

How the Inputs Behave Together

Across 38,000 active member records, the four inputs are only weakly related. App opens and points balance, which together make up 60 percent of the score's weight, correlate at 0.12. Visits correlate more strongly with points balance, as expected, since both reflect spending, but weakly with app opens.

This pattern matters because of what kind of measure the score is. Jarvis et al. (2003) distinguished reflective measures, in which indicators are effects of an underlying construct and should therefore correlate with one another, from formative measures, in which indicators combine to form a composite and need not correlate. The vendor's score is formative: its inputs form it rather than reflect it. A formative score is only as meaningful as the choice of its inputs and weights, because there is no underlying construct whose presence the inputs confirm. Jarvis and colleagues found that formative measures were often treated as if they were reflective, which produces misleading conclusions about validity. Here, the low correlations are not a flaw in themselves, but they mean the score's meaning rests entirely on whether the inputs and weights capture loyalty, which the domain analysis shows they do not.

The Survey Test

Last spring, Thornbury surveyed 1,150 loyalty members, asking which grocer they would choose if all were equally convenient, how much of their monthly grocery spending went to Thornbury and how likely they were to recommend it. These questions measure the attitude component of loyalty and an estimate of share of wallet, the behavioral component relative to competitors. The correlation between members' engagement scores and an index of their survey responses was 0.22. A measure of loyalty should correlate far more strongly with direct measures of loyalty than that.

A score whose two largest inputs barely move together, and which tracks members' stated loyalty only faintly, is measuring something other than loyalty.

What the Rising Average Means

The board has read the rise from 54 to 61 as rising loyalty. The review suggests a more likely explanation. Over the same year, Thornbury redesigned its app and added weekly digital coupons, both of which raise app opens. Points balances also rose because a points expiration rule was suspended. Neither change implies members prefer Thornbury more or give it a larger share of their spending.

Applying Churchill's Later Steps

Churchill's paradigm continues beyond the domain. After specifying the construct, a developer generates candidate items, collects data, purifies the measure by dropping items that do not hang together, and then assesses reliability and validity, including whether the measure correlates with other measures it should relate to and fails to correlate with measures it should not. The vendor's documentation describes none of these steps. The weights of 30, 30, 25 and 15 percent were set by the vendor's product team, according to the documentation, to reflect the relative importance of each behavior to the vendor's own business model, which sells app engagement tools. No evidence is offered that the weights reflect anything about loyalty. Under Churchill's paradigm, the score has skipped every step after item selection, and its items were not chosen with the construct in view.

A further check Churchill recommended is whether a measure distinguishes groups it should distinguish. Thornbury's survey identified members who said they would choose Thornbury over every competitor even if all were equally convenient, and members who said they would choose a competitor. If the score measured loyalty, the first group should score much higher. In the survey sample, the two groups' average scores differed by four points on the hundred-point scale, a difference smaller than the month-to-month movement the board reads as meaningful.

A Replacement in Two Parts

The review recommends replacing the single score with two measures that match the two components of loyalty. For behavior, Thornbury should report share of wallet estimated from the survey and, where available, the share of a member's grocery spending that card and payment data suggest goes to Thornbury, along with visit frequency. For attitude, a short quarterly survey item asking members to compare Thornbury with alternatives would track relative preference. Reporting the two separately would show the board whether members are shopping more because they prefer the store or because it is convenient, a distinction the current score cannot make.

References

Churchill, G. A., Jr. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64-73. https://doi.org/10.1177/002224377901600110

Dick, A. S., & Basu, K. (1994). Customer loyalty: Toward an integrated conceptual framework. Journal of the Academy of Marketing Science, 22(2), 99-113. https://doi.org/10.1177/0092070394222001

Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199-218. https://doi.org/10.1086/376806