GB793 Unit 4 Empirical Article Appraisal Example

Meeting the People the Calls Helped: A Unit 4 Appraisal of a Field Experiment on Beneficiary Contact

[Student Name]

Purdue University Global

GB793: Advanced Topics in Management

Unit 4 Empirical Article Appraisal

[Instructor Name]

[Date]

This appraisal concerns a real published study; its proposed rivals and tests are the candidate's.

The Article and Its Claim

The study under appraisal is Grant et al. (2007), published in Organizational Behavior and Human Decision Processes. Its claim, in the authors' terms, is that contact with the beneficiaries of one's work increases persistence, because contact makes the impact of the work vivid. In the field experiment, callers who solicited donations for a university scholarship fund were randomly assigned to one of three conditions. In one, they met briefly with a scholarship recipient who described how the scholarship had helped him. In a second, they read a letter from a recipient but did not meet him. In a control condition, they had no contact. A month later, callers who had met the recipient spent substantially more time on the phone and raised substantially more money than callers in the other conditions, whose performance did not change.

Section One: The Design's Strengths

The study has three strengths that much research on motivation lacks, and an appraisal should state them plainly.

Random assignment in a working organization. Callers were assigned to conditions by the researchers, not by choice or by managers. That rules out the most common problem in field research on motivation, which is that motivated people seek out the experiences researchers then credit with motivating them.

Behavioral outcomes from records. Persistence and money raised came from the call center's own records, not from callers' reports. Self-reports of effort are easily shaped by what participants think researchers want to hear. Records of minutes on the phone and dollars pledged are not.

A delay before measurement. Performance was measured a month after the contact, not immediately. That makes it unlikely the effect reflects a momentary mood from meeting the student.

Section Two: Four Limits

The sample was small. The experiment involved a small number of callers at a single call center. Small samples produce imprecise estimates, and an effect estimated from a few dozen people may be larger than the true effect, since only large effects reach significance in such samples. The direction of the result is credible; its size is less certain.

One setting. The callers were students working part-time in a university fundraising operation, raising money for scholarships that went to students like themselves. The similarity between callers and beneficiary may have made the contact unusually powerful. Whether the effect would appear among, for example, insurance claims staff meeting a customer is a separate question.

The mechanism was not measured in the field. The field experiment showed that contact changed behavior. The claim that it did so by increasing perceived impact rests largely on the article's laboratory study, where the mechanism could be measured. The field result is consistent with that mechanism but does not establish it.

The comparison conditions differ in more than one way. The contact condition differed from the letter condition in the presence of a person, the length of the experience and the attention it involved. Any of these could account for the difference, a problem of construct validity in the treatment itself rather than of internal validity (Shadish et al., 2002).

Section Three: The Authors' Mechanism Against Three Rivals

The authors' mechanism is perceived impact: meeting the beneficiary made callers see that their work mattered. It extends the task significance dimension of job design that Hackman and Oldham (1976) proposed, the degree to which a job affects other people's lives. Three rivals could produce the same field result.

Attention from researchers. Callers in the contact condition took part in a visible event organized by researchers, which may have signaled that their work was being watched. Being observed can raise effort without any change in perceived impact.

Felt obligation to a visitor. Having met a specific person who thanked them, callers may have felt a personal obligation to that individual, a social debt rather than a sense of impact. The two would predict similar behavior at first but differ in how long the effect lasts and whether it transfers to calls for other causes.

Relief from routine. Call center work is repetitive. A break from routine to meet a visitor may have refreshed callers in a way that any interesting break would. This rival predicts that a meeting with someone unconnected to the work would produce a similar effect.

The field experiment shows that meeting a beneficiary changes behavior; it does not by itself show why, and three plausible reasons remain open.

Separating the Rivals

Each rival can be separated from perceived impact by a condition the original study did not include. Attention could be tested with a condition in which researchers hold an equally visible meeting with no beneficiary present. Obligation could be tested by measuring persistence on calls for a cause unrelated to the visitor; perceived impact predicts the effect would carry over to similar work, while obligation to one person predicts it would not. Relief from routine could be tested with a break of equal length featuring an unrelated speaker. The laboratory study in the article addressed part of this by measuring perceived impact directly, which strengthens the authors' case, but a field replication with these conditions would make it much stronger.

What a Replication Would Need

A replication that could settle the mechanism would keep the original design's strengths and add what it lacked. It would assign several call centers, or several teams within a large one, rather than individuals at one site, so that the estimate rested on more than a few dozen people. It would include the attention, unrelated-visitor and unrelated-cause conditions described above. It would measure perceived impact in the field, with a short survey after the contact and again at the end of the month, so that changes in the proposed mechanism could be linked to changes in behavior. And it would follow callers for longer than a month, since an effect that fades quickly suggests a different mechanism from one that lasts.

What the Evidence Establishes

The study establishes, with a strong design, that brief contact with a beneficiary increased fundraising callers' persistence and results in one organization over a month. That is a valuable finding, and the use of random assignment and behavioral records makes it more credible than most evidence on job design. The claim that perceived impact is the mechanism is plausible and supported by laboratory evidence, but the field experiment alone leaves attention, obligation and relief from routine as live alternatives. The effect's size and its generality beyond a setting where callers and beneficiaries were so alike remain open.

References

Grant, A. M., Campbell, E. M., Chen, G., Cottone, K., Lapedis, D., & Lee, K. (2007). Impact and the art of motivation maintenance: The effects of contact with beneficiaries on persistence behavior. Organizational Behavior and Human Decision Processes, 103(1), 53-67. https://doi.org/10.1016/j.obhdp.2006.05.004

Hackman, J. R., & Oldham, G. R. (1976). Motivation through the design of work: Test of a theory. Organizational Behavior and Human Performance, 16(2), 250-279. https://doi.org/10.1016/0030-5073(76)90016-7

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.