PS451 Unit 6 Performance Appraisal Paper Example

Rated a Two by One Foreman and a Five by the Other: A PS451 Unit 6 Performance Appraisal Paper

[Student Name]

Purdue University Global

PS451: Selection and Assessment in Organizations

Unit 6 Performance Appraisal Paper

[Instructor Name]

[Date]

The writer, the tree care company, the climbers and every figure are composite and illustrative; the sources cited are real.

The Old Appraisal

Each spring, foremen at our Chattanooga company rated every climber on a single line: overall performance, from one to five. The rating set the raise. Foremen received the form by email with no instructions and returned it within a week.

Caleb, a climber with four years at the company, spent March and April on the utility line clearance crew and May and June on a residential crew while another climber recovered from surgery. Both foremen rated him. The line clearance foreman gave him a two, writing that he was "slow and argues about cuts." The residential foreman gave him a five, writing that he was "the most careful climber I have." Caleb's raise was set by averaging the two, which satisfied nobody and described nothing.

When I looked across all thirty-four climbers, the problem was wider. Average ratings differed by almost a full point between foremen, and two foremen gave nearly everyone a four or five.

What a Rating Actually Measures

Scullen et al. (2000) analyzed two large sets of managers, each rated by seven people: two bosses, two peers, two direct reports and themselves. They separated the sources that make up a rating: how well the person generally performs, how well they perform on the specific dimension, the rater's own rating habits, the rater's vantage point and random error. In both samples, each rater's personal habits explained over half of why ratings differed, and the manager's real performance, overall and on the specific dimension combined, explained less than half as much as those habits.

Put simply, the number on the form describes the foreman holding the pen at least as well as the climber being rated. Caleb's two ratings were largely two foremen's habits. The line clearance foreman values speed above all, because utility contracts pay by the mile, and dislikes being questioned. The residential foreman values care, because a mistake there lands on someone's roof.

Neither foreman was wrong about Caleb; each was rating a different job using the same form.

A Hundred Years of Trying to Fix Ratings

DeNisi and Murphy (2017) reviewed a century of appraisal research. Much of the early work tried to find a rating format that would eliminate errors; no format did. Research on rater training proved more productive, and later work turned to the purposes of appraisal, the context and the reactions of the people rated, recognizing that raters are motivated actors rather than neutral instruments. Their review suggests that improving appraisal requires attention to what is rated, who rates and why, not just a better form.

Defining What Good Looks Like

We started with what is rated. Using the job analysis from Unit 1, four senior climbers and two foremen identified five dimensions that matter across crews: hazard assessment and rigging, cutting quality and control, safety practices and gear, communication with the ground and productivity. For each, they wrote behavioral examples at three levels. For communication with the ground, a low rating describes "drops pieces without a clear call or before the ground worker answers"; a middle rating, "calls every drop but sometimes before the zone is fully clear"; a high rating, "calls every drop, waits for an answer and stops work when a ground worker is out of position."

Separating productivity from the other dimensions mattered most. The line clearance foreman could now rate Caleb low on productivity and high on safety, rather than folding both into one number.

Training the Raters

Woehr and Huffcutt (1994) compared four kinds of rater training across many studies. Training that teaches raters to avoid errors such as leniency or halo reduced those patterns but did little for accuracy. The approach that improved accuracy most was frame-of-reference training, in which raters score sample performances, see how experts scored the same ones and adjust until they share one yardstick. Training raters to observe more carefully also helped.

We used frame-of-reference training. All seven foremen watched six short helmet-camera videos of climbers working, rated each on the five dimensions using the new anchors, and then compared their ratings with those agreed by the senior climbers. Where ratings differed by more than one point, the group discussed what they had seen. By the sixth video, most foremen were within a point of the agreed rating on every dimension.

A Calibration Meeting

Training alone does not erase rater habits. After the spring ratings, the foremen met for two hours to review the distribution of their ratings side by side, without names. Two foremen whose ratings were much higher than everyone else's saw that pattern for themselves. Where a climber had been rated by two foremen, they discussed the difference. The meeting does not force ratings to match; it requires a reason for differences.

Separating Pay From Development

DeNisi and Murphy note that the purpose of a rating shapes it. When ratings decide pay, raters tend to be more lenient and employees more defensive. We split the two uses. The five-dimension ratings feed a development conversation each spring. Raises are now based on certifications earned, tenure steps and a separate productivity measure from job records, with the dimension ratings used only to confirm that a climber meets the safety standard.

What Changed for Caleb

Under the new system, both foremen rated Caleb high on safety and communication. The line clearance foreman rated his productivity in the middle, and Caleb agreed that he works slower than most on utility jobs because he questions cuts near energized lines that he considers unsafe. That disagreement, once buried in a single number, became a useful conversation. The two of them now review contested cuts with the utility's safety contact rather than arguing at the base of the tree.

Limits

The new system takes more of each foreman's time, about an hour per climber per year. Frame-of-reference training will need repeating each year as new foremen arrive. Even with training, Scullen and colleagues' findings suggest that rater effects will remain large. Ratings from more than one source, such as a senior climber, may help, and we will try adding one peer rating next year.

Conclusion

A single overall rating told the company more about its foremen than its climbers. Scullen and colleagues explain why: rater tendencies dominate ratings. Woehr and Huffcutt show that frame-of-reference training improves accuracy, and DeNisi and Murphy show why purpose and context matter as much as format. Five anchored dimensions, trained foremen, a calibration meeting and pay decisions separated from development ratings give a clearer picture of each climber's work.

References

DeNisi, A. S., & Murphy, K. R. (2017). Performance appraisal and performance management: 100 years of progress? Journal of Applied Psychology, 102(3), 421-433. https://doi.org/10.1037/apl0000085

Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85(6), 956-970. https://doi.org/10.1037/0021-9010.85.6.956

Woehr, D. J., & Huffcutt, A. I. (1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67(3), 189-205. https://doi.org/10.1111/j.2044-8325.1994.tb00562.x