Technical Affairs

Mike Aamodt, Associate Editor

This month's column features a response to a reader's question about rating scales and yet another attempt at HR humor.

Question

In developing a performance appraisal instrument, what is the best type of scale that should be used?

Answer

This is a frequently asked question -- one that is best answered with the standard psychology response of "it depends." In this case, it depends on what it is that you want to measure and what you intend to do with the data. For example, some scales are better for providing feedback to employee whereas others are better at eliminating common rating errors (e.g. halo, leniency). Commonly used performance appraisal scales fall into four categories: comparison to other employees (relative performance), frequency of behavior, absolute performance, and extent to which organizational expectations were met.

Comparison to Other Employees

Scales tapping performance relative to other employees are used when an organization is concerned about rating leniency or when employees must be compared for such purposes as promotions or layoffs. Common employee comparison methods include rank order, paired comparisons, and forced-distribution. Forced-distribution scales can only be used when performance is normally distributed. Examples of other employee comparison scales are:

Decision Making

Much better than other officers
Better than other officers
The same as other officers
Worse than other officers
Much worse than other officers

Report Writing Ability

Well above average
Above average
Average
Below average
Well below average

Employee comparison scales probably should not be used when the purpose of the appraisal is to provide feedback to employees or assign merit pay. Telling an employee that he is worse than other employees does not provide the specific behaviors necessary to improve performance. Furthermore, when employee comparison scales are used in such cases, it is possible for a good employee to receive a low rating. For example, suppose that we have done a great job of selecting and training employees. As a result, all of our employees are performing at outstanding levels. However, in an employee comparison system, several of our outstanding performers will be rated as "below average."

Frequency of Behavior

Performance can also be rated on the frequency with which certain behaviors occur. For example, we expect our employees to follow safety guidelines. So, we ask supervisors to decide whether their employees always, almost always, often, seldom, or never follow the safety rules. As you can imagine, it is often difficult for a supervisor to distinguish between levels such as almost always and often. Some frequency scales (e.g. Behavioral Observation Scales) provide specific guidelines. For example:

1. Employee followed safety rules less than 65% of the time
2. Employee followed safety rules 65-74% of the time
3. Employee followed safety rules 75-84% of the time
4. Employee followed safety rules 85-94% of the time
5. Employee followed safety rules 95-100% of the time

5. Was never tardy
4. Was late once
3. Was late two-three times
2. Was late four-ten times
1. Was late more than ten times

As you can imagine, there are few jobs or performance dimensions in which a supervisor would be able to rate the exact frequency with which behaviors occur. Exceptions would be such dimensions as tardiness and absenteeism.

Absolute Performance

A third approach is to rate employees on their absolute performance. An example of such a scale would be:

Outstanding
Good
Fair
Poor

The problem with such a scale is that terms such as outstanding or poor have different meanings for different people. Furthermore, absolute terms such as "fair" or "poor" are often emotionally charged. In fact, research indicates that employees consider ratings of "fair" or "satisfactory" to be bad ratings.

Extent to Which Organizational Expectations are Met

A fourth approach (the one I like most) is to rate employees on the extent to which their behavior meets the expectations of the organization. For example:

Greatly exceeds expectations
Exceeds expectations
Meets expectations
Falls slightly below expectations
Falls well below expectations

Such an approach allows for high levels of feedback and can be applied to most types of employee behavior. The key to this approach is to have behavioral statements that correspond to an organization's expectations. Some behaviors, however, are not suitable for such a scale. Take for example the expectation that police officers should always wear their seat belts. If they wear their seat belts all the time, the officers have met expectations (a rating of three): there is no way to get a higher rating because one cannot wear a seat belt more often than always and thus can never exceed expectations. To solve this problem, it is often necessary to divide behaviors into several categories and then provide an overall rating. For example, in the scale below, a supervisor would place check marks next to the appropriate behavioral elements and then make an overall dimension rating.

Use of Weapons and Vehicle

Behavioral Elements

Behaviors that can meet or exceed expectations

Effectively handled vehicle in pursuit or emergency response situations
Demonstrated marksmanship above minimum requirements

Behaviors that can only meet expectations

Weapons ratings are current and meet minimum requirements
Weapons were carried in an appropriate manner

Behaviors falling below expectations

Applied force when use of force was not justified
When force was justified, did not use appropriate type and/or amount
Vehicle was not operated in a safe manner or in accordance with department policy
When weapons were used, reports were filed late and/or contained inaccurate information
Carried a weapon not issued and/or approved by the department

Dimension Rating

5. Performance could not realistically be better than exhibited by this officer
4. Officer's use of weapons and vehicle went beyond expectations
3. Officer met expectations
2. Minor problems occurred in this area
1. Serious problems occurred: officer needs immediate and extensive improvement

The four approaches just discussed are all useful for certain situations. In fact, it is certainly conceivable that in any performance appraisal instrument, all three scale types would be used for at least one dimension. The key to choosing the appropriate scale is to ask "What am I trying to measure?" and then choosing the scale that best accomplishes your goal.

Though you can use a variety of scales within one performance appraisal instrument, it is not a good idea to mix the scale types within a dimension. Take the dimension below as an example. The dimension contains relative terms (average, below average), absolute terms (outstanding, terrible), and expectancy terms (above expectations).

Customer Service

Outstanding
Above expectations
Average
Below average
Terrible

After a scale has been chosen, a decision must be made regarding the number of points on the scale. Though five is probably the most common, the rule for determining the number is simple: Have as many scale points as you need to measure the behavior in question. If you find yourself struggling for an adjective to describe the fifth point on the scale, four points are probably enough.

A final thought. Though the choice of the right scale is important, research and case law are clear that the most important aspect of a performance appraisal system is the way in which it is administered (e.g. rater training, feedback to employees, appeal procedure, perceived fairness, consistent application of standards).

HR Humor

Mike's Celebrity Employment Interviews

Interviewer: What type of job do you want?
Marv Albert: Something I can sink my teeth into

Interviewer: How did you hear about our position opening?
Evander Holyfield: I kept my ears to the ground

Interviewer: Why did you leave your last job?
P.J. Carlesimo: They said I choked under pressure

Interviewer: What type of position are you looking for?
Bill Clinton: Any position is fine with me

Interviewer: What did you do at your last job?
Steven Spielberg: A lot of copying

Interviewer: We don't seem to have a copy of your resume
Mike McCurry: I'm sorry. It usually takes me nine months to produce a document

Interviewer: Why do you want to be a police officer?
Pamela Lee Anderson: I like to make busts

Interviewer: How would you describe your current skills?
Lorena Bobbitt: On the cutting edge

Interviewer: What can I get you to drink?
Latrell Sprewell: Punch

Mike Aamodt, a Professor of Psychology at Radford University serves as our Associate Editor for the Technical Affairs column and as our unofficial humor editor. If you have a technical question you want answered/discussed, wish to comment on this month's article, or want to share a humor item please contact Mike. He may be reached by email(maamodt@runet.edu), phone [(540) 831-5513)] or fax [(540) 831-6113].