Technical Affairs

Mike Aamodt, Associate Editor

This month's column contains an insightful response to the last Technical Affairs Column on Understanding Regression. Dr. Doverspike's response is an excellent example of a different perspective on using regression to eliminate variables that don't account for unique variance. At the end of the column is another piece of HR Humor.

On Organizational and Individual Utilities: A Short, Rambling Essay on Implications for Testing and Assessment

by Dennis D. Doverspike

In the recent testing and assessment literature, I have noticed a disturbing trend where authors propose that Industrial Psychologists, and like minded compatriots in the assessment field, are concerned with the value of tests for individuals. The problem with this assumption is that it is misleading, and perhaps even false at the level of our profession as a whole, unless one adopts a trickle down theory (i.e., testing has value for companies and society, making both more efficient, and in the long run this serves the interest of individuals). By the very nature of the field, Industrial Psychologists are concerned with the utility of tests for organizations or for employers, and not with the utility for individuals.

This distinction between organizational and individual utility is a critical one. In this essay, I will argue that the distinction helps to explain two areas of misunderstanding:

1. Why individuals often see the results of testing as inaccurate or unfair.

2. The difference between the approach toward testing taken by Industrial Psychologists and Clinical Psychologists.

There are many other implications that arise from this difference between organizational and individual utilities. However, at this point I wish to restrict my arguments to the two preceding points.

Industrial Psychologists are concerned with utility, but the utility they are concerned with is the utility of tests for companies or for organizations. Utility is defined here as the gain received as the result of the use of an assessment device, where this gain can usually be measured in dollars. In theory, organizations make a large number of decisions, so that even if they make some errors, the correct decisions overwhelm the incorrect decisions. As a result, even when testing or assessment is a less than perfect diviner of the future, it can be shown or proved that tests can result in substantial organizational utility. As a practitioner of the fine craft of Industrial Psychology, I am willing to accept those studies which demonstrate the substantial economic gain which can result from a testing program, but to the best of my knowledge, the vast majority of this literature is devoted to the question of organizational utility.

Individuals, as opposed to organizations, do not normally make a large number of decisions. Typically, they only have one or two opportunities to make critical, life altering decisions. For example, in selecting a graduate school or deciding to go to graduate school, the individual often may only be making one or two decisions, the psychology department may be making thousands of decisions This is the basis of many businesses such as life insurance. Considering the case of insurance is an interesting academic exercise, since with insurance, individuals are actually betting against themselves. Thus, individual utilities are quite likely to be different from organizational utilities, as individuals do not have the opportunity to see if errors or decisions cancel each other out.

Consider the marginal student applying for graduate school at Midwestern University (This example is based in part on the April, 1999 ACN Technical Affairs Column). This student, Richard, has dreamed all of his life of getting a Ph.D. in psychology. He has told all of his friends that he is going to get a Ph.D. in psychology. He believes that if he can just get a Ph.D., he will be rich beyond his wildest dreams. He knows that his GRE score is only 900 and his GPA a 3.0, but his grandmother was sick during college and he cared for her and she died the day he took the GRE. He knows that if he could have spent more time on schoolwork, he would have had a 4.0 GPA. He also knows that he could have scored much higher on the GRE, but he does not test well.

Richard knows he can do well in graduate school. Even if he flunks out, he figures what will it have cost him -- a couple of thousand dollars -- a reasonable investment for the chance to be happy and rich. Richard figures that he will suffer a substantial gain if he is admitted and can succeed, no real loss if he is admitted and does not succeed, and a huge loss if he is not admitted at all. For the Midwestern University, the case is clear, Richard is a poor bet and should be rejected because his rejection adds to the organizational utility; the cost of additional testing to determine otherwise would be cost prohibitive. For Richard, he will be heartbroken if he is not admitted, and plans to call the admissions director and beg for a chance.

The important point in the preceding example is that both Richard and the University are correct from their points of view. For Richard, individual utility suggests that he should take a chance, even though he may fail, and attempt the program at Midwestern University. For the University, the decision is clear, they will and should reject Richard.

Individual versus organizational utilities also help to explain the difference between the type of testing and tests a Clinician might use (see Aamodt, 1999) and the type of tests an Industrial Psychologist might use. In the typical situation, the Clinician is involved in helping individuals make decisions that maximize their utilities, the Industrial Psychologist is interested in helping an organization make decisions which maximize their utilities. This helps to explain why a clinician might use several tests to measure the same construct.

Consider the following analogy: You have gone to a doctor and have been tested for lung cancer. The test comes back and indicates that you have lung cancer and should have your lung removed. The normal reaction, after the initial shock, would be for you to ask for a second test or a second opinion. Of course, the HMO might reply that there is no need for a second test, second tests are costly and they do not add to the prediction of cancer. Regression equations have indicated that a second test does not add to the amount of additional variance explained. For the HMO, which tests 50,000 people a year, adding a second test at a cost of $1,000 a person, simply makes no economic sense. On the other hand, from the patient's perspective, spending $1,000 seems like a small amount to pay compared to the cost of making a mistake and removing a healthy lung, even if the odds of the first test being wrong are very low. For the patient, the decision is clear, test a second time.

This situation applies as well to Richard. Perhaps Richard believes that if the University would grant him an interview, they would see that the GRE underestimated his verbal and quantitative ability. Richard would gladly spend the money to drive to the University for the interview, even if it turns out he does poorly in the interview and is still rejected. The University knows, however, that interviews add little to prediction and are costly. If the University agreed to interview every student then the faculty would never have time for teaching or research. Richard's individual utility tells him that he should be given another test, the University's consideration of organizational utility leads to a conclusion of no additional tests.

If Richard had gone to a Clinical Psychologist for vocational advice that Clinician might have been more than willing to give Richard multiple tests, although in these days of managed care Clinical testing is changing as well. The Clinical Psychologist would figure, correctly, that at the level of Richard making an individual decision as to whether he should pursue a graduate program, the addition of a test would in the long run have individual utility.

The point then is that there can be a discrepancy between individual and organizational utilities. Sometimes, individual utilities and organizational utilities can clash. When they do, we often end up with human consequences - an applicant for employment or education who feels that they have been treated unfairly. Both the organization and the individual can be correct, from a utility standpoint, at the same time. The individual could be right that if they were just given one more test, then they would be given the job or the position in graduate school and would turn out to be very successful. However, the organization is also correct in that the cost of acquiring that additional information is excessive when averaged across many decisions. Overall, whether discussing a test or a test battery, multiple factors must be considered by the organization including reliability, validity, length, cost, practicality, objectivity, and perceived fairness. In the end, unless we can achieve perfect validity, no matter how fair the test appears to be from the organization's point of view, there will still be individuals who feel that they have been treated unfairly and that had they just been given one more test or a different test the results might have been different.

References

Aamodt, M. (1999, April) Understanding regression. Assessment Council News, 9-12.

Dr. Dennis Doverspike is an associate professor of psychology at the University of Akron. He is also the vice-president of Barrett and Associates, a human resource management consulting firm. His research interests include personnel selection and job evaluation.

HR Humor

ACN Reader Melissa Marshall sent the following piece of HR Humor

What the recruitment ad said	What the ad really meant
Join our fast-paced company	We have no time to train you
Casual work atmosphere	We don't pay enough to expect that you will dress-up; a couple of the real daring guys wear earrings
Must be deadline oriented	You will be six months behind schedule on your first day
Some overtime required	Some time each night and some time each weekend
Duties will vary	Anyone in the office can boss you around
Must have an eye for detail	We have no quality control
Career-minded	Female applicants must be childless (and remain that way)
No phone calls please	We've filled the job; our call for resumes is just a legal formality
Seeking candidates with a wide variety of experience	You will need it to replace the three people who just left
Problem-solving skills a must	You are walking into a company in perpetual chaos
Requires team leadership skills	You will have the responsibilities of a manager, without the pay or respect
Good communication skills	Management communicates, you listen, figure out what they want, and do it

Mike Aamodt, a Professor of Psychology at Radford University serves as our Associate Editor for the Technical Affairs column and as our unofficial humor editor. If you have a technical question you want answered/discussed, wish to comment on this month's article, or want to share a humor item please contact Mike. He may be reached by email (maamodt@runet.edu), phone (540) 831-5513 or fax (540) 831-6113.