Technical Affairs

Mike Aamodt, Associate Editor

In the last issue of the ACN, we provided a basic understanding of correlation. In this issue, we will take correlation a step further and explain the basics of regression. At the end of the column is another piece of HR Humor submitted by an ACN reader.

Understanding Regression

In the last edition of the ACN, we discussed how correlation is used to show relationships between two variables. Though correlation is the basis for regression, regression analysis lets us do some things that simple correlations do not: make precise predictions, combine small correlation coefficients, and get rid of excess baggage.

Making Precise Predictions

Suppose that you conduct a validity study and find that the correlation between scores on a civil service exam and performance in the police academy is .45. From these results we would conclude that the two are highly related and that applicants who score high on the test will perform better in the academy than applicants who score low on the test. Though such information is useful, it may not tell us all we need to know. That is, we know that an applicant scoring 85 on the exam should do better in the academy than one scoring 75, but how well would the applicant scoring 75 perform if we hire him? Would he still be able to pass the academy? Regression analysis can help answer such questions by allowing us to take a score on a test, enter it into a regression equation, and obtain an applicant's predicted score on some measure of work performance (e.g., academy grades, supervisor ratings).

Combining Small Correlations

If you remember from our discussion on correlation, we like to see correlations of at least .20 between a test and a measure of performance. So, suppose that we have a personality measure that correlates .15 with police performance and a cognitive ability test that correlates .13 with performance. Based on these small correlations, we would probably be disappointed and lose hope of ever winning the Nobel Prize for HR Validity Studies. However, regression analysis might be able to save the day. That is, with multiple regression, we can combine two small correlations into one big correlation (if the two predictors do not correlate with one another, we can simply add the correlations, but it is usually more complicated than that).

The late I/O psychologist Dan Johnson likened the use of regression to a fishing trip. During our trip, we can try to catch one huge fish to make our meal, or we can catch several small fish that, when cooked and placed on a plate, make the same size meal as one large fish. With selection tests, we try for one or two tests that will correlate with performance at a high level. Unfortunately, such big correlations are as hard to get as it is to catch a fish large enough to feed the entire family. But by combining several tests with smaller validities, we can predict performance as well as we could using one test with a very high validity.

Getting Rid of Excess Baggage

One of the nice things about multiple regression is that, in addition to combining small correlations, it also tells us if we have too many variables measuring the same thing. That is, suppose our selection battery contains a personality test and an unstructured interview. The personality test correlates .25 with performance and the unstructured interview correlates .20 with performance. We start to get excited because if we add the two together, we would have a multiple correlation (R) of .45. However, after entering our data into a regression analysis, we find that our equation "threw out" the interview because it was measuring the same thing our personality test was measuring C social skills and extroversion. So, even though we thought we were measuring two different constructs, our test and interview were actually measuring the same thing (they were highly correlated).

A few years ago, I was asked by a clinical psychologist to validate the test battery he was using to select police officers. As I reviewed his battery, I was stunned to see that every applicant was administered three different measures of cognitive ability and three different personality tests. When I asked the psychologist why he used so many similar tests, he told me that he "got something different from each one of them." However, since the three cognitive ability measures were highly correlated as were the three personality tests (he scores them pass/fail), I doubted that the extra tests provided any new information.

To test this idea, I entered the test scores into two separate regression equations - one to predict his overall ratings of "suitability" and one to predict supervisor ratings of on-the-job performance. As expected, one personality test and one cognitive ability test predicted his suitability ratings - the other tests did not help predict his ratings (the nerd way to say this is that the other tests did not account for unique variance). I was unsuccessful in explaining the results to him in terms of statistics so I finally said "What the results show is that you can make the same decisions with two tests as you would with six. The difference is that you will save about $100 in testing costs per applicant." That he understood!

The moral of this story is that when selecting employees, the test battery should not contain several measures of the same knowledge, skill, or ability. If we go back to our example of making a meal, once we caught enough fish (a cognitive ability test), there was no need to catch more (more cognitive ability tests). Instead, to make the perfect meal, we should add a salad (personality test), some bread (structured interview), and desert (perhaps biodata). Too much of the same thing makes a boring meal and a wasteful selection battery.

Interpreting Regression Results

After conducting a regression analysis, you need to interpret two things: the significance of the regression and the regression equation.

Significance of the Regression

When reading an article that uses regression as its statistical analysis, we usually encounter a table such as the one shown in Table 1. Table 1 shows the results of the regression analysis we use at Radford University to select our I/O graduate students. We began our analysis by entering the students' overall GPA, GPA in their junior/senior years, GPA in psychology courses, total GRE score (verbal + quantitative), and a rating we assign to their reference letters. Because many of the three measures of GPA overlap, the regression determined that the best predictors of students' graduate GPAs are their GRE scores, GPA in their junior/senior years, and the mental agility rating from their reference letters.

Table 1

	r²	R²	p <
GRE Score	.26	.26	.0001
Jr./Sr. GPA	.08	.34	.0166
Reference letters	.05	.39	.0361

Before interpreting Table 1, let's quickly discuss some common symbols in regression. A small r (r) refers to a correlation between a predictor and a criterion. A capital r (R) refers to a multiple correlation between several predictors and a criterion. If the symbol is squared (r² or R²), it refers to the percentage of variance that is accounted for by the correlation. For example r = .40 indicates that there is a correlation between two variables of .40, whereas r² = .16 indicates that the test with a validity of .40 accounts for 16% of the variability in performance (.4 x .4 = .16).

In Table 1, the r² indicates the amount of variance accounted for by each variable in the equation (this figure is determined by squaring the partial correlation coefficient). The R² indicates the amount of variance accounted for by the model. In the example above, the GRE score will predict 26% of the variability in graduate grades. If we use both the GRE and the Jr./Sr. GPA, we can predict 34% of the variability (.26 + .08). If we use all three variables, we can account for 39% of the variability (.26 + .08 + .05). The R² of .39 represents a multiple correlation (R) of .62 (we took the square root of the R²). The third column of the table shows the probability level (we want all of these figures to be lower than the magical .05 level of confidence). If you recall from the discussion of correlation in the last issue of the ACN, this level of validity/correlation is very high.

The Regression Equation

Now that we know we can significantly predict graduate school performance with GRE scores, the GPA during the junior and senior years, and reference letters, it is time to create a regression equation to make our predictions. In its simplest form, the results of a regression analysis yield a regression equation. This equation looks something like:


     Y =  c + (b1) (x1)

Where Y is the predicted value of some variable, c is a constant (in algebra we would call this the intercept and represents the predicted score on the criterion if the scores on the predictor were zero), b1 is the weight we give our predictor (in algebra we would call this the slope and represents the amount of change we would expect in the predictor for each unit of change in the predictor), and x1 is the score on a predictor. Though the constant and the weight can be calculated by hand, we normally let the computer do the work by using a program such as SAS, SPSS, or Excel.

To go back to our early example of the civil service exam, our formula might look like this:


     Predicted academy grades = 35 + (.50) (civil service exam score)

where the 45 is our constant (intercept) and the .50 is our regression weight. Thus if our applicant scored 75 on the test, his predicted academy average would be 35 + (.50) (75) = 72.5. If 70 were needed to pass the academy, we would predict that our applicant would successfully complete the academy. However, if another applicant scored 60 on the civil service exam, his predicted academy average would be 65.

Now, lets look at a more complicated regression equation that came from our attempt to predict graduate student performance. As shown in Table 1, we have three variables that predict a student's graduate GPA. The equation that we obtained for 1998 (we update this equation every year as we add new data) is:


     predicted graduate GPA = 1.36 + (.0009) (GRE)
                              + (.296) (Jr/Sr GPA)
                              + (.518) (reference score)

The 1.36 is the constant (intercept) and the .0009 is the weight that is multiplied by the GRE score, the .296 the weight that is multiplied by the junior/senior year GPA, and the .518 the weight that is multiplied by the mental agility score on the reference letters. Let's use two hypothetical students as an example. Jenny Craig has a GRE score of 1200, a junior/senior GPA of 3.80, and a reference rating score of .60. Richard Simmons has a GRE score of 900, a junior/senior GPA of 3.0 and a reference rating score of .20. The formula to predict the students' graduate GPAs would be:


     Jenny's GPA   = 1.36 + (.0009) (1200) + (.296) (3.80) + (.518) (.60)
                   = 1.36 + 1.08 + 1.12 + .3108
                   = 3.87

     Richard's GPA = 1.36 + (.0009) (900) + (.296) (3.00) + (.518) (.20)
                   = 1.36 + .81 + .89 + .104
                   = 3.16

At Radford, we typically have about 70 students apply for our 15 openings so we only accept students whose predicted graduate GPA is at least a 3.60. Using the data from the above example, we would accept Jenny and her 3.87 predicted GPA and reject Richard and his 3.16 predicted GPA.

Hopefully, the above discussion will make it easier for you to understand regression. To practice what you have learned, go take a look at these recent articles in Public Personnel Management that used regression. I hope that they now make perfect sense to you!!!

Practice Articles

Gilbert, G. R., Hannan, E. L., & Lowe, K. B. (1998). Is smoking stigma clouding the objectivity of employee performance appraisal? Public Personnel Management, 27(3), 285-300.

Look at Table 3 in this article. The column that says "regression coefficient" is the regression weight. The column that says "p-value" is the significance level for the variable. You will recognize the R² column.

Tang, T. L., & Ibrahim, A. H. S. (1998). Antecedents of organizational citizenship behavior revisited: Public personnel in the United States and in the Middle East. Public Personnel Management, 27(4), 529-549.

Look at tables 2 and 4 in this article. Don't pay attention to the "F Change" column as it is not needed to interpret the table. The column titled "Beta" contains the regression weights.

Young, B. S., Worchel, S., & Woehr, D. J. (1998). Organizational commitment among public service employees. Public Personnel Management, 27(3), 339-348.

Look at Table 1 in this article. At the top of this table you will see the R² of .72 that is significant at the .001 level. Also notice that the authors have a column called "parameter estimate." These are the regression weights. The column that says "Sig. t" is the significance level of the variable - in the sample table in this ACN article, our heading was "p < " rather than the "Sig. t" in the Young et al. article.

HR Humor

The following joke was sent by an ACN reader. Keep those jokes coming!!!

A few months ago, there was an opening with the CIA for an assassin. These highly classified positions are hard to fill, and there is extensive testing and a background check involved before an applicant can even be considered for the position. After extensive testing, the field was narrowed to three applicants: two men and one woman.

The three finalists were asked to come to CIA headquarters to take a secret test administered by the HR Department. The personnel analyst took one of the men to a large metal door and handed him a gun.

"We must know that you will follow instructions no matter what the circumstances," the personnel analyst explained. "Inside this room, you will find your wife sitting in a chair. Take this gun and kill her." The man got a shocked look on his face and said "You can't be serious! I could never shoot my own wife!" "Well," says the personnel analyst, "you're definitely not the right person for this job."

So the personnel analyst brought the second man to the same door and handed him a gun. "We must know that you will follow instructions no matter what the circumstances," the personnel analyst explained. "Inside this room, you will find your wife sitting in a chair. Take this gun and kill her." The second man looked quite aghast, but nevertheless took the gun and went in the room. All was quiet for about five minutes, then the door opened. The man came out of the room with tears in his eyes. "I tried to shoot her, I just couldn't pull the trigger. I guess I'm not the right man for the job." "No," the personnel analyst replied, "you don't have what it takes. Take your wife and go home."

Now the personnel analyst was down to the one woman. Again the analyst led her to the same door and handed her the same gun. "We must know that you will follow instructions no matter what the circumstances," the personnel analyst explained. "This is your final test. Inside this room, you will find your husband sitting in a chair. Take this gun and kill him." The woman took the gun and opened the door. Before the door even closed all the way, the personnel analyst heard the gun start firing. One shot after another for 13 shots. Then all hell broke loose in the room. The analyst heard screaming, crashing, and banging on the walls. This noise went on for several minutes, then all went quiet. The door opened slowly, and there stood the woman. She wiped the sweat from her brow and said "You guys didn't tell me the gun was loaded with blanks. I had to beat him to death with the chair!"

Mike Aamodt, a Professor of Psychology at Radford University serves as our Associate Editor for the Technical Affairs column and as our unofficial humor editor. If you have a technical question you want answered/discussed, wish to comment on this month's article, or want to share a humor item please contact Mike. He may be reached by email (maamodt@runet.edu), phone (540) 831-5513 or fax (540) 831-6113.