Statistics Tips and Enigmas

by Chuck Schultz

Statistics Tip: Scraping the Bottom of the Band

Employees who possess certain combinations of characteristics perform a job better than those who lack important attributes. Good selection tests effectively measure the important characteristics. However, tests take only a brief slice of each applicant's behavior. They do not measure all the important attributes and the ones they do measure they measure imperfectly. As a result, there is a less than perfect relationship between test scores and job performance.

Since the test measures imperfectly, the person strongest in the characteristics measured may have received a mediocre score. Using the statistic called the standard error of measurement, you can mark off a confidence interval. This allows you to see the probability that a person within the interval might equal the person with the top score in the characteristics measured. You may want to consider all applicants whose scores are within a band of scores not significantly lower than the best score.

A valid test offers good evidence about which of the applicants are best qualified. Most likely the person with the top score will be the best performer. But that might be only a 25 percent chance leaving a 75 percent chance it will be one of the other applicants. The chance may be 20 percent that the number two scorer will be the best, and there is a small chance that the applicant with the most potential is in the lower half of the score list.

The chance that an excellent performer has one of the top five scores is quite good if the test has measured the most important attributes of good performance. However, sometimes a crucial characteristic is hard to measure and you may wish to measure it more precisely. On the other hand, that characteristic may be rare in the applicant population. You may want to interview more than five candidates to increase the chance of finding one with the crucial characteristic.

Selecting randomly within the band will, in the long run, lower performance of your workforce. Then what is the use of banding? The band provides the opportunity to refine your measurement over several candidates. Additional accurate information can help you ascertain which people within the band will be the best performers.

The lowest scoring person in the band may have a ten percent chance of out performing the top scoring person. Chances are much less that the lowest scoring person in the band can out perform all of those scoring higher. If you don't consider other qualities or if you assess them unreliably picking any one but the person with the top score will reduce test utility. That is, it will lead to a less productive workforce.

Statistics Enigma: There Is No Unbiased Estimate of Sigma

We often compute an unbiased estimate of the population variance. Although the symbols may vary, many writers use s² to indicate that unbiased estimate. The average of all possible values of s² is the expected value of ², which denotes the population variance.

While s² is an unbiased estimate of ², s is not an unbiased estimate of . How can that be? Because s² is squared and its distribution is positively skewed. The mean of a skewed distribution is different from the median. Over half of the unbiased estimates of the population variance are smaller than the population value. That is, the median is less than the mean, which is the unbiased estimate. The population value, ², is the same as the mean value of s², but it is greater than the median value. The median divides sample into the 50 percent with the larger values from the 50 percent with smaller values..

The distribution of s, the distribution of the square roots of s²s, is symmetrical. Mean s equals median s, and it is smaller than . We can use S² to indicate the actual sample variance, rather than the variance estimate of the population from which the sample comes. S² is smaller than ² because samples are less likely than the parent population to contain values at both extremes.

The formula for s² employs N - 1 in the denominator, while S² uses N. The N -1 corrects for circumstance that the sample variance is most often smaller than that of the population. So s² equals [N / (N -1)] S².

(Editor's Note: Originally, at the request of the author, this column was unattributed. However the clamor of "Author, Author!" persuaded me to ask and Chuck to agree to attributing the articles. Chuck Schultz's name will now appear with the monthly statistics tips and enigmas. I would like to thank him again for volunteering to contribute these articles. Since keeping up with Chuck's moves and traveling is sometimes difficult you may send comments and/or questions for Chuck to the Editor who will make sure that he gets them.)