Test Security and Administration: Some Almost Real Cases

Terry McKinney

While test security is serious and important, I have often felt that humor can help deliver a message. I recently found the following on the Internet:

Two young engineers fresh out of college put in applications for an engineering position with a private sector employer. Both applicants having the exact same qualifications, they were asked to take a test by the hiring manager to help decide which one to hire. Upon completion of the test, both applicants had missed only one of the questions. After reviewing the tests, the manager decided to speak with the losing applicant first:

Manager: Thanks for your interest, but we have decided to give the position to the other applicant.

Applicant: Why would you do that? We both got 9 questions correct - why choose the other applicant over me?

Manager: We have made our decision not based upon the correct answers, but on the question you missed.

Applicant: And just how would one incorrect answer be better than the other?

Manager: Simple. The other applicant put answered "I don't know" for question 5. Your answer was "Neither do I."

The following describe some security problems or situations that I have heard about (and I may have experienced one or two of these in my career). To set the record straight - no one should infer that all of these situations or any specific situation occurred with my current employer, the City of Phoenix Personnel Department.

All of these situations are true. Some aspects of the situations have been changed to protect the innocent or others.

The Case of the Extra Printer

A Personnel Department in a large organization had its own "print shop" and the same printer had been employed for several years. Testing staff did not participate in the printing operation. At some point it was discovered that this printer had been running a couple of "extra" copies of every examination printed by this agency. It ended up that this individual never attempted to get any gain from these examination booklets. The individual would get library books on the subject to study and try to figure what the correct answer was - merely for his own entertainment. After an investigation was completed, the individual had much more time to read library books as he was terminated.

The Case of the Busy Analyst

For a public safety promotional examination, the operating department provided an experienced analyst with a hand written copy of a writing exercise with the correct responses. The analyst gave the material to a fairly new typist to put in final form. She typed the material letter perfect and returned it to the analyst. The analyst, being very busy (as all of us are) thanked her and asked his fairly new assistant to make copies and put them in the file. A week of so later the analyst retrieved the material and took it to the test site. Soon after the exam material was distributed, one of the test takers raised his hand and asked "Fred, why are the answers included with the questions?" At which point the analyst collected the test material, returned to the personnel office, and began giving serious thought to retirement.

The Case of the Weekend Walk

A senior analyst and her then husband were taking a weekend walk around the community's civic center. As they passed the Library Auditorium the senior analyst turned to her husband and said "That is the room we used yesterday to test 400 people for Administrative Aide ------ and that looks like a copy of the test booklet still laying on one of the tables." As a matter of fact it was one of the test booklets. Luckily, this event had no relationship to their later divorce.

The Case of All in the Family

Frank had worked for the agency for several years as a senior engineering technician. Frank's wife Leslie had some engineering background and wanted very badly to work for the same organization as Frank. So when engineering aide opened for recruitment, Leslie applied and was invited to take the written test. Leslie was terrified of written exams and Frank decided to take the test for her. The agency involved did not typically require a photo id. The personnel aide checking folks in at the examination did not think any thing was out of the ordinary - Leslie is one of those names that can be either male or female. A day or two after the examination the Personnel Department received a call from another agency employee wanting to know why Frank was taking a test for a job that was five pay ranges lower than his current job. After a detailed investigation, Leslie kind of got her wish. Frank and Leslie did get to opportunity to spend more time together, as Frank became unemployed.

The Case of the Helpful Manager

The Human Resources Department had worked with the same subject matter expert in Real Estate Division (the Assistant Real Estate Supervisor) for several years. This individual had been involved in developing several job related written exercises. A week before the 1997 examination was to be administered, the County Manager's Office received a copy of the 1995 examination with a note asking how does one sign up for the study group that the Assistant Real Estate Supervisor was conducting for his people. The examination was quickly postponed, new written problems created with the assistance of another subject matter expert. Disciplinary action was taken against Assistant Supervisor - though not as severe as Human Resources staff would have desired.

The Case of the Changing Directions

The agency had used the same form of an oral direction test for several years. A new enthusiastic analyst decided to make some revisions to the test. Rather than saying "put the letter X under the picture of the first hammer," it was changed to "put the letter A under the picture of the second hammer." The first time the revised test was used, staff was amazed - some candidates were madly erasing their answer sheet soon after the oral tape started and there were several answer sheets turned in that would have received perfect scores had the old answer key been used.

The Case of the Eyes Have It

A fairly new analyst was setting up a demonstration or performance examination for Highway Maintenance Worker II. Part of the examination was a tool identification section. Exam administration began early on a Saturday morning. Midmorning a senior analyst stopped by to see how the examination was going and was shocked to see that the tool identification section was being given within easy view of the applicant waiting area.

The Case of the Swift Candidate

An agency has a practice of allowing candidates to compare their answer sheet to the key after scores have been sent out. This gives candidates the opportunity to check for any scoring errors. Roberta Swift is an agency employee and takes many examinations - generally doing fairly poorly on them. About five years ago, it was noticed that Roberta had discovered scoring errors on several of her exam papers - usually just enough to pass her. The agency modified its procedures by removing all #2 pencils from the review area and providing pencils with colored lead. This worked well for a year or so, until it was noticed that Roberta had discovered scoring errors on several of her exam papers - usually just enough to pass her. The agency then started putting the applicant's answer sheet in a plastic cover and taping it. This worked well for a year or so, until it was noticed that Roberta had discovered scoring errors on several of her exam papers - usually just enough to pass her. The agency now photocopies the applicant's answer sheet. This practice has been in effect for over a year and so far Roberta Swift has not found a way to find scoring errors in her test papers.

The Case of It Is in the Mail

The Test Research Unit of a large geographically spread out agency routinely had remote test sites return test material by mail. Once some test material was several days over due and the Test Research Unit checked with the remote site. They indicated that they had mailed material to the Research Unit a number of days earlier. Several days later the Test Research Unit got a phone call from the Budget & Research Unit asking what they should do with a package of test materials that had been floating around their Unit for the past week.

Each testing situation has the possibility of a security breach. The carelessness or oversight of HR/Personnel staff causes some. Others are caused by intentional misconduct of the applicant. We can never be too careful or too concerned with security. Hopefully we can learn from the mistakes of other (unnamed) fellow professionals.

Terry McKinney is the Personnel Test Analyst with the City of Phoenix. He can reached at tmckinne@ci.phoenix.az.us or 602-261-8861. This article is adapted from Terry's presentation at the 1998 IPMAAC Conference in Chicago.