Practice Exchange
by Ilene Gast, Associate Editor
This column highlights innovative public sector programs at the Federal, state, and local level. If you are conducting a project that would interest the ACN readers, or if you know someone who is, please let me hear from you. I can be reached by phone at (202) 305-0590, by fax at (202) 305-3664, or electronic mail at ifgast@aol.com.
Performance America
by Sandra Shoun
OPM's Personnel Resource and Development Center (PRDC) is establishing a new consortium, "Performance America," to help public sector agencies assess, monitor, and improve their performance. Initiatives such as the National Performance Review, the Government Performance and Results Act, and the Chief Financial Officers Act have underscored the need for public sector agencies to become high-performance, customer-driven organizations. Now, more than ever, agencies must demonstrate their programs' value by linking organizational performance to important, measurable, financial and non-financial outcomes. In addition, continued government downsizing and restructuring intensify the challenge of motivating and revitalizing employees, any organization's most important resource.
Many public sector organizations have met these challenges by conducting organizational assessment surveys. However, surveys vary in the dimensions covered, and in how the dimensions are defined and translated into survey questions. The lack of a common language for measuring effectiveness has hampered efforts to track performance within individual agencies and throughout the public sector. With its Organizational Assessment Survey (OAS), OPM has developed a common language for assessment. The OAS measures aspects of organizational performance that characterize "high performing" organizations. Because it gives government agencies a common language for measuring their effectiveness, comparisons across agencies become possible.
OPM will administer the OAS in Federal, State and local agencies that join the Consortium. Participating agencies will receive a profile describing their agency along these dimensions of organizational performance. The Consortium will develop a database that agencies can use "benchmark" their performance against that of similar organizations, and learn from the experiences of other Consortium members. In addition, the Consortium will facilitate sharing "best practices" in government and in business.
Agency officials who are interested in learning more about the Performance America Consortium should contact David Dye at (202) 6060-3742.
Assessing Language-Learning Ability at INS
by Patricia Harris
Why Do We Assess Language-Learning Ability?
One of INS' major challenges in the selection of Border Patrol Agents is to ensure that individuals hired at the entry level possess or develop proficiency in speaking the Spanish language. Effective communication in the major native languages of individuals who live near or travel to the nation's borders is vital to the Border Patrol's law enforcement efforts. Accordingly, law enforcement along our Mexican border requires that Border Patrol Agents be proficient in the Spanish language.
A significant percentage of individuals who apply for entry-level Border Patrol Agent positions possess little or no proficiency in the Spanish language. Prior to 1992, training efforts aimed at teaching these individuals to speak Spanish have resulted in undesirable and costly failure rates at the Border Patrol Training Academy. To address the problem, INS added an Artificial Language Test (ALT) to its selection battery to screen out applicants who would have difficulty learning Spanish at a level required to pass the Academy's Spanish language training course.
How Does the Artificial Language Test Measure Language-Learning Ability?
The Artificial Language Test is a 50-question written test that is designed to simulate the grammatical and syntactical structures of neo-Latin languages such as Spanish and French. During the test, examinees receive a training booklet which teaches a variety of grammatical rules and demonstrates how to apply those rules using an artificial language. Examinees, for example, are taught how to form feminine and masculine singular nouns, pronouns, and adjectives; and feminine and masculine plural nouns, pronouns, and adjectives. In addition, examinees learn how to form verbs, adjectives, and adverbs; and the possessive of nouns and pronouns. The training booklet includes a glossary of "words" in the artificial language.
Examinees are allowed sufficient time to study the training booklet, after which they answer test questions which require them to apply all of the rules they have learned. They are allowed to keep and refer to the training booklet throughout the testing session. Consequently, although it is important for examinees to understand the grammatical rules in the test, it is not necessary for them to memorize the training information.
How Effective is the Artificial Language Test?
Research conducted to determine how well the ALT predicts the Spanish language-learning ability of Border Patrol Agent trainees has shown excellent results (cf. Diané, Brogan, and McCauley, 1991). When the ALT scores of Border Patrol Agent trainees were compared statistically with the trainees' final scores on the Border Patrol Academy Spanish Language training course, a consistently high, positive relationship was found. Trainees who received high scores on the ALT also tended to perform very well in the Spanish Language training course. Trainees who received low ALT scores tended to score lower in the Academy's Spanish language training.
Another indication of the usefulness of the ALT has been its effectiveness in reducing the rate of failure and attrition at the Border Patrol Training Academy. A study conducted by the Academy's Basic Training Branch compared the attrition rates of trainees who were hired before the ALT was added to the Border Patrol selection battery with the attrition rates that were evident after the test was adopted. This study revealed that use of the ALT resulted in a 76% reduction in attrition rate due to failure to learn Spanish. Prior to adding the ALT, the attrition rate associated with failure to learn Spanish was 11.3%. After the ALT was adopted, this rate decreased to 2.7% (cf. Immigration and Naturalization Service, Border Patrol Academy, 1993).
In an environment in which the cost of training a single person is $18,000, the savings resulting from use of the ALT are significant. In view of INS' projected hiring of 1,000 new recruits per year over the next five years, the savings will approach $7.5 million during that time period.
For further information, please contact Magda Colberg or Patricia Harris the at INS' Personnel Research and Assessment Division in the Office of Human Resources and Development, at (202) 305-0600.
References
Diané, C.D., Brogan, F.S., & McCauley, D.E. (1991). A validation of artificial language tests for Border Patrol Agents. Washington, DC: U.S. Office of Personnel Management.
Immigration and Naturalization Service, Border Patrol Academy (1993). Language aptitude testing and the performance of Border Patrol Agents during basic training. Glynco, GA: Author.
© Copyright 1996 by the IPMA Assessment Council. All rights reserved.