IPMAAC Oral Examinations Seminar

In 1993, the International Personnel Management Association Assessment Council (IPMAAC) Training Committee began an effort to develop a professional seminar on structured oral examinations. Many IPMAAC members contributed time and information to the seminar development effort. The development of the seminar has been completed in 2001 and the seminar was offered in 2001 in Trenton New Jersey for the personnel assessment staff of the State of New Jersey, and has since been conducted twice in Baltimore, Maryland as part of a six-course certification program for Personnel Assessment Specialists in Maryland State agencies.

Following is a listing of persons who contributed to the preparation of the seminar by developing or contributing materials, or reviewing draft materials:

Nancy Abrams*- Fairport, NY (consultant)
Sue Christopher - State of Wisconsin
Bruce Davey - Bruce Davey Associates (consultant)
Roy Fales - State of New Jersey
James Johnson - State of Tennessee
Paul Kaiser - State of New York
Pat Maher - Personnel & Organizational Development (consultant)
Mabel Miramon - State of California, Personnel Board
Robert Schneider - State of Pennsylvania & MAPAC
Kris Smith* - Darany and Associates (consultant)
Charley Sproule* - Sproule & Associates (consultant)
Judy Trabert* - City of Rochester, New York
Joel Wiesen - Applied Personnel Research (consultant)
Mike Willihnganz* - State of California, Personnel Board

* IPMAAC Training Committee Chairs who led various stages of seminar development

The seminar contains an extensive amount of materials, information, practical examples, and exercises to apply what is learned. Participants review the latest research findings, and receive many examples of the development and use of oral examinations. To help participants apply the instructional information, the job of a Personnel Assessment Specialist (PAS) is used in workshops throughout the seminar. For example, participants write critical incidents for the PAS job, develop different types of oral examination questions, develop rating scales and scoring standards based on job analysis data, train oral examiners on some of the topics covered in a workshop on “Selecting and Training Examiners”, administer an oral examination, and rate candidate responses.

Schmidt and Hunter, in a 1998 review of the validity of alternative measurement methods, found structured interviews to be one of the top three predictors of job performance. The 2000/2001 IPMA/NASPE “Recruitment and Selection Benchmarking Report” found that Oral Examinations were in the top three most frequently used selection methods in the jurisdictions surveyed (along with Written Tests, and Training and experience Ratings). Oral examinations were in the top three selection methods judged “most effective in identifying well-qualified employees.” Based on the survey results, oral examinations were judged “most effective” for supervisory positions, management positions and senior executive positions.

Following is the content outline and schedule for the three-day Oral Examinations Seminar. The emphasis of the seminar is on the development of reliable, valid, job-related, fair, effective and efficient structured oral examinations.

Section	Topic	Day and approx. length
I.	Introductions and course overview	Day 1 am 1 hour
II.	Overview of oral examinations Types of oral examinations and their benefits When is an oral examination appropriate? Combining an oral with other examination types Research findings - validity, reliability, fairness, structure Legal findings and guidance Components of a structured oral examination	Day 1 am 2 hours
III.	Job analysis for oral examination development Review of job analysis data for oral test planning Factors commonly assessed in oral examinations Selecting or developing rating factors or dimensions Use of the critical incident technique Critical Incident Exercise Documentation Some other uses of Critical Incidents	Day 1 pm 3 hours
IV.	Oral test question development Types of oral test questions and situations, when to use them, advantages and disadvantages Exercise - development of stimulus materials	Day 2 am 3 hours
V.	Rating scales and scoring Alternative scaling and scoring methods Exercise - development of rating scales and scoring standards	Day 2 pm 3 hours
VI.	Selecting and training oral examiners Sources of raters and guides for selecting raters Rater training - guidelines, topics, and examples Rater training exercises	Day 3 am 3 hours
VII.	Oral test administration Scheduling Facilities and equipment Candidate notification and processing Staffing Candidate orientation and study periods Examination monitors Multiple panels and alternate test forms Taping Test security Test administration procedures Exercise - Conduct of an oral examination (using the questions and scoring procedures from unit IV and V exercises)	Day 3 pm 2 hours
VIII.	Evaluation of the oral examination Review of test statistics Feedback and information on exam effectiveness Examination review sessions	Day 3 pm 1/2 hour
IX.	Other topics of interest to participants, and course evaluation and feedback Cost issues Problems and solutions Other topics Course evaluation	Day 3 pm 1/2 hour

The three-day seminar can be offered to a maximum of 25 participants.

A one-day summary version of the training is also available. The one-day training program excludes the practical "hands-on" exercises, and provides very limited time for discussion

Contact

For more information about training programs and costs, contact:

IPMA-HR
1617 Duke Street
Alexandria, VA 22314
Phone: (703) 549-7100
Fax: (703) 684-0948
Email: ipma@ipma-hr.org