Boston '96: A "Wicked Good" Conference

by Beverly Waldron

IPMAAC's 1996 conference in Boston has come and gone. I'm sure the 220+ who were there will agree with me when I say that Kris Smith and Warren Bobrow, along with the Program and Host committee members, did an excellent job of continuing the tradition of great conferences. Those of your who were unable to attend missed informative speakers, sessions, workshops, and discussions (formal and informal); a 20th birthday party; great host committee activities; and some cool Boston weather.

After last year's conference ACN Editor Dan Masden ran summaries of some of the conference proceedings in the ACN. It was a great idea and it's something we want to continue. It's impossible to cover the entire 4 day conference in a newsletter article but with the help of some friends we'll try and give you a summary of some of the conference happenings.

Things got started on Sunday with 8 half-day workshops and the IPMAAC Board meeting. The workshops covered a wide spectrum of assessment issues (a crash course in personnel selection, item writing, oral exams, in-house development of video tests, selecting commercial exams, basic and intermediate statistics, and a "courtroom" case study.) Later conversations indicated that the workshops were helpful and informative for those who attended. For information on the Board meeting be sure to read Marianne's President's Message. Not to be forgotten, Sunday night's opening reception provided everyone with an opportunity to talk with old friends and acquaintances and to make new ones. (As a side benefit I was also able to "obtain" commitments for several future ACN articles.)

Guest Speakers

The conference had general sessions featuring five highly respected guest speakers. Dr. Robert M. Guion (Distinguished Professor Emeritus, Bowling Green State University) was the keynote speaker for the opening session Monday morning. He shared his insights on the value and hazards associated with changes in personnel selection. His presentation title was "Innovation, Like Cholesterol, Can Be Good or Bad."

On Tuesday morning, Dr. Michael Campion, the WRIPAC invited speaker, shared findings from his latest research on improving the psychometric properties of the selection interview through structuring. Dr. Campion, who is a Professor of Management at Purdue University, also discussed the many ways interviews may be structured and provided recommendations for research and practice.

MAPAC's invited speaker, Dr. Lyle M. Spencer, spoke in general session on Wednesday morning. Dr. Spencer, Managing Director of the Hay McBer Center for Research and Technology, addressed the identification and assessment of competencies. He shared some of the latest research findings and addressed key issues in competency assessment including the economic value of competence, new concepts of competence which result in increased predictive power, and human resource information systems which can automate many components of the competency assessment process.

Thursday's closing session featured Drs. Marvin Dunnette and Leatta M. Hough. Dr. Dunnette is the Chairman of Personnel Decisions Research Institute and the founder of Personnel Decisions, Incorporated. Dr. Hough is a founder of Personnel Decisions Research Institute with a distinguished career helping organizations to develop advanced human resource management systems. They ended our conference with a relevant presentation on the future of personnel testing. Perspectives on how forces of change (technology, changing demographics, role of government, decreased funding, increased expectations, restructuring, downsizing, outsourcing, definition of customer, political & legal initiatives, and advances in our field) will impact on personnel testing in areas such as data bases, measurement methods, job/task analysis, variables measured, validity models, uses of information, "customer" reactions, and how we do our work were discussed.

(Note: We hope to have copies of the presentations or handouts from the guest speakers available on the IPMAAC home page.

Awards

On Tuesday morning a general session was held for the presentation of the Student Paper and Bemis Awards. T.R. Lin, chairman of the Student Paper Competition Award Committee, presented a certificate to the competition winner, Jaliza Cader of the University of Tennessee at Chattanooga, for her paper "Reactions of Simulated Job Applicants to a Personality Inventory." A certificate was also presented to Kathryn Carlisle Smith of the University of Akron for her paper "A Situational Judgement Test: Criterion and Construct Validity Evidence." In a future issue we will have summaries of these papers.

John Worcester, chairman of the Stephen E. Bemis Memorial Award Board, introduced this year's nominees (Jennifer French, Doris Maye, Terry McKinney, and James Sharf) and Anita Ford presented the 1996 Bemis Award to Jim Sharf. (For more information on this year's award winner and submitting nominations for the '97 award see the articles contained elsewhere in this issue of the ACN.)

Business Meeting and Presidential Address

On Wednesday afternoon, President Marianne Bays treated us to a brief business meeting. She banged the gavel a few times and let us know that financially IPMAAC is doing well. She also announced the election winners (see the President's Message) and recognized the 1996 committee chairs for their service. (She did call me a "charming nag" and I haven't yet figured out whether to be flattered or insulted. . .)

The brief business meeting allowed for an expanded "presidential address." In addition to hearing from Marianne Bays, IPMA President Wayne Etheredge addressed the group and presented a plaque of appreciation to Marianne. Their remarks were followed by presentations from two panels consisting of past IPMAAC presidents. In the first presentation, Nancy Abrams, Charles Sproule, Doris Maye and Bruce Davey discussed the APA Standards revisions including the responses and comments sent by the IPMAAC and IPMA review committees. (For more information please see the article Nancy prepared for the April issue of the ACN. Revising the Standards is going to be a long process and Nancy has promised to keep us updated with articles for the ACN.)

The second panel was composed of Ted Darany, Kaye Evleth, Jim Johnson, Chuck Schultz, and Joel Wiesen. They each addressed management and technical issues in personnel assessment about which they have a particular interest or concern. Briefly: Joel discussed affirmative action and fairness in testing; Ted talked about adding to business success, return on investment and the role personnel selection has to play in the process; Jim addressed the difference between technicians and professionals and the role that personnel specialists should be playing within their organizations; Chuck spoke on matching workload demands to resources; and Kaye focused on the improving, cooperative relationship between IPMAAC and IPMA. This was an interesting panel discussion and I hope to be able to get the thoughts and ideas presented in written form for inclusion in a future issue of the ACN.

We have included a picture that was taken at the end of the meeting which features some of the past presidents of IPMAAC. It's nice to know that so many of our former officers are still active in the organization.

Tutorials

Three half-day foundation track tutorials were offered during the regular conference. They included: "Job Analysis Made Easy," "Alternative Assessment Methods in Personnel Selection," and "The Basics of Designing Job Simulation/Assessment Center Exercises." These tutorials were well attended and received a great deal of praise. It is expected that similar foundation tutorials will be offered as part of future conferences. In addition a tutorial on "Mechanical Aptitude Testing" was offered. (Remember if your are interested in this type of training you don't have to wait for the conference. IPMAAC can provide training in these and other areas through our training workshops.)

Sessions

I asked a number of individuals to provide us with summaries/reviews on the presentations made during the conference. I didn't get as many as I had hoped for but am very grateful to those individuals who did provide us with information. Below you will find summaries for several of the conference sessions.

Effective Assessment of Writing Ability by Jean Tozer

David MacAlpine of the B-PAD Group and Jennifer French of Darany & Associates offered two strategies for testing writing skills: MacAlpine a direct assessment using video and French a simplified approach to a "traditional" indirect method.

MacAlpine described a video-based model developed for San Francisco International Airport. Airport Service Aide applicants were shown a brief "video-vignette" simulating a personal injury accident, then given 20 minutes to write a simple narrative report of their observations. Responses were evaluated only for accuracy and the clarity of communication; grammar, punctuation, sentence structure, spelling, etc. were not evaluated. The content of the video-vignette was derived from actual incident reports on file. The evaluation system was developed by having experienced Aides watch the video and write reports. From these reports, a consolidated list of 30+ report elements was compiled. Applicants' reports were compared to this consolidated list and awarded points for each one included. No points were subtracted for extraneous items.

This selection procedure proved very successful for the Airport and had no notable adverse impact. A similar procedure will soon be developed by B-Pad for the Ontario provincial government.

MacAlpine offered these tips for developing similar instruments:

use both trainers and end-users as SME's to develop scoring standards
use actual reports of critical incidents as a source for item development
shoot several re-takes of the video to allow for technical problems
give applicants instructions on the screen, in writing and orally

French reviewed the pros & cons of direct vs. indirect assessments of writing ability. Direct methods (such as the video-based measure above) have high acceptability with applicants due to strong face validity, but they are costly to administer and less reliable than indirect methods. Indirect measures are inexpensive to administer, but can be time consuming to develop.

French presented a literature review which summarized the validity coefficients for six types of indirect assessment as follows:

Paragraph organization	.458
Prose groups	.568
Error recognition	.592
Construction shift	.645
Sentence correction	.705
Usage	.707

French offered a strategy for significantly reducing the development time for Error Recognition items. Instead of having candidates read and identify the type of error, French recommends asking them to simply say whether the question is correct or incorrect. Work samples should be used to identify the appropriate reading level. Samples also provide a source of material for item sentences.

French has used this method successfully for entry-level police officer selection. When used as part of the assessment process, such items had an uncorrected correlation with training criteria of .29, compared with the total cognitive test and bio-inventory correlation of .50.

Dealing with Test Challenges by Charles Sproule

This symposium covered two topics. Suzanne Holloway of the Sacramento County California Human Resources Department presented on "Preparing for Civil Service Appeals." Phil Carlin of the Carlin Consulting Group presented on "The Personnel Specialists Role as Witness in Litigation."

Copies of papers on these topics are available from the presenters. Their addresses and phone numbers can be found in the 1996-1997 IPMAAC membership directory. A brief summary of what was covered in the presentations follows.

Preparing for Civil Service Appeals

Suzanne Holloway described the rules and procedures for appeals in Sacramento County, including reasons for appeals, timeliness requirements, contents of appeals, the appeal investigation report submitted to the Commission which will hear the appeal, and the report and recommendations prepared by the Human Resources Department. Guidance was provided for the selection specialist on how to prepare for the appeal, including information gathering tips and a format for preparations of a written response to the appeal for the information of the Commission.

Types of appeals were described including those related to examination development, passing scores, and examination administration. For each type of appeal, guidance was provided on topics to investigate and report.

The presentation included a list of the information which the selection specialist should gather prior to testifying at the appeal hearing and a list of questions to consider. Materials which should be taken to the hearing were listed and the hearing process was described. Guidance was provided on what to include in your appeal response presentation and how to make the presentation, such as: be brief, stick to the facts, stay focused, avoid defensive responses, state the reasons for your recommendations, and stay calm.

The Personnel Specialist's Role as Witness in Litigation

Phil Carlin provided advice and guidance on preparing to testify based upon his experience in litigation.

Some of the advice included:

begin to prepare right now
be accurate in your use of language, use full and complete sentences
be aware that your normal work products may be used in court, and prepare them on a day-to-day basis with this in mind
understand and correctly apply concepts, such as adverse impact or business necessity
testify only to that which you personally know, do not guess
practice your presentation
prepare and educate your attorney on personnel assessment, just as your attorney will prepare you on the legal procedures to be followed
be cautious of mistakes your attorney may make in questioning you, such as repeating a question previously asked by the attorney for the plaintiff

Those attending the session received a copy of a paper by David Friedland which elaborates on the above advice and contains additional guidance.

Strategic Succession Planning by Anne Smith Soileau

A good boss can make a lousy job "OK," but a lousy boss can ruin anything. Did you know the number one reason people leave their jobs is THE BOSS? This is one good reason we should assure that future bosses are good bosses and not just those employees who have lasted the longest.

Harry Brull of Personnel Decisions, Incorporated, defines succession management as "the process by which top management makes sure the right people are ready at the right time to lead the organization." In addition to assuring readiness for changing needs, strategic succession planning enables an organization to fill key openings from within without delay and with confidence. If you grow your own executives and managers, you will have fewer mistakes and fewer surprises.

After clarifying organizational goals and core competencies needed at the various levels in the organization, assessment and development of current employees are key ingredients to succession management. Once you know what you have and you know what you need, you must act to fill the gap. Although all employees need development, with limited resources invest in the large group of employees who are good where they are and the small group with the greatest potential. However, you should avoid self-cloning, neglecting the individual's desires, and writing off unpolished gems and tarnished stars. Best practices include ongoing substantive assessment of employees (not just when they apply for promotion), incentives for development and commitment to continuous improvement.

After identifying the superior employees to invest in, recognize that the most significant way to develop an employee is with challenging assignments. Managers as coaches is another key to development.

With the "maturing" of the Baby Boomers, skill in succession management will become a competency we all need.

New Frontiers in Recruiting, Assessing and Certifying Candidates for the Modern Office Support Job: Innovations from the State of Wisconsin by Alice Jackson

Alice Jackson and Stacie Meyer described a new process that combines a multiple choice examination and an objective inventory questionnaire in the selection process for their office support positions with Wisconsin State Government. The two examinations, entry and advanced office support, replaced 15 examinations that had overlapping test content. The test batteries assess applicant knowledge about basic office support functions through the use of a multiple choice examination and an inventory of potential job related skills, abilities, and interests. The new exam results in a database of individuals with a "test score" and 65 special qualifications (e.g., PC software, shorthand, job environment preferences such as reception duties, public contact, etc.), which enables agency users to customize or match applicant qualifications or interests with each position's job requirements. Additionally, the applicant has the opportunity to indicate interest in one or more job classifications in either the entry level or the advanced level examination. The two test batteries and inventories gather applicant qualifications that can be matched to job requirements for over 45 job classifications (16 entry level, 13 advanced level, and 18 supervisory level). One examination gathers information for all entry level vacancies while the other assesses advanced and supervisory level KSA's.

The register created from these examinations provides timely staffing for over 10,000 positions in the Wisconsin State Government civil service system. It is estimated that more than 200 vacancies will be filled each year using these two databases.

The presentation also addressed the extensive systems analysis and programming required to implement this innovative recruitment, examination, and selection process.

To date, each examination has been given five times with more than 3,000 applicants. The process has produced advantages for both applicants and agencies. Wisconsin sees this flexible recruitment, examination, and certification model as a forerunner to be "cloned" for other job classifications in its civil service system.

Building Customer Service into the Selection Process

Donna Terrazas made a presentation on a subject which we are all hearing a great deal about - improving the customer service of the selection process and doing things better, quicker, faster. Donna described a process which they have implemented at the East Bay Municipal Utility District that seems to be answering these demands. Working with their users, they identified problems and developed a streamlined process to address hiring department concerns that the selection process was overly cumbersome, non-responsive and time consuming. Key elements of the streamlining are:

provide a range of possible choices for recruitment and testing with defined benefits and consequences to the hiring supervisor;
provide high quality and timely service with pre-agreed time lines, deliverables and success criteria;
provide flexibility and choice while ensuring that the selection process is equitable and fair to all candidates;
share accountability for time lines, scheduling and recruitment/testing decisions with the hiring supervisor;
improve the continuity of service and information through assignment of a designated analyst to each large operating department or group; and
reduce time lines to conduct recruitment and exam processes through quarterly prioritizing and advance planning, pre-recruitment outreach for minority applicants, and clear definition of roles and responsibilities.

She provided a great deal of information on the implementation and impact of this program. Some of the positive impacts included:

greater department involvement and recognition of their role in the selection process;
improved ability to plan and prioritize the workload;
increased number of closed promotional exams;
improved affirmative action outreach efforts and results;
increased communication with line departments; and
an improved "image" for the Exam Unit.

Donna's presentation described a workable program that answers many of the complaints and demands being placed on personnel selection groups today.

ECN: The Electronics Communications Network

Marianne Bays, Jim Johnson, and Bill Waldron discussed IPMAAC's electronic communications network. Much of the material presented at this session is covered in this issue's From the Mouse Pad article so we will not duplicate it here.

(Note: many of the papers and handouts from the conference sessions will be available on the IPMAAC home page.)

Roundtables

Each day during the lunch break several roundtable discussions were held. This format is becoming very popular with conference attendees. The small, informal format provides for great interaction and some informative and lively discussions. I think we will find that it continues to be a welcomed regular feature of future conferences.

Committee Meetings

Committee meetings were held late Monday afternoon. I was a little disappointed that nobody showed up for the ACN committee meeting, but that was ok. I took the opportunity to drop in on the other committee meetings. I saw many of the same, reliable volunteers who help out year after year (we couldn't do it without them) and was pleased to see some new people joining in. But quite frankly, there weren't enough people at the committee meetings.

It's really very simple. IPMAAC functions through its committees. Committees function through their members. Without committee members the work doesn't get done. So pick a committee and get involved! (Not being at the conference is not a valid excuse for not joining up and helping out.) End of lecture/nag/sermon.

Birthday Party

On Tuesday afternoon WRIPAC threw IPMAAC a 20th birthday party. WRIPAC members set a party atmosphere by decorating the room with helium filled balloons, noise makers and tons of brightly colored streamers. (The occasional explosion of a balloon just added to the festive mood.) Everyone was treated to ice cream, cake and a truly original rendition of "Happy Birthday." Imagine if you will, a group of distinguished, professional WRIPACers wearing antennae of glittering balls, stars, and hearts singing happy birthday. Did I mention that they had inhaled enough helium to make them sound like a real "Mickey Mouse" chorus? We've included a picture so you can see how wonderful they looked. I'm just sorry we can't provide you with a sound bite so you can hear how "wonderful" they sounded. Thank you WRIPAC, it was a great birthday party!! (Even the hotel staff appreciated what a grand party it was. I overheard one of them saying "Expletive deleted, it's going to take us till midnight to clean this place up!")

Social

One of the important aspects of the conference is the opportunity to network with peers, visit with old friends, and make new ones. The Host Committee did their part to facilitate this by having several excellent group activities and providing information to those who wanted to explore Boston on their own.

Monday evening there was a trip to Boston's Computer Museum, a really neat place to visit. The group participated in a scavenger hunt requiring the use of the museum's computer hardware and software. Ten teams headed by "geeks" (poor word choice don't you think) set out to answer a series of questions. I don't want to make anyone feel bad by announcing the winning team, instead I will just say the best team won and those other "geek" (in this instance, a good word choice) team captains can protest all they want. (Way to go team!) Following a good meal, the group spent time exploring the museum and "surfing the net." A number of people took this opportunity to check out IPMAAC Online!.

Tuesday evening a large number of attendees took advantage of the Freedom Trail tour the Host Committee arranged. This walking tour gave participants a great way to see some of Boston's historic sites. The tour ended in Boston's North End and many took the opportunity to enjoy one of the fabulous Italian restaurants located there.

The Boston Red Sox vs the Cleveland Indians baseball game was on tap for Wednesday night and a large group made the trip to historic Fenway Park. Those who went saw a very competitive game that went into extra innings. At least three of us were still there at 1 a.m. when a home run in the bottom of the 15th gave the Red Sox the win. (Sorry about that to the IPMAAC Cleveland fans who were there.)

Final Notes

As I mentioned earlier, we all enjoyed the cool Boston weather. The highs were in the low 70's making for a refreshing break from summer's heat. (Coming from Florida, it meant I just had to make a trip to Fileen's Basement for a spring jacket. But that's ok, it'll make a nice winter coat later on this year!)

I think everyone will agree with me when I say that this year's conference was a success, both professionally and socially. I'd like to offer Kris, Warren, and their committee members congragulations on a successful conference and one more thank you for all their hard work in making the conference valuable and entertaining. THANK YOU!

It's not to soon to start thinking about the '97 conference. The conference will be held in Newport Beach, California, and the dates are June 22 - 26. Mike Willihnganz and David Dye, as co-Program Chairs, will soon be issuing the Call for Proposals. They would appreciate your assistance - prepare and submit a proposal or get in touch with them about helping out with the program committee. Mabel Miramon would also like to hear from you if you are willing to help out with the Host Committee. (Please, don't make me nag again. If you can help out, give them a call. They can be reached as follows: Mike Willihnganz: (916) 732-6051) or mwillih@smud.org; David Dye: (202) 606-3742 or david.dye@opm.gov; and Mabel Miramon: (909) 275-3559 or pers.pers2.mmiramon@co.riverside.ca.us)

If you're wondering about the "wicked good" in the title for this article, it's a New England term meaning truly excellent. (If you like regional expressions or want a translation for something you heard while in Boston, you might check out The Wicked Good Guide to Boston English.)

The Practice Exchange and Legal Affairs columns will be back in the October issue of the ACN.

One last, final note: to those of you who promised to send in articles for the ACN, I look forward to hearing from you in the near future.