Originally published: . Volume 13, No. 5. Source pages: 2.
by J. Nelson In January, Marc Zimmermann, Gig Costelloe, and I decided to push the issue regarding proper selection testing at the state board. Readers who have been following this story in recent issues of the Times know that the impetus for the problem is the finding that the EPPP has adverse impact on African-Americans. We filed a complaint that the Board did not have the skill set to handle the selection testing problems that have emerged.
We received a response to our complaint that stated the board members are “uniquely qualified" to do whatever it is they do, presumably including designing selection programs. At the same time, they said the matter had nothing to do with their practice of psychology, to which we might ask, “Then whose practice of psychology does it have to do with?" However, they said they were open to looking at the problem and to send them whatever we have.
So we sent them copies of Brian Sharpless’ articles showing adverse impact. Next we’re sending them some articles, including a copy of Samuel Messick's classic, “Validity of Psychological Assessment: Validation of Inferences from Persons Responses and Performances as Scientific Inquiry into Score Meaning."
We’re also going to send them copies of some EEOC guidelines indicating that once adverse impact is found you’re in a different ball game.
I frankly don’t know how to educate board members by sending them bits and pieces of science through the mail. It’s not like Zimmermann, Costelloe, and I learned this overnight. We have a century of combined experience in the trenches of preemployment selection testing, in a region of the country where we deal with adverse impact and its moral challenges.
In their letter, the board asked if we knew of another test without adverse impact to let them know. This question indicates that the members are not thinking in the big picture. Predictive validity and fairness are not in the test itself but how the tests are used. So here are a few tidbits of wisdom about designing selection programs.
It is not a good idea to use a single hurdle. You should use a multi-test battery that includes low or no adverse impact measurements.
You should set your cut off, if you have to use one at all, very carefully. Keep in mind the absurd idea that someone at the 49th percentile is considered unqualified while someone at the 50th is considered qualified.
It would be a good idea to come up with creative ways, that do not negatively impact your predictive validity, that reduce adverse impact. For instance, remedial training for those scoring in the low average range.
Content validity is not enough, especially with only 2.6% African-Americans contributing to the task analysis, as is the case with the EPPP.
In private industry people are competing for a scarce resource–a job position. The board has no such similar problem. They can give as many licenses as they want, so there is little danger of reverse discrimination.
These are just a few ideas to consider when designing complex selection systems. It is by no means an exhaustive list of issues to be addressed by the applied psychological scientist on the road to validity.
This article is part of a historical newspaper archive. Read the original issue for its original layout, photographs and graphics.