Last month the Association of State and Provincial Psychology Boards (ASPPB) quietly posted a message that the new Part 2 of the national licensing exam will officially launch on January 1, 2020. ASPPB officials first announced the new test, the EPPP-2, in 2017 as optional for its member jurisdictions. Then as resistance mounted, ASPPB’s Board of Directors decided that they would combine parts and make the entire exam mandatory, this coupled with a 100% fee increase. An outcry followed, then ASPPB backpedaled and made the roll-out optional––for the time being.
The upcoming January launch begins a “voluntary adoption” program, a carrot and stick for the controversial new test product, whose scientific basis is coming under more and more scrutiny.
The new test is optional––but whether it remains that way is highly doubtful, some say. In this article we review the behind-the-scenes decisions and interests impacting the test and those hoping to become licensed psychologists.
The National Exam
The current test, called the Examination for Professional Practice in Psychology or EPPP, is the national licensing exam required for candidates seeking a state psychology license.
ASPPB purchased the rights to the exam sometime around 2013, and since then the EPPP is the top money making product for the non-profit corporation. The EPPP-2, first priced at $600 then lowered to $450, would increase testing revenues for ASPPB by 75%, boosting the firm’s yearly income by $3,750,000.
The current EPPP is expensive at $600 plus administration fees. At a recommended 50th percentile cut-off, many candidates have to take the test more than once. The test contains 225 items, with a fourhour time limit. To compare, physicians pay $605 for an eight-hour exam, and Social Worker candidates pay about $250 for a 170-item exam.
On-going criticisms about the scientific validity, the practical usefulness of the new exam, and the possible discriminatory impact of the entire EPPP selection approach, appear to have done little to deter ASPPB from its goal.
In the latest of a list of scientists voicing concerns, researchers lead by University of North Texas professor Jillian Callahan, PhD, are set to publish a critique in the flagship journal of the American Psychological Association, The American Psychologist.
Based on a pre-publication draft of the article, the authors will be addressing the need for stronger scientific methods in the EPPP-2 development, the suitability of the test for its intended use, impact on minorities, and legal vulnerabilities.
ASPPB has gone through several roll-out efforts, first to persuade and encourage member jurisdictions to accept the new test, and then to force the new exam on states. The current effort, “voluntary adoption,” includes a fee of only $100 for Part 2 of the exam for “Beta Candidates.” After the “beta exam” closes in 2021, this fee will be $300 for early adopters. After January 1, 2022, the fee increases to $450.
It is not clear what happens to those states who refuse to accept the EPPP-2 for its candidates. ASPPB officials note, “… At this time, it is optional for licensing boards (jurisdictions) to sign on to require the EPPP (Part 2 – Skills).”
Since ASPPB owns the tests, they will likely make the combined exam mandatory again, said one insider.
Only nine of 64 possible jurisdictions have joined in to “adopt” the additional exam so far, totaling only 14% of ASPPB “members.” These are Arizona, Guam, Nevada, Newfoundland and Labrador, and Prince Edward Island. Starting in February, Missouri has signed on and starting in March, Manitoba has signed on as early adopters. Finally, Georgia has agreed to be an early adopter starting November 2020.
Show Me the Money
The ASPPB is a private, nonprofit, 501(c) tax-exempt corporation located in Tyrone, Georgia. The company states its mission is to “Facilitate communication among member jurisdictions about licensure, certification, and mobility of professional psychologists.” The “members” are about 64 regulatory boards from across the United States and Canada. These boards pay dues to be a member of ASPPB.
Tax records indicate that ASPPB grossed $6,686,286 in 2017; $5,973,841 in 2016; and $5,284,952 in 2015.
Total revenue for 2017 was $6,645,731 and for 2016 was $5,933,473. For 2015, revenues were $5,254,097.
Over the last five years, from 2012 to 2017, total revenues have increased from $4,274,419 to $6,645,731 or 55%.
Assets and balances for 2017 were listed at $8,629,194. In 2016 assets totaled $8,462,637, and in 2015 totaled $7,712,532.
Of total revenues in 2017, ASPPB spent 2,268,203 on salaries and other types of compensation. Records indicate they have 12 employees and the highest compensated is the CEO, Dr. Steven DeMers, at $270,784. Another four employees’ salaries fall between $134,771 and $111,823. Board members receive between $6,800 and $12,800.
All listed compensation for 2017 together totals $839,747. An additional $1,098,096 was paid to Pierson Vue Minneapolis for exam administration.
To compare, in 2016 they listed 12 employees, again the most highly compensated was Dr. DeMers at $243,842. Others fell between $131,949 and $125,860.
ASPPB’s main income producing product is the national exam for psychologists, with revenue of $5,378,524 in 2017. This was 80% of total revenues for the year.
In 2016 exams and related fees grossed $5,296,421, or 89% of all ASPPB venues. In 2015 this amount was $4,775,213 and in 2014 it was $4,826,421.
The company has some other products, such as the Psychology Interjurisdictional Compact (PSYPACT), a service to coordinate psychologists working across state lines. This product generated $357,708 in 2017.
The organization spends liberally on the other activities including $1,169,743 on travel, $978,143 on other salaries and wages, $240,951 on other employee benefits, $375,418 on information technology, and $240,143 on conferences.
While many members are government officials, ASPPB does not follow open meetings laws. Deliberations and decisions are private. “If you are not a member or staff of an ASPPB Member Psychology Regulatory Board or an individual member, you are not eligible to access this section of our website,” they write. Their conferences are also closed and for members only.
This arrangement––where a corporation, formed of state board representatives, operates as a test publisher, with influence and special access to government officials, and also a captive market––seems ripe for conflict of interest. The Times asked one CPA to look over the information and he said, “Of course there is influence and COI.”
“With a lot of cash sitting on the balance sheet, the strategy is to maximize expenses,” said an MBA in reviewing the information for the Times. “The extra profits are likely to go into perks rather than price cuts,” he said.
Scientific Criticisms Continue to Mount
In the latest of a series of criticisms, University of North Texas professor Jillian Callahan, PhD, and coauthors will address concerns about the scientific quality of the new exam in an upcoming issue of The American Psychologist.
In the pre-publication draft posted on the internet, the authors write, “… the EPPP Part 2 has yet to be subjected to a broader validation process, in which the suitability of the test for its intended purpose is evaluated. Implementation of the EPPP Part 2 before validation could have negative consequences for those seeking to enter the profession and for the general public …” And, “For jurisdictions implementing the EPPP Part 2, failure to gather and report the evidence required for use of a test in a forensic context may also open the door for legal challenges.”
Other critics have pointed to similar problems, one being the lack of the need for additional test hurdles.
“There is no evidence that the public is facing some sort of previously unheard of crisis in terms of safety from currently practicing psychologists,” said Dr. Amy Henke. In 2016, while serving as a director for the Louisiana Psychological Association, Henke took the lead to pass a Resolution opposing the new test. She pointed out that multiple checks on competency already exist for psychologists and appear to be working to protect the public.
“Trainees are already held to high standards through a variety of benchmarks,” Dr. Henke wrote in the Resolution, “including but not limited to: APA approval of doctoral programs, multiple practicums where competency is repeatedly assessed, completion of formal internship training (also approved and regulated by APA and APPIC), and supervised post-doctoral hours obtained prior to licensure. There is no evidence to suggest this is not sufficient for appropriate training.”
Henke and others pointed to existing multiple hurdles that candidates already must clear, including two year’s supervision, a written exam, oral exam, background check, and jurisprudence exam. Additionally, the law allows the board to require additional physical and psychological assessments whenever needed.
However, Dr. Emil Rodolfa, from Alliant University and also then a program developer at ASPPB, questioned if these standards are enough, saying that supervisors have “… difficulty providing accurate evaluations of their supervisees to others who may have to evaluate the supervisee’s competency.”
Henke also said, “I am particularly concerned about regulatory boards encroaching ownership of training standards. The goal of a regulatory board, in my personal opinion, is to provide the least restrictive amount of guidelines possible in order to protect the safety of the public.”
Rodolfa disagrees and said, “Licensing boards have a mandate to ensure that the professionals they license are competent. Competence is comprised of the integrated use of knowledge, skills, attitudes and values.”
Henke and others point out that the evidence from disciplinary statistics suggests that problems are very rare. For the most recent year with records, total reported disciplinary actions across the U.S. and Canada range from 159 to 222, with only nine to 17 licenses being revoked nationally. (See table.) Data from the ASPPB Disciplinary Data System: Historical Discipline Report show rates of disciplinary actions for psychologists to be consistently low. For an estimated 106,000 psychologists nationwide, the disciplinary rates remain around 1–2 per 1,000.
Louisiana’s rate is similar to the national average. For the year 2018–2019 there were two disciplinary actions. For the year 2017–2018 there were also two disciplinary actions. And for the year 2016–2017 there were three disciplinary actions and in 2015– 2016 there was one disciplinary action which is on appeal. And from 2014–2015 there was one disciplinary action.
Critics argue that a second test can have very little impact on such a low disciplinary rate.
Other criticisms center around the poor scientific quality of the test for making high-stakes decisions about candidates’ careers. In 2009 Brian Sharpless and Jacques Barber authored “The Examination for Professional Practice in Psychology (EPPP) in the era of evidence-based practice,” for Professional Psychology: Research and Practice.
“Professional psychology has increasingly moved toward evidence-based practice,” said the two authors. “However, instruments used to assess psychologists seeking licensure, such as the Examination for Professional Practice in Psychology (EPPP), have received relatively little empirical scrutiny.” They write, “… there is a paucity of criterion, predictive, and incremental validity evidence available.”
Dr. DeMers responded in the same journal attempting to clarify issues and giving some information not published. He agreed with some of the recommendations, according to the summary of his article.
Industrial-Organizational Psychologist Dr. William Costelloe, Chair of the I-O and Consulting Psychology Committee of LPA, told the Times, “… predictive validation studies must be conducted.” This type of research proof is not optional, he said. “Well conducted, scientifically based predictive validation studies must be conducted if the EPPP2 is intended to be used as a selection tool,” Costelloe said.
In April 2018, ASPPB CEO, Dr. Stephen DeMers, met with members of the Louisiana State Board of Examiners of Psychologists and representatives of Louisiana Psychological Association (LPA). After the meeting, Dr. Kim VanGeffen, Chair of LPA Professional Affairs, said, “Dr. DeMers acknowledged that, currently, there is not really any research on the validity of the EPPP-2,” VanGeffen said. “There do not seem to be any plans to obtain predictive validity nor does the EPPP2 committee believe that establishing this type of validity is necessary,” she said.
Dr. Marc Zimmermann, past LSBEP board member and Chair of the LPA Medical Psychology Committee, also attended. “He [Dr. DeMers] stated that there is no predictive validity,” said Zimmermann. “He also threw in that none of the national tests had predictive validity. He reported that content validity was the accepted standard because a test with predictive validity could not be constructed,” said Dr. Zimmermann. “… DeMers had the temerity to try to sell us something that does not meet the standard that psychological tests being published are expected to have.”
Other critics are concerned about the discrimination aspects of the EPPP. In a December 2018 study of New York psychologist candidates, Brian Sharpless, PhD, demonstrated that the EPPP has differing fail and pass rates for different races. Blacks had a failure rate of 38.50% and Hispanics had a failure rate of 35.60%. Whereas, Whites had a failure rate of 14.07% and Asians had a failure rate of 24%. The difference is large enough for African-Americans and Hispanics to constitute discrimination.
The study, “Are demographic Variables Associated with Performance on the Examination for Professional Practice in Psychology (EPPP)?” is published in The Journal of Psychology: Interdisciplinary and Applied.
ASPPB’s Rough Roll-Out
Keeping its members cooperative with its product plans has been difficult for ASPPB. In 2016 the firm announced the EPPP–2 and told its members, licensing boards across the United States and Canada, that the use of the new test would be “optional.”
However, amid criticisms ASPPB did an about face in late 2017 and announced that the new exam would be mandatory after all, and be combined with the current test. And, the price would increase 100%, from $600 to $1200.
“The ASPPB Board of Directors, based on a number of factors, including feedback from our member jurisdictions and input from our legal counsel, has determined that the EPPP Part 2 is a necessary enhancement, and therefore an essential component of the EPPP,” wrote DeMers.
Objections mounted, mostly from student and early career psychologist organizations.
In July 2018, Dr. Amy Henke, now serving on the Louisiana State Board of Examiners of Psychologists (LSBEP), and LSBEP members of sent a blistering letter to the ASPPB Board of Directors, to the ASPPB members, and to the administrators of state psychology boards across the US and Canada.
Objections from Henke and others involved technical and scientific issues, but also the criticism that there is no problem that the new test needs to solve.
“LSBEP does not believe that data exists demonstrating that psychologists are not already held to high standards of competence,” they wrote. “The data that exists in terms of complaints and disciplinary actions toward psychologists also does not support the theory that competency problems abound in the field of psychology.”
The LSBEP also criticized ASPPB’s role and said that the decision is “…an overstep.”
“We are concerned that ASPPB has lost sight of their original mission, which from this board’s understanding was limited to facilitating communication between various member jurisdictions,” the LSBEP members pointed out, and that mandatory decisions on EPPP-2 do not fit this role but rather the role of a vendor providing a product.
Following this, in August 2018, ASPPB President Sharon Lightfoot, PhD, announced that the ASPPB Board of Directors voted to rescind their 2017 decision to mandate the second exam.
“We will continue toward launch of the Enhanced EPPP in 2020,” Lightfoot said, “and make it available to states and provinces interested in serving as early adopters. We are lifting the requirement for use of the Enhanced EPPP and are lifting the deadline for implementation.”
In December 2018, ASPPB decided to use a carrot and stick approach for the new exam. According to an October 24, 2018 letter from Lightfoot, if Louisiana chooses to decline the use of EPPP-2, individuals here will not be allowed to take EPPP-2 even if they wish to do so.
“Only applicants who are registered through a jurisdiction that has adopted the Enhanced Exam, and who have passed the knowledge portion of the exam, will be allowed to take the skills portion of the exam,” said Lightfoot.
Also, those test-takers from compliant states will pay reduced fees as a reward for early adoption of the additional exam, while those from late adopters will pay $450.
Sources at the Louisiana State Board of Examiners believe ASPPB is forcing states to use the EPPP-2 by prohibiting individuals from taking the exam in a state which does not require its use. They say this would make it difficult for psychologists who obtain licensure in a state which does not use the EPPP-2 to obtain licensure in a state which does use the EPPP-2. This policy, if adopted, is punitive, they say.
Is Resistance Futile?
ASPPB appears to be doggedly maintaining it’s commercial course, despite the mounting criticisms that the second exam is not scientifically well-constructed or actually needed for public safety. One source close to the state board said they see very little way to avoid having to accept the new exam eventually–– that efforts to stop ASPPB were futile.
If critics are correct, and the second exam is wasteful spending, the cost and additional regulatory hurdle will be born entirely on the backs of new psychology license hopefuls.