Originally published: . Volume 9, No. 4. Source pages: 1, 9, 10.
Science & Education LSU’s Dr. Zhang Helps Consulting Psychologists with Data Visualization
Louisiana State University Assistant Professor Dr. Don Zhang is taking on a decades old problem––how do psychologists meaningfully communicate their findings to others, so that the value of research is clear and used properly?
Dr. Zhang was highlighted by author Michael Litano in two recent issues of The Industrial Psychologist, published by the Society of Industrial- Organizational Psychologists. Litano interviewed Zhang and other experts for, “Lost in Translation: Visually Communicating Validity Evidence.” The aim was to better describe and understand the critical role data visualizing efforts have for psychologists, who must find meaningful ways to visually and verbally communicate complex issues like validity, to clients, customers, students, and others.
Don Zhang, PhD, is an Assistant Professor in the Department of Psychology at Louisiana State University, was interviewed for part one, and helped author part two of the “Lost in Translation” article.
Zhang’s current research focuses on data visualization, specifically communicating validity information to relevant stakeholders. His research involves the application of psychological principles to the workplace. He is also interested in the individual differences in risk taking propensity and the relations to life and work outcomes such as job performance, counter productivity, and safety compliance. His work has been published in various international journals and books.
In “Lost in Translation,” Litano and experts point to the meaning of reliability and validity, and review the different types.
“In psychology, we are trying to measure more elusive characteristics such as personality, interests, et cetera, and we have to settle for a less precise scale than the bathroom scale,” Dr. Zhang said in the article. “In those situations, we use surveys or expert judges to measure characteristics that we cannot see. But our measurement instruments function very similarly. Like a bathroom scale, we want our psychological instrument to produce consistent results every time, which is a lot more difficult. Our job is to create the most accurate scale as possible even though we are trying to measure more elusive things.”
Regarding types of validity, Dr. Zhang said, “Content-related validity answers the question: ‘Is the content of my test relevant to the construct we are trying to measure?’” and used an NFL example, showing that all tests used need to be relevant to the construct of athletic ability.
“The 40-yard dash would be a better test than a hot dog eating contest because, in theory, speed is one aspect of athletic ability, and hot-dog eating is not. You also need to make sure all aspects of athleticism are measured: If you only use the bench press but do not ask the players to run, you are missing out on important aspects of a person’s athletic ability. […] Psychological measures work the same way. If a survey is designed to measure conscientiousness, it needs to have all the items related to the concept of conscientiousness.”
“Construct validity is the degree to which the instrument is measuring the construct it intends to measure,” he said. “It’s easy to look at the numbers on a bathroom scale and be confident that it is your weight. But if you get a 4.5 on a conscientiousness test, how can we be sure that the score reflects your conscientiousness and not something else?”
Do the test results predict behaviors in the real world? “This is called criterion validity,” Dr. Zhang said. “…we want to know is if a particular characteristic is related to the outcome of interest. Does athletic ability relate to success in the NFL? If they have nothing to do with each other, then we know that athletic ability is not important criteria when predicting NFL success.”
In part two of “Lost in Translation,” authors try and answer the question. “How does one effectively communicate validity evidence using only visualization?”
The authors noted the difficulty they had obtaining good examples of validity visualization from volunteer psychologists. Most were too academic and technical, confusing, or poor examples of validity.
For the second article Dr. Zhang takes on the challenge, using an example of how the information in a typical scatterplot is understood and then he compares the same data being explained in an expectancy table.
“The plot … [see Figure I] illustrates the relationship between ACT and college GPA, which has a validity of r = .30: not particularly impressive,” he wrote. “At a glance, the relationship depicted in the scatter plot appears equally unconvincing. Yet, scatter plots are the primary method for visualizing linear relationships.”
“Alternatively, one can use an expectancy chart […]. Expectancy charts communicates the relationship between two variables (e.g., ACT score and GPA) by presenting the proportion of the sample with score above a cut-off criterion (e.g., GPA above 3.5) at a given score interval on the predictor (e.g., ACT score between 25 to 27).”
“Believe it or not, the expectancy chart … [Figure 2] is generated with the same data as the scatter plot. Based on the expectancy chart, one can easily see the predictive efficiency of the ACT.”
“Students in the top quintile of ACT have approximately 70% chance making the Dean’s List (GPA above 3.5) at most universities, whereas students in the bottom quinile of ACT have a less than 25% chance (25% takes on a whole new meaning in this context).”
Dr. Zhang says that despite the benefits of these types of displays for communicating about validity, there are no accessible tools available.
So, he’s developed his own.
“In order to facilitate the calculation of nontraditional effect size displays, I’ve created a free-to-use web application that allows scholars and practitioners to easily generate and visualize a variety of nontraditional effect sizes such as expectancy charts, CLES, and binomial effect size displays with their own data,” Zhang says.
The site is Shiny Alternative Effect Size Calculator or ShinyAESC, found at https://dczhang.shinyapps.io/ expectancyApp/).
Don Zhang, PhD, Assistant Professor in the Department of Psychology at Louisiana State University, received his PhD in Industrial and Organizational Psychology from Bowling Green State University.
He works closely with organizations to create evidence-based plans and deliver solutions that are tailored to their unique needs. He can be reached at zhang1@lsu.edu (Twitter: @zdon89). His lab web page is https://sites01.lsu.edu/faculty/ zhanglab/
Dr. Don Zhang, LSU Assistant Professor, at the IO psychology track of the Louisiana Psychological Association in 2017. Dr. Zhang is helping psychologists find ways to better describe their validity results with data visualization methods. Figure I. Dr. Zhang’s example of traditional scatterplot, and a .30 correlation between ACT scores and GPA. “Not particularly impressive,” he points out in the TIP article. Figure II. Same data placed in expectancy chart. Clients can much more easily see the students in the top quintile of ACT have 70% chance of making Dean’s List while those In the bottom quintile have less than a 25% chance. (Charts courtesy of Dr. Zhang.)
This article is part of a historical newspaper archive. Read the original issue for its original layout, photographs and graphics.