Cheryl Berryman
How are we reading Arizona’s COVID-19 metrics?
A short two-day survey
Summary
July 12, 2020
Most people read graphs based on what is presented in front of them. Less than half of those surveyed could correctly interpret graphs with more than one data set. Complex graphs can be misleading if used with unrelated data, especially if one group visually overshadows the other.
​
It would be beneficial to isolate unrelated metrics to individual graphs. Summarized line graphs should be used more often, as they are easier to scan. Incomplete or pending data sets could be emphasized by color markers.
Introduction
Are you able to tell how these graphs are trending from top-left to bottom-right?
These are Arizona's monthly COVID-19 numbers from March 1, 2020 to June 28, 2020.
PCR tests are mostly increasing.
Serology tests are increasing then decreasing.
Percent positive cases are mostly increasing.
Total tests performed are mostly increasing.
The Problem
Arizona Department of Health Services's (AZDHS) COVID-19 metrics is a heated topic. Given that the data released to the public varies each day and is being debated on whether cases are increasing or decreasing, we wanted to see how two COVID-19 graphs from the state's press briefings are generally being read.
The charts on the left came from the state's COVID-19 news briefing on June 29, 2020. At that point it was abundantly clear that our cases were trending upwards.
For the purposes of this study, case data from March 1, 2020 to July 4, 2020 was extracted from AZDHS's dashboard and recreated.
Research Goal
Determine how trends and forecasts are generally identified for two types of AZDHS graphs (stacked bar and combination).
Method
​
29 MTurk respondents were recruited for a 10-minute survey.
A hamburger eating competition between imaginary characters, Bob and Gene, served as a cover story for the data.
​
​
Bob and Gene represented PCR and serology, respectively. The number of burgers eaten represented positive cases and the number of burgers ordered represented tests administered.
Participants were instructed to look at the "overall picture" despite variations and to consider each graph independently. Respondents then determined how the graphs appeared to increase or decrease over time and forecasted how the graph would continue to trend.
Results
Survey performance did not significantly differ between stacked bar graphs and combination graphs.
Overall, 46% of participants were able to correctly identify trends and forecasts on both graphs.
Interestingly, respondents were evenly split between interpreting the graph in detail or looking at the bigger picture.
For example, in the stacked bar graph, you could say that over time it appears to be increasing in the middle and decreasing toward the end. But based on our reality, we know the graph is showing a slow increase in total tests. The last 4–7 days are still being filled throughout the week.
Zooming out to a monthly view, this becomes clear: PCR tests are increasing, and serology tests are increasing then decreasing.
AZDHS's combination graph is a better summarized visual, but the right tail also has incomplete data and it only features half of the implicated picture.
While the blue area represents the number of PCR tests administered, the orange line graph represents the percentage of positive cases from PCR and serology tests.
Compared to the stacked bar graph from earlier, respondents made more inferences between the two groups of data.
69% believed the percent of positive cases on the graph was solely based on the number of PCR tests performed.
48% believed increased PCR testing caused increased positive cases based on this graph.
Conclusion
On average, extracting information from graphs comes naturally if the answer is easy to find. Assumptions can only be made based on present evidence. For Arizonians who may be interpreting AZ COVID-19 metrics as a downward trend, we can close the knowledge gap by explicitly noting where the inconsistencies are in the data.
This is one way we can move forward together through this crisis.
Suggestions
​
Issue 1: Combining unrelated data into one graph will mislead people into thinking everything on the graph is related.
​
-
Solution 1: Because positive cases reported from AZDHS include serology numbers, separate positive case percentages from PCR test numbers into individual line graphs.
​​
Issue 2: Incomplete data will mislead people into believing we are trending downwards.
​
-
Solution 2a: Mark pending data sets by lightening the color or highlighting the discrepancy. Recent news briefings have vastly improved in this area for case-by-day graphs. This method should be applied to all graphs with incomplete data.
-
Solution 2b: Map out forecasts after known cases have been filled. This will help set expectations and improve communication.
​​
Issue 3: Daily bar graphs are difficult to trend and appear to be cluttered.
​
-
Solution 3: Provide more weekly overviews of the data to reduce cognitive load during news briefings. Similar to solution 1, separate out the data into simple, linear graphs.
​
​