With the daily AQI rosa chart and monthly AQI trend chart, Jaeri Suh, an MS-HCI student, conducted the initial design evaluation over two weeks. She recruited three participants from diverse age groups with limited knowledge of air quality and data literacy, representing our target audience. Simultaneously, she engaged three participants with a design background from Georgia Tech’s MS-HCI program. The evaluation aimed to understand how participants interacted with our data visualization, assessing their initial impressions and comprehension in line with our intended learning goals.
Before the interviews, Jaeri prepared a heuristic evaluation sheet for note-taking during the sessions. Subsequently, she conducted 30-minute semi-structured interviews with each of the six participants, prompting them to think aloud while interacting with the data visualization.
The interview questions were categorized into three sections for both the daily chart and the monthly trend AQI chart:
- Describe what you see on the chart and your interpretation.
- Identify any perceived issues or problems.
- Provide recommendations for improvement.
We initiated by asking about their initial impression of the chart. Following the participants’ interpretations, the interviewer revealed the correct information. Subsequently, participants were asked about the factors that caused confusion and suggestions for improving the design prototype.
The evaluation provided valuable insights for our research team. All six participants were unaware that the maximum pollutant value determines the day’s AQI. They expressed confusion about color similarities and differences. Additionally, two participants unfamiliar with air quality terms questioned the abbreviation “PM2.5,” highlighting the importance of avoiding assumptions about knowledge levels. Feedback regarding chart format was notable, with four participants suggesting a bar chart for conveying maximum values.
Regarding the monthly trend data, participants found the chart visually overwhelming, suggesting a need to reduce background illustrations and color usage. None of the participants recognized the rose charts and questioned the directionality of triangles. Furthermore, the interaction instructions for comparing daily AQI between dates using the rose chart were not effectively conveyed. At the interview’s conclusion, five out of six participants expressed a desire for a short introduction video before engaging with the data visualization, recognizing the general lack of awareness about AQI pollutants among most people.
In summary, our evaluation session helped us identify the need to first explain what AQI is and why it is created before delving into how it is computed. Providing related contextual information is essential. We shifted our design strategy from exploring different visualization representations to constructing narratives first. Consequently, our team plans to create an introduction video incorporating animations to provide a brief context on the Air Quality Index before engaging with the charts.