Georgia Tech @ CHI 2024
Alt Chi
Highlight on alt.chi
Queering/Cripping Technologies of Productivity
Sylvia Janicki, Alexandra Teixeira Riggs, Noura Howell, Anne Sullivan, Abigale Stangl
In this paper, we contribute three design manifestos that start from our queer, crip experiences to resist dominant designs and practices of productivity. Through our manifestos, we explore tensions in glitching three technologies of productivity (Mendeley, Figma, and ChatGPT) by reorienting their intended uses and design scripts. By sharing our perspectives and design processes, we invite new ways of relating to technologies of productivity, offer design provocations for queering and cripping technologies in HCI, and call for building intersectional coalitions that contribute towards a slow, non-linear resistance.
Case Studies
Case Studies: Activism
Balancing Expertise: Designing an Eviction Defense Web Tool with Legal Experts and Tenants
Chian-Jr Chiu, Xiao Luo, Betsy DiSalvo
This paper investigates a long-term collaboration between students and service organizations in a context that relies heavily on experts while engaging with a marginalized and gate-kept community. The project was motivated by the Atlanta Volunteer Lawyers Foundation (AVLF)’s initiative to address the eviction crisis in a southern U.S. city by leveraging the organizational expertise, the knowledge of computing and design graduate students, and engagement with those who have faced eviction. This paper chronicles the journey of a student team as they adapt to changing personnel and project objectives. It analyzes the distinctive challenges and opportunities encountered during this collaboration and offers recommendations for effectively planning, designing, and developing products that re- quire expert knowledge and for long-term student service projects.
Interactivity
Hector Fan, Jay Bolter
In this research, we investigate the potential of enhanced interactivity in providing new possibilities for interactive narratives within Virtual Reality (VR). We introduce “Rekindle,” a first-person narrative experience in VR. The narrative offers the interactor an embodied experience as a gay protagonist on a journey to reclaim lost memories, set against the backdrop of a dystopian future where a regime enforces stringent heteronormativity through the manipulation of memories. Central to this experience is the Memory Retrieval Mechanism, which empowers the interactor to explore and retrieve scattered memories. This activity reinforces immersion and the interactor’s connection within the narrative. By integrating coherent interactions grounded in narrative, “Rekindle” facilitates the creation of dramatic agency, thereby enhancing the narrative by making the story more compelling and consistent.
Journal
Environmental Activism
Post-growth Human–Computer Interaction
Vishal Sharma, Neha Kumar, Bonnie Nardi
Human–Computer Interaction (HCI) researchers have increasingly been questioning computing’s engagement with unsustainable and unjust economic growth, pushing for identifying alternatives. Incorporating degrowth, post-development, and steady-state approaches, post-growth philosophy offers an alternative not rooted in growth but in improving quality of life. It recommends an equitable reduction in resource use through sensible distributive practices where fulfillment is based on values including solidarity, cooperation, care, social justice, and localized development. In this paper, we describe opportunities for HCI to take a post-growth orientation in research, design, and practice to reimagine the design of sociotechnical systems toward advancing sustainable, just, and humane futures. We aim for the critiques, concerns, and recommendations offered by post-growth to be integrated into transformative HCI practices for technology-mediated change.
Social Activism B
“What’s Your Name Again?”: How Race and Gender Dynamics Impact Codesign Processes and Output
Judith Uchidiuno, Jaemarie Solyst, Jonaya Kemper, Erik Harpstead, Ross Higashi, Jessica Hammer
Creating technology products using codesign techniques often results in higher end-user engagement compared to expert-driven designs. Codesign sessions are typically structured in flexible and informal ways to achieve equal design partnerships, especially in adult-child interactions. This generally leads to better design output; however, it may also increase the enactment of socially constructed stereotypes and biases in ways that negatively affect the experiences of racial minorities and girls/women in design spaces. We codesigned a video game with a K-5 afterschool program located in a working-class, rural, predominantly white county over 20 weeks. We uncover ways that the codesign process and different activity types can create a permissive environment for enacting behaviors that are harmful to minorities. We discuss ways to manage and restructure codesign programs to be more conducive for children and adults from diverse backgrounds, ultimately leading to healthier design partnerships.
Late-Breaking Work
Pratham Mehta, Harsha Karanth, Haoyang Yang, Timothy Slesnick, Fawwaz Shaw, Duen Horng Chau
Surgical planning for congenital heart diseases requires a collaborative approach, traditionally involving the 3D-printing of physical heart models for inspection by surgeons and cardiologists. Recent advancements in mobile augmented reality (AR) technologies have offered a promising alternative, noted for their ease-of-use and portability. Despite this progress, there remains a gap in research exploring the use of multi-user mobile AR environments for facilitating collaborative cardiovascular surgical planning. We are developing ARCollab, an iOS AR application designed to allow multiple surgeons and cardiologists to interact with patient-specific 3D heart models in a shared environment. ARCollab allows surgeons and cardiologists to import heart models, perform gestures to manipulate the heart, and collaborate with other users without having to produce a physical heart model. We are excited by the potential for ARCollab to make long-term real-world impact, thanks to the ubiquity of iOS devices that will allow for ARCollab’s easy distribution, deployment and adoption.
Jamal Paden, Arpit Narechania, Alex Endert
During visual data analysis, users may inadvertently focus more on certain aspects of data, affecting analysis outcome(s). Existing tools primarily rely on visual cues (e.g., highlight already visited data) to increase user awareness of such analytic behaviors. We believe this single, visual modality is a passive form of guidance that adds to users’ cognitive load already engaged in analysis. We investigate how a dual modality (visual guidance and haptic feedback) can capture users’ attention and more actively guide them in their pursuits. We interface an existing visual data analysis tool with a gaming mouse. This enhanced system tracks user interactions and communicates biases by vibrating the mouse (haptic) and simultaneously displaying contextual information in the tool (visual). A formative study with nine users revealed that this dual modality can increase analytical awareness but the haptic mouse vibrations can be distracting and disturbing, informing the design of future multimodal user interfaces.
Embracing Embodied Social Cognition in AI: Moving Away from Computational Theory of Mind
Manoj Deshpande, Brian Magerko
As artificial intelligence becomes more integral to daily life, the need to design AI systems capable of understanding human interactions is increasingly important. This paper delves into the integration of social cognition in AI, tracing back to its historical foundations and examining seminal theories like Newell’s Bands of Cognition, Minsky’s Society of Mind, etc., which have emphasized the importance of social cognition since AI’s inception. We highlight the shortcomings of traditional computational theory of mind approaches, particularly in their failure to capture the embodied nature of social cognition. Advocating for including embodied socio-cognitive perspectives, we draw on theories such as Participatory Sensemaking and frameworks like Observable Creative Sensemaking. The paper further demonstrates the practical implementation of these concepts in AI through two case studies: one in co-creative dance AI and another in text-to-image generative AI systems.
Yuchen Zhao, Tulika Banerjee, Na Liu, Jennifer Kim
Maintaining a healthy lifestyle requires ongoing and consistent effort. To address this challenge, a variety of technologies, such as wearable devices, have been investigated to encourage individuals to adopt healthy habits. We introduce Grow with Me that combines an AR virtual pet with fitness tracking. In our application, the user’s goal achievement level is connected to the health and mood of the user’s virtual dog. As the user makes progress, their virtual dog learns new tricks and provides new interaction possibilities. We conducted a mixed-methods within-subjects study with 21 participants who used both the 2D and AR versions of our application. Our results uncovered that the AR pet elicited stronger emotional responses, and participants expressed a greater willingness to use AR as a motivational tool. In this paper, we will present our study and its findings, as well as the potential of AR as a powerful tool for motivating health-related actions.
Guttation Sensor: Wearable Microfluidic Chip for Plant Condition Monitoring and Diagnosis
Qiuyu Lu, Lydia Yang, Aditi Maheshwari, Hengrong Ni, Tianyu Yu, Jianzhe Gu, Advait Wadhwani, Haiqing Xu, Andreea Danielescu, Lining Yao
Plant life plays a critical role in the ecosystem. However, it is difficult for humans to perceive plants’ reactions because the biopotential and biochemical responses are invisible to humans. Guttation droplets contain various chemicals which can reflect plant physiology and environmental conditions in real-time. Traditionally, these droplets are collected manually and analyzed in the lab with expensive instruments. Here, we introduce the Guttation Sensor, the first on-site and low-cost monitoring technology for guttation droplets. This innovative device employs a paper-based wearable microfluidic chip capable of collecting and conducting colorimetric detection of six chemicals. We discuss this technology’s design and implementation, conduct evaluations on tomato plants, and envision how such a technology could enhance the human-plant relationship.
Zijie Wang, David Munechika, Seongmin Lee, Duen Horng Chau
Computational notebooks, such as Jupyter Notebook, have become data scientists’ de facto programming environments. Many visualization researchers and practitioners have developed interactive visualization tools that support notebooks, yet little is known about the appropriate design of these tools. To address this critical research gap, we investigate the design strategies in this space by analyzing 163 notebook visualization tools. Our analysis encompasses 64 systems from academic papers and 105 systems sourced from a pool of 55k notebooks containing interactive visualizations that we obtain via scraping 8.6 million notebooks on GitHub. Through this study, we identify key design implications and trade-offs, such as leveraging multimodal data in notebooks as well as balancing the degree of visualization-notebook integration. Furthermore, we provide empirical evidence that tools compatible with more notebook platforms have a greater impact. Finally, we develop SuperNOVA, an open-source interactive browser to help researchers explore existing notebook visualization tools. SuperNOVA is publicly accessible at: https://poloclub.github.io/supernova/.
The Social Construction of Generative AI Prompts
Atefeh Mahdavi Goloujeh, Anne Sullivan, Brian Magerko
As text-to-image AI tools grow in capability and widespread use, research has focused on studying individualistic user prompt crafting strategies. Recognizing that technologies are socially constructed, this paper examines prompt engineering through a social lens. We propose reframing prompt engineering as a socio-cultural practice shaped by collective knowledge building. Through qualitative analysis of 19 semi-structured interviews with members of the MidJourney community, a text-to-image generative AI tool, we identify four socio-engagement themes: proprietary/solitary, derivative, collaborative, and provocative prompting. These themes reveal a space of social engagement modes based on personal values and motivations from individual exploration to influencing the prompt community and highlight a fine line between being inspired by others’ prompts and maintaining creative ownership. We argue that understanding distinct social engagement preferences can inform the design of AI tools to facilitate transparent prompt reuse mechanisms, integrate collaborative features, or preserve ethical concerns about prompt sharing.
Jenna Kang, Emily Layton, David Martin, Thad Starner
Many products attempt to provide captioning for Deaf and Hard-of-Hearing individuals through smart glasses using automatic speech recognition. Yet there still remain challenges due to system delays and dropouts, heavy accents, and general mistranscriptions. Due to the imperfections of automatic speech recognition, there remains conversational difficulties for Deaf and Hard-of-Hearing individuals when conversing with hearing individuals. For instance, hearing conversation partners may often not realize that their Deaf or Hard-of-Hearing conversation partner is missing parts of the conversation. This study examines whether providing visual feedback of captioned conversation to hearing conversation partners can enhance conversational accuracy and dynamics. Through a task-based experiment involving 20 hearing participants we measure the impact on visual feedback of captioning on error rates, self-corrections, and subjective workloads. Our findings indicate that when given visual feedback, the average number of errors made by participants was $1.15$ less $(p = 0.00258)$ indicating a notable reduction in errors. When visual feedback is provided, the average number of self-corrections increased by $3.15$ $(p < 0.001)$, suggesting a smoother and more streamlined conversation These results show that the inclusion of visual feedback in conversation with a Deaf or Hard-of-Hearing individual can lead to improved conversational efficiency.
Panel
Panel: Sustainabilities and HCIs from the Souths
Sustainabilities and HCIs from the Souths
Vishal Sharma, Christianah Titilope Oyewale, Eldy Lazaro Vasquez, Eunice Sari, Joycelyn Longdon, Laura Cabrera-Quiros, Maryam Mustafa, Pushpendra Singh
Sustainability is a multifaceted concept with environmental, social, and economic dimensions, and its issues have manifested unevenly worldwide. Addressing sustainability requires nuanced global perspectives; however, conversations on sustainability within CHI have traditionally been centered on the Global North. This online-only panel at CHI 2024 attempts to shift the focus to the Global Souths to nurture a more pluriversal perspective on sustainability by understanding the meaning, means, politics, implications, and rhetoric of sustainability from the standpoint of the Souths, amplifying voices that are often unheard and under-represented in HCI. The panel aims to draw a global CHI audience to have meaningful discussions, build a truly international community, and collectively envision pathways addressing sustainability issues in and through HCI as they transcend geographical borders.
Paper
AI and Interaction Design
VAL: Interactive Task Learning with GPT Dialog Parsing
Lane Lawley, Christopher MacLellan
Machine learning often requires millions of examples to produce static, black-box models. In contrast, interactive task learning (ITL) emphasizes incremental knowledge acquisition from limited instruction provided by humans in modalities such as natural language. However, ITL systems often suffer from brittle, error-prone language parsing, which limits their usability. Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. By using LLMs only for specific tasks—such as predicate and argument selection—within an algorithmic framework, VAL reaps the benefits of LLMs to support interactive learning of hierarchical task knowledge from natural language. Acquired knowledge is human interpretable and generalizes to support execution of novel tasks without additional training. We studied users’ interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.
Assistive Technologies: Work – Independent Living with Neurodiversity
Designing for Strengths: Opportunities to Support Neurodiversity in the Workplace
Kaely Hall, Parth Arora, Rachel Lowy, Jennifer Kim
Supported employment programs have demonstrated the ability to enhance employment outcomes for neurodivergent individuals by offering personalized job coaching that aligns with the strengths of each individual. While various technological interventions have been designed to support these programs, technologies that hyperfocus on users’ assumed challenges through deficit-based design have been criticized due to their potential to undermine the agency of neurodivergent individuals. Therefore, we use strengths-based co-design to explore the opportunities for a technology that supports neurodivergent employees using their strengths. The co-design activities uncovered our participants’ current strategies to address workplace challenges, the strengths they employ, and the technology designs that our participants developed to operationalize those strengths in a supportive technology. We find that incorporating strengths-based strategies for emotional regulation, interpersonal problem solving, and learning job-related skills can provide a supportive technology experience that bolsters neurodiverse employees’ agency and independence in the workplace. In response, we suggest design implications for using neurodiverse strengths as design requirements and how to design for independence in workplace.
Ha-Kyung Kong, Saloni Yadav, Rachel Lowy, Daniella Ruzinov, Jennifer Kim
Securing employment and housing are key aspects of pursuing independent living. As these activities are increasingly practiced online, web accessibility of related services becomes critical for a successful major life transition. Support for this transition is especially important for people with autism or intellectual disability, who often face issues of underemployment and social isolation. In this study, we conducted semi-structured interviews and contextual inquiries with neurotypical adults and adults with autism or intellectual disability to understand common and unique goals, strategies, and challenges of neurodiverse adults when searching for employment and housing resources online. Our findings revealed that current interfaces adequately support practical (e.g., finance) goals but lack information on social (e.g., inclusivity) goals. Furthermore, unexpected search results and inaccessible social and contextual information diminished search experiences for neurodivergent users, which suggests the need for predictability and structured guidance in searching online. We conclude with design suggestions to make neurodivergent users’ online search experience an opportunity to demonstrate their independence.
Body and Wellbeing
Vedant Das Swain, Lan Gao, Abhirup Mondal, Gregory Abowd, Munmun De Choudhury
Algorithmic estimations of worker behavior are gaining popularity. Passive Sensing–enabled AI ( PSAI ) systems leverage behavioral traces from workers’ digital tools to infer their experience. Despite their conceptual promise, the practical designs of these systems elicit tensions that lead to workers resisting adoption. This paper teases apart the monolithic representation of PSAI by investigating system components that maximize value and mitigate concerns. We conducted an interactive online survey using the Experimental Vignette Method. Using Linear Mixed-effects Models we found that PSAI systems were more acceptable when sensing digital time use or physical activity, instead of visual modes. Inferences using language were only acceptable in work-restricted contexts. Compared to insights into performance, workers preferred insights into mental wellbeing. However, they resisted systems that automatically forwarded these insights to others. Our findings provide a template to reflect on existing systems and plan future implementations of PSAI to be more worker-centered.
Heidi Biggs, Shaowen Bardzell
We introduce the concept of disorientation as an emerging critical methodological strategy for design research in HCI. Disorientation is a phenomenological concept developed by queer feminist theorist Sarah Ahmed that acknowledges the spatio-embodied ‘orientations’ of societal and cultural norms and the queering potential of ‘disorientations’. We use humanistic close reading to analyze three examples from queer, feminist, and more-than-human work in HCI. Our interpretation focuses on how HCI researchers utilize disorientation as a methodological strategy for questioning norms of technologies as well as generatively, toward alternatives. We discuss the tenets of disorientation and several tactics we saw emerge in practice for other practitioners to build upon. Finally, we reflect on implications for the field, as disorientation requires vulnerability and willingness to undergo change, acknowledges embodied knowledge that emerges before interpretation, and suggests the possibility of generative and alternative orientations stemming from those epistemological commitments.
Understanding the Effect of Reflective Iteration on Individuals’ Physical Activity Planning
Kefan Xu, Xinghui (Erica) Yan, Myeonghan Ryu, Mark Newman, Rosa Arriaga
Many people do not get enough physical activity. Establishing routines to incorporate physical activity into people’s daily lives is known to be effective, but many people struggle to establish and maintain routines when facing disruptions. In this paper, we build on prior self-experimentation work to assist people in establishing or improving physical activity routines using a framework we call “reflective iteration.” This framework encourages individuals to articulate, reflect upon, and iterate on high-level “strategies” that inform their day-to-day physical activity plans. We designed and deployed a mobile application, Planneregy, that implements this framework. Sixteen U.S. college students used the Planneregy app for 42 days to reflectively iterate on their weekly physical exercise routines. Based on an analysis of usage data and interviews, we found that the reflective iteration approach has the potential to help people find and maintain effective physical activity routines, even in the face of life changes and temporary disruptions.
Communication and Collaboration
Daehyun Kwak, Soobin Park, Inha Cha, Hankyung Kim, Youn-kyung Lim
Designing user experiences for group recommendation systems (GRS) is challenging, requiring a nuanced understanding of the influence of social interactions between users. Using Spotify Blend as a real-world case of music GRS, we conducted empirical studies to investigate intricate social interactions among South Korean users in GRS. Through a preliminary survey about Blend experiences in general, we narrowed the focus for the main study to relationships between two users who are acquainted or close. Building on this, we conducted a 21-day diary study and interviews with 30 participants (15 pairs) to probe more in-depth interpersonal dynamics within Blend. Our findings reveal that users engaged in implicit social interactions, including tacit understanding of their companions and indirect communication. We conclude by discussing the newly discovered value of GRS as a social catalyst, along with design attributes and challenges for the social experiences it mediates.
Creativity: Visualizations and AI
Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools
Atefeh Mahdavi Goloujeh, Anne Sullivan, Brian Magerko
Generative Artificial Intelligence (AI) has witnessed unprecedented growth in text-to-image AI tools. Yet, much remains unknown about users’ prompt journey with such tools in the wild. In this paper, we posit that designing human-centered text-to-image AI tools requires a clear understanding of how individuals intuitively approach crafting prompts, and what challenges they may encounter. To address this, we conducted semi-structured interviews with 19 existing users of a text-to-image AI tool. Our findings (1) offer insights into users’ prompt journey including structures and processes for writing, evaluating, and refining prompts in text-to-image AI tools and (2) indicate that users must overcome barriers to aligning AI to their intents, and mastering prompt crafting knowledge. From the findings, we discuss the prompt journey as an individual yet a social experience and highlight opportunities for aligning text-to-image AI tools and users’ intents.
Data Visualization: Geospatial and Multimodal
DeepSee: Multidimensional Visualizations of Seabed Ecosystems
Adam Coscia, Haley Sapers, Noah Deutsch, Malika Khurana, John Magyar, Sergio Parra, Daniel Utter, Rebecca Wipfler, David W. Caress, Eric Martin, Jennifer Paduan, Maggie Hendrie, Santiago Lombeyda, Hillary Mushkin, Alex Endert, Scott Davidoff, Victoria Orphan
Scientists studying deep ocean microbial ecosystems use limited numbers of sediment samples collected from the seafloor to characterize important life-sustaining biogeochemical cycles in the environment. Yet conducting fieldwork to sample these extreme remote environments is both expensive and time consuming, requiring tools that enable scientists to explore the sampling history of field sites and predict where taking new samples is likely to maximize scientific return. We conducted a collaborative, user-centered design study with a team of scientific researchers to develop DeepSee, an interactive data workspace that visualizes 2D and 3D interpolations of biogeochemical and microbial processes in context together with sediment sampling history overlaid on 2D seafloor maps. Based on a field deployment and qualitative interviews, we found that DeepSee increased the scientific return from limited sample sizes, catalyzed new research workflows, reduced long-term costs of sharing data, and supported teamwork and communication between team members with diverse research goals.
Design Tools B
Bitacora: A Toolkit for Supporting NonProfits to Critically Reflect on Social Media Data Use
Adriana Alvarado Garcia, Marisol Wong-Villacres, Benjamín Hernández, Christopher Le Dantec
In this paper, we describe the design and evaluation of the toolkit Bitacora, addressed to practitioners working in non-profit organizations interested in integrating Twitter data into their work. The toolkit responds to the call to maintain the locality of data by promoting a qualitative and contextualized approach to analyzing Twitter data. We assessed the toolkit’s effectiveness in guiding practitioners to search, collect, and be critical when analyzing data from Twitter. We evaluated the toolkit with ten practitioners from three non-profit organizations of different aims and sizes in Mexico. The assessment surfaced tensions between the assumptions embedded in the toolkit’s design and practitioners’ expectations, needs, and backgrounds. We show that practitioners navigated these tensions in some cases by developing strategies and, in others, questioning the appropriateness of using Twitter data to inform their work. We conclude with recommendations for researchers who developed tools for non-profit organizations to inform humanitarian action.
Drone Interaction
Swarm Body: Embodied Swarm Robots
Sosuke Ichihashi, So Kuroki, Mai Nishimura, Kazumi Kasaura, Takefumi Hiraki, Kazutoshi Tanaka, Shigeo Yoshida
The human brain’s plasticity allows for the integration of artificial body parts into the human body. Leveraging this, embodied systems realize intuitive interactions with the environment. We introduce a novel concept: embodied swarm robots. Swarm robots constitute a collective of robots working in harmony to achieve a common objective, in our case, serving as functional body parts. Embodied swarm robots can dynamically alter their shape, density, and the correspondences between body parts and individual robots. We contribute an investigation of the influence on embodiment of swarm robot-specific factors derived from these characteristics, focusing on a hand. Our paper is the first to examine these factors through virtual reality (VR) and real-world robot studies to provide essential design considerations and applications of embodied swarm robots. Through quantitative and qualitative analysis, we identified a system configuration to achieve the embodiment of swarm robots.
Education and AI B
Yasmine Belghith, Atefeh Mahdavi Goloujeh, Brian Magerko, Duri Long, Tom McKlin, Jessica Roberts
As generative AI rapidly enters everyday life, educational interventions for teaching about AI need to cater to how young people, in particular middle schoolers who are at a critical age for reasoning skills and identity formation, conceptualize and interact with AI. We conducted nine focus groups with 24 middle school students to elicit their interests, conceptions of, and approaches to a popular generative AI tool, ChatGPT. We highlight a) personally and culturally-relevant topics to this population, b) three distinct approaches in students’ open-ended interactions with ChatGPT: AI testing-oriented, AI socializing-oriented, and content exploring-oriented, and 3) an improved understanding of youths’ conceptions and misconceptions of generative AI. While misconceptions highlight gaps in understanding what generative AI is and how it works, most learners show interest in learning about what AI is and what it can do. We discuss the implications of these conceptions for designing AI literacy interventions in museums.
Explainable AI
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Upol Ehsan, Samir Passi, Q. Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, Mark Riedl
Explainability of AI systems is critical for users to take informed actions. Understanding who opens the black-box of AI is just as important as opening it. We conduct a mixed-methods study of how two different groups—people with and without AI background—perceive different types of AI explanations. Quantitatively, we share user perceptions along five dimensions. Qualitatively, we describe how AI background can influence interpretations, elucidating the differences through lenses of appropriation and cognitive heuristics. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design. Carrying critical implications for the field of XAI, our findings showcase how AI generated explanations can have negative consequences despite best intentions and how that could lead to harmful manipulation of trust. We propose design interventions to mitigate them.
Highlight on Diversity In HCI
Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCI
Jordan Taylor, Ellen Simpson, Anh-Ton Tran, Jed Brubaker, Sarah Fox, Haiyi Zhu
LGBTQ+ people have received increased attention in HCI research, paralleling a greater emphasis on social justice in recent years. However, there has not been a systematic review of how LGBTQ+ people are researched or discussed in HCI. In this work, we review all research mentioning LGBTQ+ people across the HCI venues of CHI, CSCW, DIS, and TOCHI. Since 2014, we find a linear growth in the number of papers substantially about LGBTQ+ people and an exponential increase in the number of mentions. Research about LGBTQ+ people tends to center experiences of being politicized, outside the norm, stigmatized, or highly vulnerable. LGBTQ+ people are typically mentioned as a marginalized group or an area of future research. We identify gaps and opportunities for (1) research about and (2) the discussion of LGBTQ+ in HCI and provide a dataset to facilitate future Queer HCI research.
Designing an Archive of Feelings: Queering Tangible Interaction with Button Portraits
Alexandra Teixeira Riggs, Sylvia Janicki, Noura Howell, Anne Sullivan
How can tangible, wearable design encourage affective, embodied reflections on queer history? We expand Queer HCI scholarship, using queer theory to inform the design of wearable experiences that explore archives of gender and sexuality. Our project, “Button Portraits,” invites individuals to listen to oral histories from prominent queer activists by pinning archival buttons to a wearable audio player, eliciting moving personal impressions. We observed 17 participants’ experiences with “Button Portraits,” and with semi-structured interviews, surfaced reflections on how our design evoked personal connections to history, queer self-identification, and relatability to archival materials. We offer the following design directions: (1) designing tangible archives of feeling; (2) queering tangible, wearable interactions in design; (3) designing for personal, archival experiences; and (4) designing within difference. Through this work, we foreground queer stories to affect emotional reflections on marginalized histories, entangling the complex connections between bodies, feelings, histories, and shared queer experiences.
Social Justice in HCI: A Systematic Literature Review
Ishita Chordia, Leya Breanna Baltaxe-Admony, Ashley Boone, Alyssa Sheehan, Lynn Dombrowski, Christopher Le Dantec, Kathryn Ringland, Angela D. R. Smith
Given the renewed attention on politics, values, and ethics within our field and the wider cultural milieu, now is the time to take stock of social justice research in HCI. We surveyed 124 papers explicitly pursuing social justice between 2009 and 2022 to better reflect on the current state of justice-oriented work within our discipline. We identified (1) how researchers understood the social justice-relevant harms and benefits, (2) the approaches researchers used to address harm, and (3) the tools that researchers leveraged to pursue justice. Our analysis highlights gaps in social justice work, such as the need for our community to conceptualize benefits, and identifies concrete steps the HCI community can take to pursue just futures. By providing a comprehensive overview of and reflection on HCI’s current social justice landscape, we seek to help our research community strategize, collaborate, and collectively act toward justice.
Highlight on Health
Quantifying the Pollan Effect: Investigating the Impact of Emerging Psychiatric Interventions on Online Mental Health Discourse
Sachin Pendse, Neha Kumar, Munmun De Choudhury
Psychedelic-assisted therapy has shown significant promise in alleviating treatment-resistant mental illness, prompting excitement among people with lived experience of mental illness. The emerging collective perception of psychedelics as tools for mental health has been dubbed the Pollan Effect. We investigate whether the Pollan Effect carries to online community discussions concerning psilocybin-containing mushrooms (PCMs). Through a matched computational analysis of 676,875 longform Reddit posts describing PCM use spanning a decade, we provide evidence of the Pollan Effect in terms of increased health discourse around PCMs following two inception points—release of a book and subsequent documentary on PCMs. We then introduce the notion of a Pollan shift, which we witness through increased collective sharing of emotional and social experiences following the two inception points. Our findings offer insights into how online discourse could be representative of emerging social movements around new psychiatric treatments, and the role of platforms in sensemaking and research.
Highlight on Learning and Education
Michael Chang, Richmond Wong, Thomas Breideband, Thomas M Philip, Ashieda McKoy, Arturo Cortez, Sidney D’Mello
Emergent technologies like artificial intelligence have been proposed to address issues of inequity in schools, yet tend to ossify the status quo because they address needs within an already inequitable system. In this paper, we draw from speculative participatory approaches across HCI and the learning sciences, and present a novel approach to co-design that forefronts supporting historically minoritized youth in developing transformative agency to change their schools based on their valued hopes, practices, and concerns. We argue that when co-design spaces forefront relational development, expansive technological objects emerge as a byproduct. We present a case study of expansive dreaming with U.S. historically minoritized students about the use of artificial intelligence to support classroom collaboration. Methodologically, we demonstrate how physically visiting spaces of collective agency serves as a powerful perceptual bridge to imagining joyful, equitable possibilities for schooling. Our approach yields new visions for schooling and new metaphors for artificial intelligence.
Immersive Experiences: Design and Evaluation
Sungwon In, Eric Krokos, Kirsten Whitley, Chris North, Yalong Yang
The computational notebook serves as a versatile tool for data analysis. However, its conventional user interface falls short of keeping pace with the ever-growing data-related tasks, signaling the need for novel approaches. With the rapid development of interaction techniques and computing environments, there is a growing interest in integrating emerging technologies in data-driven workflows. Virtual reality, in particular, has demonstrated its potential in interactive data visualizations. In this work, we aimed to experiment with adapting computational notebooks into VR and verify the potential benefits VR can bring. We focus on the navigation and comparison aspects as they are primitive components in analysts’ workflow. To further improve comparison, we have designed and implemented a Branching&Merging functionality. We tested computational notebooks on the desktop and in VR, both with and without the added Branching&Merging capability. We found VR significantly facilitated navigation compared to desktop, and the ability to create branches enhanced comparison.
Learning and Teaching Technologies B
Christine Kwon, Darren Butler, Judith Uchidiuno, John Stamper, Amy Ogan
Despite the best intentions to support equity with educational technologies, they often lead to a “rich get richer” effect, in which communities of more advantaged learners gain greater benefit from these solutions. Effective design of these technologies necessitates a deeper understanding of learners in understudied contexts and their motivations to pursue an education. Consequently, we studied a 15-week remote course launched in 2021 with 17,896 learners that provided engineering education through a radio and phone-based system aimed for use in rural settings within Northern Uganda. We address shifts in learners’ motivations for course participation and investigate the impact of demographic features and motivations of students on persistence and performance. We found significant increases in student motivation to learn more about and pursue STEM. Importantly, the course was most successful for learners in demographics who typically experience fewer educational opportunities, showing promise for such technologies to close opportunity gaps.
Duri Long, Jiaxi Yang, Cassandra Naomi, Brian Magerko
Fostering learners’ interest remains an important challenge in computer science (CS) education. In this paper, we explore how creative music-making, tangible interfaces, and embodiment can be used toward this end. The primary contribution of this paper is Xylocode, a novel exhibit that introduces middle school age learners to computing concepts and fosters interest in CS via a tangible playspace for making music using an embodied simulation. We additionally present an in-museum evaluation of Xylocode with 29 middle school age children. Our results indicate that the exhibit fosters situational interest in computer science and leads to recognition of certain computing concepts, including arrays and global variables. Future research is needed to assess whether the exhibit leads to longer-term learning and/or interest gains and to explore why other computing concepts were not recognized by as many learners. We identify several implications and directions for future work based on our findings.
Mental Health and AI
Dong Whi Yoo, Hayoung Woo, Viet Cuong Nguyen, Michael L. Birnbaum, Kaylee Kruzan, Jennifer Kim, Gregory Abowd, Munmun De Choudhury
Early detection and intervention for relapse is important in the treatment of schizophrenia spectrum disorders. Researchers have developed AI models to predict relapse from patient-contributed data like social media. However, these models face challenges, including misalignment with practice and ethical issues related to transparency, accountability, and potential harm. Furthermore, how patients who have recovered from schizophrenia view these AI models has been underexplored. To address this gap, we first conducted semi-structured interviews with 28 patients and reflexive thematic analysis, which revealed a disconnect between AI predictions and patient experience, and the importance of the social aspect of relapse detection. In response, we developed a prototype that used patients’ Facebook data to predict relapse. Feedback from seven patients highlighted the potential for AI to foster collaboration between patients and their support systems, and to encourage self-reflection. Our work provides insights into human-AI interaction and suggests ways to empower people with schizophrenia.
Online Communities: Engagement A
Mapping the Design Space of Teachable Social Media Feed Experiences
K. J. Kevin Feng, Xander Koo, Lawrence Tan, Amy Bruckman, David McDonald, Amy Zhang
Social media feeds are deeply personal spaces that reflect individual values and preferences. However, top-down, platform-wide content algorithms can reduce users’ sense of agency and fail to account for nuanced experiences and values. Drawing on the paradigm of interactive machine teaching (IMT), an interaction framework for non-expert algorithmic adaptation, we map out a design space for \textit{teachable social media feed experiences} to empower agential, personalized feed curation. To do so, we conducted a think-aloud study (N=24) featuring four social media platforms—Instagram, Mastodon, TikTok, and Twitter—to understand key signals users leveraged to determine the value of a post in their feed. We synthesized users’ signals into taxonomies that, when combined with user interviews, inform five design principles that extend IMT into the social media setting. We finally embodied our principles into three feed designs that we present as sensitizing concepts for teachable feed experiences moving forward.
Online Communities: Engagement B
Observer Effect in Social Media Use
Koustuv Saha, Pranshu Gupta, Gloria Mark, Emre Kiciman, Munmun De Choudhury
While social media data is a valuable source for inferring human behavior, its in-practice utility hinges on extraneous factors. Notable is the “observer effect,” where awareness of being monitored can alter people’s social media use. We present a causal-inference study to examine this phenomenon on the longitudinal Facebook use of 300+ participants who voluntarily shared their data spanning an average of 82 months before and 5 months after study enrollment. We measured deviation from participants’ expected social media use through time series analyses. Individuals with high cognitive ability and low neuroticism decreased posting immediately after enrollment, and those with high openness increased posting. The sharing of self-focused content decreased, while diverse topics emerged. We situate the findings within theories of self-presentation and self-consciousness. We discuss the implications of correcting observer effect in social media data-driven measurements, and how this phenomenon shines light on the ethics of these measurements.
Politics of Datasets
Situating Datasets: Making Public Eviction Data Actionable for Housing Justice
Anh-Ton Tran, Grace Guo, Jordan Taylor, Katsuki Chan, Elora Raymond, Carl DiSalvo
Activists, governments, and academics regularly advocate for more open data. But how is data made open, and for whom is it made useful and usable? In this paper, we investigate and describe the work of making eviction data open to tenant organizers. We do this through an ethnographic description of ongoing work with a local housing activist organization. This work combines observation, direct participation in data work, and creating media artifacts, specifically digital maps. Our interpretation is grounded in D’Ignazio and Klein’s Data Feminism, emphasizing standpoint theory. Through our analysis and discussion, we highlight how shifting positionalities from data intermediaries to data accomplices affects the design of data sets and maps. We provide HCI scholars with three design implications when situating data for grassroots organizers: becoming a domain beginner, striving for data actionability, and evaluating our design artifacts by the social relations they sustain rather than just their technical efficacy.
Research Methods and Tools A
The Future of HCI-Policy Collaboration
Qian Yang, Richmond Wong, Steven Jackson, Sabine Junginger, Margaret Hagan, Thomas Gilbert, John Zimmerman
Policies significantly shape computation’s societal impact, a crucial HCI concern. However, challenges persist when HCI professionals attempt to integrate policy into their work or affect policy outcomes. Prior research considered these challenges at the “border” of HCI and policy. This paper asks: What if HCI considers policy integral to its intellectual concerns, placing system-people-policy interaction not at the border but nearer the center of HCI research, practice, and education? What if HCI fosters a mosaic of methods and knowledge contributions that blend system, human, and policy expertise in various ways, just like HCI has done with blending system and human expertise? We present this re-imagined HCI-policy relationship as a provocation and highlight its usefulness: It spotlights previously overlooked system-people-policy interaction work in HCI. It unveils new opportunities for HCI’s futuring, empirical, and design projects. It allows HCI to coordinate its diverse policy engagements, enhancing its collective impact on policy outcomes.
Research Methods and Tools B
Who is “I”?: Subjectivity and Ethnography in HCI
Tejaswini Joshi, Heidi Biggs, Jeffrey Bardzell, Shaowen Bardzell
HCI research applies ethnographic methods to understand and represent practices that involve the use of interactive systems. A subdomain of this work is interpretivist ethnography, which positions the researcher’s perspectival view [37] as central to ethnographic research and its epistemic contribution. Given this we ask: How might ethnographic researchers in HCI surface the meaning-making role of their subjectivities in research? We reflect on our prior ethnographic fieldwork on small-scale sustainable farms in Indianapolis, Indiana to bring the ethnographic “I” into focus by articulating our reflections as “impressionist tales” [64:101-124]. We ground this pursuit in sociologist Andrea Doucet’s concept of “gossamer walls” to surface researcher’s three reflexive relationships 1) with herself; 2) with participants; and 3) with her epistemic communities [34]. We build on and contribute to postmodern ethnography in HCI to clarify the epistemic virtues and methodological best practices of a more unapologetically subjective ethnographic practice in HCI.
Sound, Rhythm, Movement
Milka Trajkova, Duri Long, Manoj Deshpande, Andrea Knowlton, Brian Magerko
Co-creation in embodied contexts is central to the human experience but is often lacking in our interactions with computers. We seek to develop a better understanding of embodied human co-creativity to inform the human-centered design of machines that can co-create with us. In this paper, we ask: What characterizes dancers’ experiences of embodied dyadic interaction in movement improvisation? To answer this, we ran focus groups with 24 university dance students and conducted a thematic analysis of their responses. We synthesize our findings in an interconnected model of improvisational dance inputs where movement choices are shaped by interplays such as in-the-moment influences between the self, partner, and the environment as well as a set of generative strategies and heuristics for a successful collaboration. We present a set of design recommendations for LuminAI, a co-creative AI dance partner. Our contributions can inform the design of AI in embodied co-creative domains.
Supporting Accessibility of Text, Image and Video A
Lucy Jiang, Crescentia Jung, Mahika Phutane, Abigale Stangl, Shiri Azenkot
While audio description (AD) is the standard approach for making videos accessible to blind and low vision (BLV) people, existing AD guidelines do not consider BLV users’ varied preferences across viewing scenarios. These scenarios range from how-to videos on YouTube, where users seek to learn new skills, to historical dramas on Netflix, where a user’s goal is entertainment. Additionally, the increase in video watching on mobile devices provides an opportunity to integrate nonverbal output modalities (e.g., audio cues, tactile elements, and visual enhancements). Through a formative survey and 15 semi-structured interviews, we identified BLV people’s video accessibility preferences across diverse scenarios. For example, participants valued action and equipment details for how-to videos, tactile graphics for learning scenarios, and 3D models for fantastical content. We define a six-dimensional video accessibility design space to guide future innovation and discuss how to move from “one-size-fits-all” paradigms to scenario-specific approaches.
Supporting Communication and Intimacy
Zeyu Huang, Xinyi Cao, Yuanhao Zhang, Xiaojuan Ma
On online video platforms, viewers often lack a channel to sense others’ and express their affective state on the fly compared to co-located group-viewing. This study explored the design of complementary affective communication specifically for effortless, spontaneous sharing of frissons during video watching. Also known as aesthetic chills, frissons are instant psycho-physiological reactions like goosebumps and shivers to arousing stimuli. We proposed an approach that unobtrusively detects viewers’ frissons using skin electrodermal activity sensors and presents the aggregated data alongside online videos. Following a design process of brainstorming, focus group interview (N=7), and design iterations, we proposed three different designs to encode viewers’ frisson experiences, namely, ambient light, icon, and vibration. A mixed-methods within-subject study (N=48) suggested that our approach offers a non-intrusive and efficient way to share viewers’ frisson moments, increases the social presence of others as if watching together, and can create affective contagion among viewers.
Trust in Social Media
Profiling the Dynamics of Trust & Distrust in Social Media: A Survey Study
Yixuan Zhang, Yimeng Wang, Nutchanon Yongsatianchot, Joseph Gaggiano, Nurul Suhaimi, Anne Okrah, Jacqueline Griffin, Miso Kim, Andrea Parker
In the era of digital communication, misinformation on social media threatens the foundational trust in these platforms. While myriad measures have been implemented to counteract misinformation, the complex relationship between these interventions and the multifaceted dynamics of trust and distrust on social media remains underexplored. To bridge this gap, we surveyed 1,769 participants in the U.S. to gauge their trust and distrust in social media and examine their experiences with anti-misinformation features. Our research demonstrates how trust and distrust in social media are not simply two ends of a spectrum; but can also co-exist, enriching the theoretical understanding of these constructs. Furthermore, participants exhibited varying patterns of trust and distrust across demographic characteristics and platforms. Our results also show that current misinformation interventions helped heighten awareness of misinformation and bolstered trust in social media, but did not alleviate underlying distrust. We discuss theoretical and practical implications for future research.
User Studies on Large Language Models
Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
Zijie Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, Michael Madaio
Prompt-based interfaces for Large Language Models (LLMs) have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI applications remains a challenge, particularly during prompt-based prototyping. To address this, we present Farsight, a novel in situ interactive tool that helps people identify potential harms from the AI applications they are prototyping. Based on a user’s prompt, Farsight highlights news articles about relevant AI incidents and allows users to explore and edit LLM-generated use cases, stakeholders, and harms. We report design insights from a co-design study with 10 AI prototypers and findings from a user study with 42 AI prototypers. After using Farsight, AI prototypers in our user study are better able to independently identify potential harms associated with a prompt and find our tool more useful and usable than existing resources. Their qualitative feedback also highlights that Farsight encourages them to focus on end-users and think beyond immediate harms. We discuss these findings and reflect on their implications for designing AI prototyping experiences that meaningfully engage with AI harms. Farsight is publicly accessible at: https://pair-code.github.io/farsight.
Wellbeing and Mental Health B
I’m gonna KMS: From Imminent Risk to Youth Joking about Suicide and Self-Harm via Social Media
Naima Samreen Ali, Sarvech Qadir, Ashwaq Alsoubai, Munmun De Choudhury, Afsaneh Razi, Pamela Wisniewski
Recent increases in self-harm and suicide rates among youth have coincided with prevalent social media use; therefore, making these sensitive topics of critical importance to the HCI research community. We analyzed 1,224 direct message conversations (DMs) from 151 young Instagram users (ages 13-21), who engaged in private conversations using self-harm and suicide-related language. We found that youth discussed their personal experiences, including imminent thoughts of suicide and/or self-harm, as well as their past attempts and recovery. They gossiped about others, including complaining about triggering content and coercive threats of self-harm and suicide but also tried to intervene when a friend was in danger. Most of the conversations involved suicide or self-harm language that did not indicate the intent to harm but instead used hyperbolical language or humor. Our results shed light on youth perceptions, norms, and experiences of self-harm and suicide to inform future efforts towards risk detection and prevention.
Special Interest Group
Marios Constantinides, Mohammad Tahaei, Daniele Quercia, Simone Stumpf, Michael Madaio, Sean Kennedy, Lauren Wilcox, Jessica Vitak, Henriette Cramer, Edyta Bogucka, Ricardo Baeza-Yates, Ewa Luger, Jess Holbrook, Michael Muller, Ilana Golbin Blumenfeld, Giada Pistilli
With the upcoming AI regulations (e.g., EU AI Act) and rapid advancements in generative AI, new challenges emerge in the area of Human-Centered Responsible Artificial Intelligence (HCR-AI). As AI becomes more ubiquitous, questions around decision-making authority, human oversight, accountability, sustainability, and the ethical and legal responsibilities of AI and their creators become paramount. Addressing these questions requires a collaborative approach. By involving stakeholders from various disciplines in the 2nd edition of the HCR-AI Special Interest Group (SIG) at CHI 2024, we aim to discuss the implications of regulations in HCI research, develop new theories, evaluation frameworks, and methods to navigate the complex nature of AI ethics, steering AI development in a direction that is beneficial and sustainable for all of humanity.
Workshop
Upol Ehsan, Elizabeth Watkins, Philipp Wintersberger, Carina Manger, Sunnie S. Y. Kim, Niels van Berkel, Andreas Riener, Mark Riedl
Human-centered XAI (HCXAI) advocates that algorithmic transparency alone is not sufficient for making AI explainable. Explainability of AI is more than just “opening” the black box — who opens it matters just as much, if not more, as the ways of opening it. In the era of Large Language Models (LLMs), is “opening the black box” still a realistic goal for XAI? In this fourth CHI workshop on Human-centered XAI (HCXAI), we build on the maturation through the previous three installments to craft the coming-of-age story of HCXAI in the era of Large Language Models (LLMs). We aim towards actionable interventions that recognize both affordances and pitfalls of XAI. The goal of the fourth installment is to question how XAI assumptions fare in the era of LLMs and examine how human-centered perspectives can be operationalized at the conceptual, methodological, and technical levels. Encouraging holistic (historical, sociological, and technical) approaches, we emphasize “operationalizing”. We seek actionable analysis frameworks, concrete design guidelines, transferable evaluation methods, and principles for accountability.
Workshop on Post-growth HCI: Co-Envisioning HCI Beyond Economic Growth
Vishal Sharma, Anupriya Tuli, Asra Wani, Anjali Karol Mohan, Bonnie Nardi, Marc Hassenzahl, Morgan Vigil-Hayes, Rikke Hagensby Jensen, Shaowen Bardzell, Neha Kumar
Human–Computer Interaction (HCI) makes a significant contribution to economic growth; it is crucial to the market success of digital technologies, including digital services, platforms, and devices, which drive the economic engine. Economic growth, however, has a number of social and environmental consequences. Some HCI researchers have problematized the field’s engagement with growth, suggesting the post-growth philosophy as an alternative. Post-growth focuses on improving the quality of life centered on cooperation, social solidarity, care, justice, sharing, localized development, and other values. Orienting to post-growth could be instrumental in leading the HCI community beyond growth politics by envisioning, designing, and implementing technologies toward building a more sustainable, just, and humane society. This workshop aims to bring together HCI researchers, designers, practitioners, educators, and students to critically reimagine ways to embrace post-growth in and through HCI.
Workshop on Theory of Mind in Human-AI Interaction
Qiaosi Wang, Sarah Walsh, Mei Si, Jeffrey Kephart, Justin Weisz, Ashok Goel
Theory of Mind (ToM), humans’ capability of attributing mental states such as intentions, goals, emotions, and beliefs to ourselves and others, has become a concept of great interest in human-AI interaction research. Given the fundamental role of ToM in human social interactions, many researchers have been working on methods and techniques to equip AI with an equivalent of human ToM capability to build highly socially intelligent AI. Another line of research on ToM in human-AI interaction seeks to understand people’s tendency to attribute mental states such as blame, emotions, and intentions to AI, along with the role that AI should play in the interaction (e.g. as a tool, partner, teacher, facilitator, and more) to align with peoples’ expectations and mental models. The goal of this line of work is to distill human-centered design implications to support the development of increasingly advanced AI systems. Together, these two research perspectives on ToM form an emerging paradigm of “Mutual Theory of Mind (MToM)” in human-AI interaction, where both the human and the AI each possess the ToM capability. This workshop aims to bring together different research perspectives on ToM in human-AI interaction by engaging with researchers from various disciplines including AI, HCI, Cognitive Science, Psychology, Robotics, and more to synthesize existing research perspectives, techniques, and knowledge on ToM in human-AI interaction, as well as envisioning and setting a research agenda for MToM in human-AI interaction.