How do people perceive and interact with a Socially Interactive Virtual Environment (SIVE) that is developed as an intelligent, virtual, and spatial agent in the co-design process, or more specifically, a “Space Agent” design partner? How will these perceptions and interactions shape the human-SIVE co-design? Here, SIVE refers to AI-embedded virtual environments that can interact and collaborate with human users in a socially expressive way, and “Space Agent” refers to a socially expressive AI system with a spatial embodiment (e.g., a smart room, office, public space, etc.). This paper explores the intricate partnership between human users and SIVE through empirical studies. We aim to gain a preliminary understanding of how users perceive and engage with SIVE in the design process. Harnessing Unreal Engine, we designed and developed a virtual environment that used various visual and auditory modalities to interact with users and employed the Wizard-of-Oz (WoZ) technique to conduct in-lab experiments with 10 participants. The participants’ perspectives and interactions were video recorded, and then carefully reviewed and analyzed by multiple researchers. A semi-structured interview was also conducted with each participant after the WoZ experiment, with the transcripts analyzed through the grounded theory coding technique. By examining these qualitative data, we aim to uncover valuable insights into the dynamics between human users and SIVE during the co-design process. The findings of this study contribute to a deeper understanding of the potential benefits and challenges associated with integrating SIVE into collaborative design practices and have the potential to shape the future of human-AI design partnerships and co-creations, offering guidance for designers, developers, and researchers seeking to leverage the unique capabilities of SIVE as a design partner.
Publications:
Sharmayne Lim, Daryl Ho, Jiaze Yu, and Yixiao Wang*. 2024. Virtual Reality “Space Agent” as a Socially Interactive Design Partner: A study investigating users’ perception of and interaction with Socially Interactive Virtual Environment (SIVE) as a design partner. In Proceedings of the 12th International Conference on Human-Agent Interaction (HAI ’24). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3687272.3688295