How are Autistic People Involved in the Design of Extended Reality Technologies? A Systematic Literature Review

Research question 1 was concerned with the characteristics of published projects that include autistic people in extended reality technology-based research. A range of noteworthy themes emerged regarding the characteristics of these projects. First, desktop-based virtual reality XR systems were the most prominent, especially in early research efforts, while head-mounted displays and platforms built upon software such as Unity facilitated greater immersion and opportunities for interactions in real-time. Mobile augmented reality systems used everyday devices (i.e., smartphones) to bridge digital content with physical objects, providing unique opportunities for therapy and education.

Moreover, several studies focused on developing software content or protocols for future use in XR, indicating ongoing innovation in this field. In exploring this research question, we found disparities regarding technology semantics. For example, Smith et al. (2020) described a multimedia, video-based project as VR; however, definitions of VR (Hale & Stanney, 2014) would not align to this particular technological setup. Alternatively, Schmidt & Glaser (2021a) also designated a multimedia video-based project as VR, but they employed 360-degree videos and VR headsets, effectively placing this project within an established VR classification. Further, Amara et al. (2022) described some of their VR activities as being ‘game-like’, further muddying the waters in defining these immersive experiences. Such disparities are acknowledged by Savickaite and colleagues (2022), who suggest that: “[...] in the absence of transparent reporting standards and terminology, readers may be left confused (or unintentionally misled) by manuscripts that use the term ‘virtual reality’ to describe non-HMD devices” (np). A final point related to this research question is the nature of anticipated outcomes and research foci of the studies included. We found the earlier work focused mostly on social skills (Cobb et al., 2002; Rutten et al., 2003; Millen et al., 2011) while the later work presented more diverse and wide-ranging foci. This included (but not limited to): Promoting cultural access (Giaconi et al., 2021); public transportation training (Schmidt & Glaser, 2021a, b); and job interviewing skills (Adiani et al., 2022; Smith et al., 2020). This diverse work was published between 2020 and 2022 and suggests that research might be evolving towards research agendas and foci that are more in alignment with the needs of the autistic community; more than they have been in the past.

Most autism research with XR has historically been conducted with children, often without significant contribution from autistic adults (Happé & Frith, 2020). However, the findings of this study indicate a shift in this pattern, with an increasing number of studies involving older individuals. Despite this progress, there is still a substantial need for greater involvement of autistic adults in research. Furthermore, researchers will need to consider meaningful ways to engage younger groups in co-design to ensure their voices are also included (Newbutt et al., 2023).

Research question 2 sought to identify reported outcomes, recommendations for, and implications of participatory and co-design with autistic people for research and practice. According to Parsons and Cobb (2013) and Fletcher-Watson et al. (2019), considering the views of autistic individuals and stakeholders throughout the design process can improve the quality of research methods and lead to better translation of research into practice, and improved outcomes for autistic people. Other research and reviews came to the same conclusion (Tsikinas & Xibigalos, 2019). Therefore, it is especially interesting to note that our review located only 20 articles that report on, or included, autistic input to XR-based research. Moreover, XR-based research and technologies are especially well-placed to champion and utilize their design to include user-centered design processes (Bauer et al., 2022). This systematic literature review identified a body of experimental and applied research that highlights the benefits of involving the autistic community in priority-setting in XR-based research. Consequently, several recommendations for future research have emerged from the papers reviewed. Smith et al. (2020) outlined that careful attention should be given to cognitive load and gamification techniques when designing with this heterogeneous population. In line with this suggestion, Giaconi et al. (2021) recommended that scholars develop integrated systems of technologies capable of creating museums accessible to neurodiverse people. Moreover, the previous research of Rutten et al (2003) supports the argument for integrated systems in the newer Giaconi research, which cautioned researchers that VR alone may not be adequate as a training tool for autistic individuals, and additional support may always be necessary. Additionally, two studies (Politis et al., 2019; Schmidt & Glaser, 2021a) called for more longitudinal studies to collect research data and answer questions concerning the novelty effect of extended reality technologies because it is difficult to keep abreast of the rapid pace of technological evolution. Finally, Howard and Gutworth (2020) have suggested that future research include a wider population with larger sample size, and design immersive spaces to be more reflective of real life and tailor their training to meet autistic individual needs and preferences. We also found that two studies did not report the outcomes of involving autistic communities in their research; and we suggest that moving forward in this field, this is something researchers not only consider but include as part of their approach to including and engaging autistic people in research.

In spite of the benefits of involving autistic individuals in the design process, there are some challenges associated with this approach. Four out of the 20 selected articles highlighted these challenges (Cobb et al., 2002; Politis et al., 2019). First, stakeholders may have divergent goals and perspectives, which researchers must be able to integrate into a cohesive design process (Cobb et al., 2002; Politis et al., 2019). Second, some autistic individuals may struggle to articulate their opinions and require tailored or alternative approaches to elicit their perspectives (Cobb et al., 2002; Politis et al., 2019; Millen et al., 2011). Third, accessibility issues and barriers presented by researchers may impede the meaningful engagement of some autistic individuals in the design process (Millen et al., 2011). Fourth, co-designing with autistic communities requires significant time and effort (Millen et al., 2011). Finally, accommodating all recommendations from this diverse population may not align with the ethical guidelines of Institutional Review Boards (IRBs) (Politis et al., 2019) due to the complex nature of this process that does not always require researchers to think in ways to engage and involve diverse populations in their research fully. There can also be a lack of understanding or empathy from neurotypical researchers, which can lead to miscommunications or misguided assumptions about the needs and abilities of autistic individuals

Research question 3 investigated the nature of inclusion of autistic people in the corpus of included articles. Findings suggest that autistic people actively participated in various stages, including design, evaluation, and problem identification. Further, parents of autistic children and teachers of autistic individuals also were actively involved in problem identification, design, and evaluation stages. In addition, and in some cases, autism care providers, clinicians, and experts in autism were consulted to ensure the design considered the unique needs of autistic users. These groups were involved in various inclusive design practices, including participatory design, co-design, testing, and evaluation, leading to the identification and resolution of issues and enhancement of designs through solicitation of preferences, feedback, and recommendations to inform research and development. While each of the studies we reviewed included various levels and modalities of autistic input on the research and development process, none of the studies reported including autistic people or their stakeholders in co-creation of research questions or design methodologies. This suggests that research agendas may not be driven by autistic preferences, but instead by researchers who are presumably not autistic; thus, it would seem that a research gap exists relative to guiding research agendas within XR-based research for autism (Poulsen et al., 2022).

Based on insights gained from findings reported in Table 1 on inclusive design practices across various projects and stakeholders, including autistic individuals, parents, teachers, clinicians, and experts, we propose a design framework that characterizes autistic inclusion in XR research (Fig. 2).

The framework consists of four categories that describe autistic inclusion in XR research and development. These include: (1) expert-led design, (2) participatory design, (3) co-design for autistic people, and (4) co-design with autistic people. These categories are illustrated in Fig. 2 within four quadrants located on X and Y axes. The X axis represents agency, that is, the ability to influence design decisions related to the XR application.. The Y axis represents inclusion of autistic people and stakeholders, that is, the degree to which autistic people are included in XR research and development processes. The X and Y axes represent increasing agency and inclusion respectively.

The top two quadrants of Fig. 2 encompass those studies that utilized a co-design approach. By co-design, we refer to those design approaches that, in contrast with top-down approaches to design in which a user’s role is more passive, users are active participants in the design process who are valued as equal contributors (Roschelle et al., 2006). The bottom two quadrants represent those studies that used a participatory design approach. By participatory design, we refer to design processes that bear some similarity to co-design, but differentiate themselves in that users are not seen as equal contributors. That is, user input is solicited in participatory design, but the design team is ultimately the arbiter of decision-making (Engelbertink et al., 2020).

Within the top two quadrants of Fig. 2, we differentiate co-design as being either for autistic users (top-left quadrant) or with autistic users (top-right quadrant). Within the included studies, co-design for autistic users involved non-autistic participants, such as experts or parents, who take an active role alongside the research team in the design process. Co-design with autistic users was illustrated by research teams actively involving autistic users as designers, with design input and decision-making capabilities equal to those of the research team. Within the bottom two quadrants of Fig. 2, we differentiate participatory design as being either led by non-autistic experts (bottom-left quadrant) or being conducted with autistic people/stakeholders (bottom-right quadrant). In the included corpus of articles, non-autistic experts included teachers, therapists and medical professionals who were often included in providing input and feedback on XR designs. In contrast to this, autistic stakeholders included parents, caregivers, providers, and teachers who were invited to provide input and feedback based on their direct involvement with autistic people (i.e. as their children or students) or lived experiences as an autistic person. Within the participatory design frame, neither non-autistic experts nor autistic stakeholders had agency to make final design decisions. We illustrate these quadrants with examples in the following paragraphs.

Two articles using the expert-led design approach were Miningrum et al. (2021) and Lee. (2019), in which the design decisions are primarily made by non-autistic experts without seeking direct input from autistic individuals. Miningrum et al. (2021) describes how the design process involved input from teachers of autistic people and autism care providers, with design improvements based on feedback gathered from therapists. Similarly, Lee (2019) reports how parents and teachers of autistic children were interviewed to inform creation of social stories; however, there is no mention of involving autistic individuals themselves in the design process. Both articles demonstrate an expert-led design approach, in which design decisions were made by non-autistic experts, such as teachers, parents, and therapists, without seeking direct input from autistic individuals. The absence of direct input from these individuals indicates that the design was primarily driven by the knowledge and understanding of the non-autistic professionals involved.

Examples of participatory design approaches were found in Ramachandiran et al. (2015) and Smith et al. (2020), in which design input was sought from autistic stakeholders (such as autistic individuals themselves and their parents), but final design decisions were still made by non-autistic experts. Ramachandiran et al. (2015) report a design process that involved active participation from parents of autistic children who were interviewed to gather information about their children's essential needs. In this case, the input of autistic stakeholders (in this case, parents) played a significant role in shaping the design decisions, but these design decisions were not made by stakeholders. Similarly, Smith et al. (2020) report a design process that involved input from multiple autistic stakeholders, including autistic people, parents, teachers, and community employers, with stakeholders providing recommendations for improvement. However, a non-autistic scientific advisory board reviewed the recommendations and made final design decisions. Both articles outline participatory design approaches where input from autistic stakeholders, such as autistic individuals and parents, was actively sought to inform the design process. However, the final decisions were made by non-autistic experts, which raises questions regarding the balance between incorporating the input of autistic stakeholders and considering the expertise and knowledge of non-autistic professionals.

Two examples of co-design for autistic people using a co-design approach were Ip et al. (2022) and Wang et al. (2022), in which the design process involved collaborative input from both non-autistic experts and representatives for autistic people. Ip et al. (2022), included teachers of autistic people who played a vital role in the evaluation phase. Likewise, Wang et al. (2022), included autistic stakeholders in the planning, development, and evaluation stages, showcasing a co-design process involving multiple stakeholders (parents of autistic children, teachers of autistic people, and autism care providers). Both articles demonstrate a co-design process where non-autistic experts, such as parents, teachers, researchers, and therapists, actively collaborated to create XR experiences for autistic people.

Two studies shared examples of co-design where autistic people were actively involved in the design process and incorporated their perspectives and preferences (Ghanouni et al., 2019; Millen et al. 2011). Data were collected regarding their perceptions and experiences when visiting a museum, and were analyzed to identify necessary accommodations. The research team further collaborated with autistic students by observing their museum visits, thus allowing for direct input from the autistic community. These data, combined with observations, informed the creation of a virtual prototype that aimed to address the identified needs and preferences of autistic individuals. Similarly, Millen et al. (2011) involved autistic participants as critical co-designers, testers, and evaluators in the development of prototypes. By actively engaging autistic individuals in the design process, their unique insights and feedback were incorporated into the creation of the final prototypes. The preferences, perspectives, and experiences of autistic participants were given significant importance, ensuring that the end products were tailored to meet their specific requirements. By actively involving autistic individuals in the design process, both studies valued and prioritized autistic perspectives.

The findings reported here should be viewed in the context of the following limitations. Firstly, we did not include so-called “gray” literature to ensure included studies were of peer-review quality. In doing so, our review neglected a range of outlets (e.g., conference proceedings, published dissertations) which might have yielded work reporting co-design approaches used in research with autistic people. In addition, book chapters were also discounted that might have provided insights of research reporting co-design and participatory-based research within XR domains and fields. Another limitation relates to the terminology used in our search terms, such as neurodiverse/neurodiversity, cognitive disabilities, developmental disabilities, and other related terms, such as user-experience (UX), social robots, or AI, which could have led to missing studies that conceive of or describe co-design and in ways that may be somewhat divergent. In addition, we only searched for articles in English, thereby potentially introducing biases as the work presented would likely exclude research from non-White, the Global South, and intersectional researchers and autistic individuals. Inclusion of non-English articles may have increased the number included in our corpus.