A systematic literature review of video-based interventions to improve integrated competitive employment skills among youth and adults with Autism Spectrum Disorder

1.1Video-based interventions and people with ASD

Given the limitations of the research associated with employment training for people with ASD (Hendricks, 2010), it would seem that tailored supports and interventions are needed to improve job search and job performance skills. Specifically, skills related to the ability to effectively communicate with current and future employers (Higgins et al., 2008; Strickland et al., 2013). Video-based interventions (VBIs) have been identified as an evidence-based practice addressing the unique learning characteristics of people with ASD and their use is on the rise (Wong et al., 2015). This approach integrates technology such as video-modeling, self-modeling, feedback, prompting and virtual reality programs as part of comprehensive training packages. For the purpose of this study, we define VBIs as strategies that incorporate visual and audio cues to teach or maintain skills through model behavior or feedback.

VBIs have been used to teach people with ASD daily living (Burckley et al., 2015; Mechling et al., 2013), academic (Burton et al., 2013), and social skills (Tetreault & Lerman, 2010). For the most part, VBI studies have been conducted in schools, day treatment centers, and community living settings. A few more recent studies have used VBIs in employment settings primarily to increase job-related performance skills (Burke et al., 2013; Kellems & Morningstar, 2012), and in one recent study, to increase customer service skills (Bross et al., 2018).

Generally speaking, video modeling involves presenting a video of a model (e.g., peers, adults, self) who is performing the target behavior (Bellini & Akullian, 2007). Elements of VBI methods fall into one of several categories: (a) self-modeling in which the participant is videotaped successfully performing the target behavior (Dowrick, 1999); (b) video feedback which allows participants to watch a recorded video of themselves and self-evaluate their performance; (c) video prompting in which tasks are analyzed into video segments demonstrating each step of a task (Burke et al., 2013); and (d) augmented and virtual reality that supports practice and rehearsal of responses through computer-generated environments (Strickland et al., 2013; Chen et al., 2015).

1.2Video-based interventions for job search skills

Specifically related to job search skills, certain VBI methods have been found to improve communication and conversational skills, social behavior, and social initiations for job interviewing. The use of VBIs for other job search skills (e.g., checking job ads, social networking skills) has not yet been established. Among studies focused on job interviewing, most VBIs interventions have targeted skills such as maintaining eye contact, using facial expressions, conversational turn-taking, and exhibiting empathetic statements and questioning (Mason et al., 2012; Koegel et al., 2016). In addition, a wide range of VBI studies have targeted other forms of social communication skills, but these studies have taken place outside of employment domains and are mostly associated with school and center-based interventions for younger children (Shukla-Mehta et al., 2013). Only recently have VBIs been used to address work-related social engagement. Most recently, emergent and preliminary studies have targeted interview skills for youth with ASD (Hayes et al., 2015; Munandar, 2016; Strickland et al., 2013; Smith et al., 2014).

While VBIs have been shown to be effective in improving social and communication skills for people with ASD, this method is emerging as an intervention for addressing critical skills necessary for competitive integrated employment. It is particularly true that VBIs have just begun to target specific skills associated with the three essential phases of employment: (a) obtaining employment (i.e., job search and interviews skills); (b) learning initial skills in integrated employment settings; and (c) maintaining and advancing within integrated employment. While VBIs have been found effective for improving the development of employment skills for people with ASD, the majority of these studies have taken place in segregated settings using simulations of potential jobs, with fewer studies conducted in competitive integrated employment (Gentry et al., 2015; Van Laarhoven et al., 2012). Furthermore, among those studies that do focus on employment for youth with ASD, most have targeted learning a specific job task, with fewer addressing social communication challenges needed to obtain integrated employment or maintain and expand job duties and roles (Strickland et al., 2013; Smith et al., 2014; Hayes et al., 2015).

1.3Purpose of the study

Given the fast pace with which technology has emerged and is utilized to support students with ASD, and the nascent use of VBIs related to competitive integrated employment, there is a need to better understand the current range of VBIs research associated with the three phases of finding, obtaining, and keeping integrated competitive employment for youth with ASD. The following research questions guided this systematic review:

2.1Inclusion and exclusion criteria

Four inclusion criteria were used to screen studies containing relevant content related to the purpose of this systematic literature review: (a) at least one participant in the study had an ASD diagnosis, and aged 16 or older; (b) VBIs occurred in integrated employment settings; and (c) VBIs targeted job search, on the job training, or job maintenance. Only experimental research published in English, peer-reviewed journals prior to April 30th, 2019 were included. Because of the recent and emerging body of research associated with VBIs, the search did not set a start date, with the oldest study published in January 2009.

Publications were excluded if they were: (a) non-experimental (e.g., meta-analysis, systematic literature reviews); (b) included only audio or only visual cues; and (c) targeted domestic living skills considered to be “pre-vocational” that occurred in non-employment settings.

2.2Search procedures

We implemented several steps to identify relevant peer-reviewed published studies that met inclusion criteria. First, we searched three electronic databases (i.e., ERIC, PsycINFO, JSTOR), using keywords in multiple combinations (autis*, video*, employ*, vocation*, job*, occupation*, interview*, model*). This resulted in a total of 1,426 articles. The first round of screening involved reading the titles and abstracts to exclude studies that were non-experimental, taught non-employment related skills, and involved children as participants. The first screening narrowed the number of publications to 36. The second round of screening involved reading the full text to identify studies meeting all inclusion criteria. Twenty articles were removed from the review, as there were no participants aged 16 or older involved, or the outcomes of the study did not lead to integrated competitive employment. The second screening narrowed the number of publications to 14.

An ancestral search was then conducted by examining references from the included articles as well as examining references from six meta-analyses examining VBIs more broadly across a range of skills. No additional articles were added to the total upon conclusion of this process. We then performed a hand search of journals that yielded articles that met the inclusion criteria (e.g., Journal of Vocational Rehabilitation, Journal of Autism and Developmental Disorders). Four articles were identified through the hand search, resulting in a total of publications of 18. Interrater agreement (IRA) during the screening was completed by having two of the researchers separately review the 18 articles, with both agreeing that the 18 articles met the inclusion criteria (100% IRR). The 18 published articles included 19 studies because one article (Burke, Andersen, Bowen, Howard, & Allen, 2010) reported two separate studies.

2.3Coding procedures

Participant demographic characteristics were coded for age, race, and gender. Employment history was coded as: (a) currently employed, (b) previously employed but currently unemployed, or (c) never employed. The number of participants with ASD per study was also recorded, but for the most part, the data from participants with ASD and those without ASD were not disaggregated. Socioeconomic status was not included because there was not sufficient information about these data within the 19 studies.

Second, the studies were coded based on: (a) type of VBI (e.g., video modeling, self-modeling, video prompting, video feedback, virtual reality); (b) setting (e.g., on-the-job, school, community-based setting); (c) who implemented the intervention (e.g., researcher, teacher); and (d) whether family members, teachers, or employers were involved. Family, teachers, and employers are considered involved if they implemented the intervention, completed a fidelity check, or participated in a social validity measurement.

The specific employment skills targeted by the intervention were categorized as skills associated with job search and job performance skills. Interventions were coded as job search when the VBIs were used to teach specific job search skills (e.g. resume writing, job interviewing). The job performance code was used when the VBIs taught skills required to perform employment tasks occurring in integrated employment settings. In addition, the maintenance and generalization of the VBIs were considered, with the maintenance code used for studies that included information regarding the participants’ ability to maintain the employment skills for a certain period of time post-intervention training; and generalization was used when the study included participants’ ability to generalize skills to new employment settings.

Finally, each study was examined for adherence to the Council for Exceptional Children (CEC) Standards for Evidence-Based Practices in Special Education (Cook et al., 2014). The standards consist of 24 quality indicators (QIs) for group design studies and 22 QIs for single-subject studies. The indicators address quality across eight areas of research: (a) context and setting, (b) participants, (c) intervention agent, (d) description of practice, (e) implementation fidelity, (f) internal validity, (g) outcome measures/dependent variables, and (h) data analysis. Each QI is scored 1 if the study met the methodological criteria or a 0 if it did not. The overall scores are divided by the total number of QIs and multiplied by 100%.

2.4Interrater agreement (IRA)

To ensure the reliability of the collected data, two authors separately conducted coding and QI evaluation for 14 of the 18 articles (77.8%), which included 15 studies because one article includes two studies (Burke et al., 2010). Upon completing coding and QI evaluation for the 15 studies, the two authors met and compared results. Agreement or disagreement was scored for each item on the coding sheet. IRA was calculated by dividing the sum agreements by the sum of the agreements plus disagreements, multiplied by 100. The IRA for coding ranged from 87.5% to 100%, with a mean of 92.7%. The IRA for QI evaluation ranged from 68.2% to 100%, with a mean of 84.2%. The disagreements were resolved through discussion between the two researchers.

3.1Demographics

A total of 158 participants with ASD were identified out of 164 participants. Table 1 presents information on the participants’ demographic characteristics.

3.2Job performance skills taught using VBIs

Most studies reviewed (n = 17, 89.5%) focused on the acquisition of skills using VBIs. Among these, 14 (73.7%) measured the effectiveness of the intervention in maintaining skills. Four studies (21%) investigated whether participants generalized the skills to different settings. The length of the intervention varied considerably, with periods reported between three weeks and one year. Job performance skills were identified and categorized as either social communication skills or task performance. Twelve studies focused on task performance skills such as using equipment at the employment sites, cleaning, preparing food, shipping, and delivering fire safety education (see Table 2). One study focused on both skills (i.e., interacting with co-workers, and completing tasks with acceptable quality; Gentry et al., 2015). Social communication skills were targeted within three studies and included product promotions (n = 1), social interaction in employment sites (n = 1), and customer service (n = 1, see Table 1).

3.3Job search skills acquisition taught using VBIs

Only three articles addressed VBIs to improve job search skills among people with ASD. Results indicated that job search skills significantly improved using VBIs such as peer modeling, self-modeling, and virtual reality. All three studies demonstrated evidence of experimental control, and all participants in these three studies showed improvements in their job interviewing skills (see Table 2). Dependent variables measured in one study included physical appearance, facial expressions, body language, and spoken answers (Hayes et al., 2015). Smith and colleagues (2014) measured communication skills that contributed to successful job interviews (e.g., comfort levels, negotiation skills, sharing examples positively, establishing rapport). One study measured the content of participant responses as well as their response delivery (i.e., greetings, facial expressions, body positioning; Strickland et al., 2013).

3.4Types of VBIs

A wide range of technology was used among the 19 studies (e.g., videotape recorders, smartphones, computers, tablets, and DVDs). Software and computer applications were used in three studies: (a) “VideoTote” on computer tablets (Burke et al., 2013), (b) “VidCoach” on iOS devices (Hayes et al., 2015), and (c) Virtual Reality Job Interview Training (VR-JIT) on computers (Smith et al., 2014). A web-based interviewing skills program, “JobTIPS” was used in one study (Strickland et al., 2013).

While a range of VBIs were implemented across the 19 studies, the job performance video interventions focused on video modeling (n = 10), video self-modeling (n = 1), video feedback (n = 1), video prompting (n = 2). Two studies implemented more than one type of VBI (e.g., video modeling and video prompting; video modeling and video self-modeling). The job search VBIs tended to focus on virtual reality programs (n = 2), and one study combining video modeling and video self-modeling.

Nine studies implemented video modeling with different variations in how the video was produced and edited (e.g. scripted target behavior, natural target behavior, with or without narration, with or without written instructions). Virtual reality programs reported using online programs (e.g., JobTIPS) or computer software (e.g., VR-JIT).

3.5Implementers

Five studies had teachers or a job coach as the intervention agent (Cihak & Schrader, 2009; Gentry et al., 2015; Goh & Bambara, 2013; Mackey & Nelson, 2015; Walsh et al., 2018), while the rest were implemented by researchers. Five studies included family members (Burke et al., 2010, 2013; Van Laarhoven et al., 2012; Walsh et al., 2018) not to implement the VBI, but to complete social validity measurements or fidelity checks. Five studies involved employers who completed social validity measures (Allen, Wallace, Greene, et al., 2010; Burke et al., 2010; Kellems & Morningstar, 2012; Bross et al., 2018). Among the nine studies that included teachers, their roles include implementing VBIs included providing systematic prompts, instructions, and feedback (Bereznak et al., 2012; Van Laarhoven et al., 2012; Goh & Bambara, 2013); being a second observer (Mackey & Nelson, 2015); and participating in social validity measurement (Burke et al., 2010, Cihak & Schrader, 2009; Kellems & Morningstar, 2012, Walsh et al., 2018).

3.6Methodological rigor

Among the 19 studies, five used a group design (e.g., randomized study with control and treatment groups, delayed randomized controlled trial, modified pre/post-test group design) and 14 reported single-subject design (e.g., multiple-probe or multiple-baseline across behavior or participants). Using the CEC Standards for Evidence-Based Practices in Special Education (Cook et al., 2014), among the five group designs, the percentage of QIs met averaged 80.9%, ranging from 78.3% to 82.6%. Whereas the single-subject design studies overall quality averaged 84.3%, with a much wider range from 54.5% to 100%.

Table 2 presents how the QIs were met across all single-subject studies. Table 3 presents how the QIs were met across all group design studies. Seven QIs were met by 100% of the studies: (a) context and setting, (b) participant demographics, (c) description of intervention procedure, (d) control and manipulation of independent variable, (e) social importance of outcome, (f) definition and measurement of dependent variable, and (g) evidence of internal reliability. For QIs that were only applicable for group designs, five additional QIs were met 100% of the time: (a) clear assignment of each group, (b) low attrition, (c) low differential attrition, (d) evidence of validity, and (e) appropriate data analysis technique. For QIs that were only applicable for single-subject studies, one QI that was met 100% of the time was a single-subject graph for visual analysis.

4.1Limitations

Several limitations are associated with this review, including the inclusion and exclusion criteria. First, studies examining VBIs to improve certain pre-vocational, daily living and social skills that were not specifically employment-focused for people with ASD were excluded. Although job interviews and on-the-job social communication skills sought by employers were included in this study, most studies involving VBIs to teach communication behaviors took place with younger participants in school or community settings outside of work. Similarly, some studies that targeted daily living skills (e.g., setting a table, making a bed) were excluded. Some may argue these skills do occur in certain jobs, however, there are substantial distinctions made when teaching skills for home living as compared to those needed to perform specific employment skills. Given the very unique nature of job search and job performance skills, the scope of this review was determined to include only employment-related research.

Second, the review only included articles published in peer-reviewed journals, and this criterion may have narrowed the level of timeliness of the research, given how quickly new technologies develop. Expanding the search to recent dissertations and theses may have led to the establishment of a wider body of emergent research yet to be published. Furthermore, other related technological interventions (e.g., virtual reality, augmented reality, mobile device interfaces) may have been missed through the use of “video*” as the only technology-related search term used. Given the breadth and diversity of technology interventions emerging, the search term may have limited the number of potential VBI studies.

Third, data from the target population was not disaggregated in many of the reviewed studies. Therefore, the results do not reveal if the efficacy of VBIs toward improving employment skills vary by subpopulation (e.g., race, age, gender). However, this study still provides important information on how VBIs can be used for the population of youth and adults with ASD in integrated and competitive work settings in general. Finally, the IRR range for the QI evaluation was relatively low (68.2–100%) while the overall mean was 84.2%. However, the disagreements were resolved through a discussion between the two raters, and the final data reported reflect the consensus reached through the discussion.

4.2Implications for future research

Researchers and policymakers have indicated the struggle among young adults with disabilities to obtain integrated competitive employment (Workforce Innovation and Opportunity Act, 2018). However, the body of research examining the efficacy of VBIs in teaching job search skills remains scarce. Given such limited findings, it is clear that future research examining how VBIs can support people with ASD to develop employment skills is needed. Job seekers with ASD can benefit from evidence-based practices that improve other types of job search skills (e.g., writing resumes, using the network to find job opportunities) that have yet to be established within current published research. It was concerning that none of the studies included these types of job search skills. In addition, studies focusing on social communication skills required for obtaining and maintaining employment is an emergent area of research. Particularly, social communication skills to foster a positive work experience (Hendricks, 2010) and for customer service skills that are often thought to be a barrier for people with ASD (Bross et al., 2018). Clearly more substantive research in employment-related social communication skills is needed if youth and adults with ASD are to find and keep integrated competitive jobs.

Finally, we encourage future researchers to use the CEC Standards for Evidence-Based Practices in Special Education as a checklist when designing and reporting intervention research. Addressing all the QIs in the manual is essential for a study to be used in establishing an evidence-based practice (Cook et al., 2014). Based on the result of the QI measurement in this literature review, future studies need to explicitly report their implementation fidelity related to dosage or exposure and the training or qualification required for the intervention implementers. Meeting all these QIs is important to build empirical support for the use of VBIs to teach employment skills. Furthermore, addressing the QI criteria will benefit future researchers to understand the full context of the practice and to replicate the study.

4.3Implications for practice

Considering the positive results of VBIs, practitioners are expanding their use to support people with ASD across a range of academic, social and communication skills, including those to find and maintain employment. VBIs are generally reported as simple to implement, enjoyable to use, and effective in results (Burke et al., 2010, 2013; Cihak & Schrader, 2009). Practitioners can and do use VBIs with school and work-based learning experiences. Expanding their focus to specific and perhaps more advanced employment skill sets (e.g., customer service, quality control, problem-solving) may expand the array of occupations available to people with ASD.

The results show that VBIs were more effective when combined with another form of support, such as with in-person feedback (Goh & Bambara, 2013), written instructions (Kellems & Morningstar, 2012), and narration (Van Laarhoven & Winiarski, 2012). When compared, video modeling and self-modeling were reported to be equally effective in teaching the acquisition and maintenance of job performance skills (Cihak & Schrader, 2009). Practitioners should consider student level of comfort toward viewing themselves on video when determining which type of VBI to use. The results from the reviewed studies indicated some participants preferred having themselves as the models (Cihak & Schrader, 2009), whereas some participants reported feeling awkward watching themselves on the videos (Hayes et al., 2015). Given these results, practitioners can ask their students which mode they would prefer.

All VBIs in the studies reviewed were implemented by the researchers or teachers who took the role of job coach. The studies revealed that VBIs increased independent participation in integrated competitive work settings, which led to less supervision from job coaches (Gentry et al., 2015; Babb et al., 2018). Given that VBIs were reported as easy to implement and validated to have positive results by the employers (Bross et al., 2018; Kellems & Morningstar, 2012; Burke et al., 2010), practitioners can consider supporting co-workers and employers to implement VBIs as a form of support and accommodation to maintain and sustain high quality of job performance. Many employment settings already use videos to train new employees in certain job skills, and using a VBI as an accommodation for an employee with ASD is reasonable. Practitioners can support employers to understand the purpose and value of the VBI and support fidelity of us on the job. In addition, VBIs can be used to encourage employees with ASD to expand their social communication skills to maintain or even advance their employment status. This is especially critical given the unique contexts of most businesses to include critical social communication attributes.

This systematic review furthers our understanding of the current body of research associated with employment for people with ASD, specifically examining the use of VBIs. From this research, it would appear that VBIs produce positive results toward improving job performance skills; and such positive results suggest promising future studies using VBIs to support people with ASD to learn the skills necessary to find and obtain employment. It would be expected that increasing the scope of the research with anticipated positive findings will increase the employment rate for people with ASD.