Experimental analysis of multi-component intervention to support youth in integrated work settings
Abstract
BACKGROUND:
Work-based learning experiences provide secondary students meaningful opportunities to acquire skills needed for long-term employment. Students with disabilities engaging in problem behaviors on the job are vulnerable to having employment opportunities reduced or terminated. One way to develop appropriate work-based behaviors that may promote long-term, competitive integrated employment is to address problem behaviors in high school to ensure students enter the workforce career ready.
OBJECTIVE:
The purpose of this study was to examine the effects of a function-based, self-determined multi-component intervention on the reduction of work-based problem behaviors for secondary youth with disabilities.
METHODS:
This study used a single-case A-B-A-B withdrawal design to examine the effects of the intervention on two student participants’ off-task, work-based problem behaviors.
RESULTS:
Based on visual analysis of graphed data, results indicated a functional relation between the function-based, self-determined multi-component intervention and decrease in off-task, work-based problem behaviors for each student participant. The intervention was rated as socially valid by students and job coaches.
CONCLUSIONS:
This study provides empirical evidence to support implementation of using FBAs and self-determination interventions in combination during work-based learning experiences to decrease work-based problem behaviors for secondary students with disabilities. Limitations and implications for future research and practice are discussed.
1Introduction
Disparities in employment outcomes continue to exist between individuals with and without disabilities. Working-age adults with disabilities (ages 21 to 64) in the United States are less likely than those without disabilities to be employed, and when employed, are more likely to only work part-time (U.S. Department of Labor, 2017). More specifically, 2019 labor force data for individuals with disabilities 16-years-old and over indicated 20.7% of the labor force were individuals with disabilities compared to 68.9% of individuals without disabilities (USDOL, ODEP, 2019). Additionally, data indicate that individuals with disabilities are more likely to experience unemployment than those without disabilities (6.9% vs. 3.2%; USDOL, ODEP, 2019). Given these data, it is not surprising individuals with disabilities are more likely to live in poverty compared to individuals without disabilities (26.6% vs. 10.9%; Erickson et al., 2016). In addition to poverty, unemployment is associated with numerous negative physical and psychological health factors, such as decreased physical, psychological, familial, and social well-being, environmental safety and quality, and overall quality of life (Chapin & Holbert, 2010; Hall et al., 2013; Ra & Kim, 2016; Reichard et al., 2019). These outcomes provide evidence of the deleterious effects unemployment and underemployment have on individuals with disabilities and their families.
There are numerous individual, systemic, social, and environmental barriers to employment for individuals with disabilities (Lindstrom et al., 2016). Specifically, a key factor for why individuals with disabilities struggle to gain and maintain long-term, competitive integrated employment is because of a lack of appropriate social behaviors in the workplace (Agran et al., 2016). Employers have rated work-based social behaviors (e.g., asking supervisor for assistance, clarifying ambiguous instructions) as the most important skills needed to be successful on the job (Lindsay et al., 2016). Unfortunately, individuals with disabilities often fail to acquire these important work-based social behaviors, and employers typically have little tolerance when individuals exhibit problem behaviors on the job (Agran et al., 2016).
One way to develop work-based social behaviors that may promote long-term, competitive integrated employment is to address these behaviors in high school to ensure students enter the workforce ready to work. To prepare students for competitive integrated employment, the Workforce Innovation and Opportunity Act (WIOA; 2014) mandates secondary schools and vocational rehabilitation personnel collaborate to ensure students with disabilities receive high-quality work-based learning experiences (WBLEs). However, students with disabilities often engage in problem behaviors on the job, presenting barriers for job coaches to provide effective instruction that helps students develop important work-based social behaviors. Research indicates secondary schools and teachers are not adequately addressing these behaviors in high school. For example, Agran et al. (2016) found that although secondary teachers perceived work-based social behaviors to be among the most important for job success, these behaviors were not the social skills most frequently taught in classrooms. Agran et al. also acknowledged that more production-ready, social behaviors could be taught during actual work experiences, so students develop and generalize the necessary skills to be successful on the job.
To gain work-based social skills in high school, students with disabilities should be provided opportunities to participate in WBLEs. Paid and unpaid work experiences in high school have been identified as one of the strongest predictors of post-school employment, education, and independent living for youth with disabilities (Mazzotti et al., 2016, 2020). These work experiences may include job shadowing, work sampling, service learning, internships, and/or apprenticeships (Cease-Cook et al., 2015; Luecking & Luecking, 2015). WBLEs provide youth opportunities to explore various career areas, work in different job settings, learn different working styles, and gain necessary work-based social skills to maintain employment overtime (Cease-Cook et al., 2015). These experiences are intended to increase both basic job-seeking and workplace skills to improve post-school outcomes (Luecking & Luecking, 2015).
Although WBLEs provide youth with opportunities to acquire important job-related skills, those who engage in work-based problem behaviors on the job are likely to need more intensive supports to ensure employment can be maintained over time (Hughes et al., 2006; Kittelman et al., 2016). Addressing student problem behavior in work-based settings can be supported by using functional behavioral assessments (FBAs) and implementing function-based interventions in the workplace. FBAs are shown to be effective for gathering information related to identifying the “function” of problem behaviors and developing interventions to address those functions (see Anderson et al., 2015). FBAs have been validated as an effective means for intervening on problem behaviors for students with disabilities across a variety of settings (e.g., Anderson et al., 2015).
When considering the effectiveness of FBAs in schools, conducting FBAs during WBLEs is one strategy teachers and job coaches can use to address work-based problem behaviors. Hughes et al. (2006) conducted FBAs and brief functional assessments with four high school students with moderate intellectual disability (ID) engaging in challenging behaviors, who participated in job training in integrated community settings (i.e., YMCA, book warehouse, nursing home). Target problem behaviors included being off-task, stopping work to talk with others, noncompliance, and banging books. Based on FBA results, function-based interventions were developed to deliver auditory prompts to students engaging in work-based tasks based on the function (i.e., escape or attention-maintained) of each student’s problem behaviors. Results indicated a reduction in problem behavior during job training in the integrated community settings for all participants (Hughes et al., 2006).
Most recently, Nittrouer et al. (2016) examined the use of person-centered planning, FBAs, and a self-determination intervention to reduce work-based problem behaviors of three young adults with autism and/or ID. Results indicated using person-centered planning and FBAs to develop individualized goals and self-management interventions were effective for reducing problem behavior and increasing task completion on the job for all participants. An innovative component to the Nittrouer et al. intervention was the addition of a self-determination intervention (i.e., goal-setting, self-management) to address work-based problem behaviors. There is a plethora of research that indicates involving students in the goal-setting and self-management process can increase self-determination, self-advocacy, and potentially post-school employment outcomes (Mazzotti et al., 2016, 2020; Wehmeyer et al., 2013). Inclusion of a self-determination component in the Nittrouer et al. study provides evidence that involving students in the FBA process and teaching students to self-set goals and manage their behavior on the job may be a beneficial intervention for reducing problem behavior.
Although there is some evidence for using FBAs during WBLEs to address problem behaviors for secondary students with disabilities, further validation is needed. Hughes et al. (2006) and Nittrouer et al. (2016) provided evidence for the use of FBAs in integrated, community-based work settings; however, further research is necessary to provide evidence for the use of FBAs and function-based interventions to reduce problem behavior during integrated WBLEs for secondary youth with disabilities. Additionally, research suggested the need for incorporating self-determination skill development (e.g., goal-setting, self-management) in WBLEs to ensure students with disabilities are successful on the job (Shogren et al., 2018). Mueller and colleagues (2012) also iterated the need to integrate self-determination (i.e., self-advocacy, self-regulation) into the FBA process. Three key transition principles were identified, including: (a) self-advocacy (i.e., student participation in FBA process and intervention implementation); (b) self-regulation (i.e., self-monitoring of behavior); and (c) wraparound service delivery (i.e., interagency collaboration; Mueller et al., 2012). Similarly, Kittelman et al. (2016) outlined procedures for FBAs and function-based interventions that utilize the key principles identified by Mueller and colleagues (2012). Therefore, based on the: (a) strength of research related to using FBAs and function-based interventions in various settings; and (b) limited research to support the use of FBAs in combination with self-determination interventions in WBLEs, the purpose of this study was to examine the effects of a function-based, self-determined multi-component intervention on the reduction of work-based problem behaviors for secondary youth with disabilities. The following research questions were addressed: (a) Was there a functional relation between the use of a function-based, self-determined multi-component intervention and reduction in work-based problem behaviors for two secondary youth with disabilities?; (b) What were job coaches’ and students’ perceptions of using the function-based, self-determined multi-component intervention to decrease student problem behaviors on the job?
2Method
2.1Participants
Participants included two high school students with disabilities, two job coaches, and one special education teacher. Prior to data collection, the researchers obtained Institutional Review Board (IRB) approval from the university and school district. Following IRB approval, written consents were obtained from the special education teacher, job coaches, and parents, and assents were obtained from students indicating willingness to participate in the study. Student participants met the following inclusion criteria for participation: (a) were high school students with disabilities; (b) received services under the Individuals with Disabilities Education Act (2004); (c) participated in an off-campus, integrated, work-based program to learn employment skills; (d) were identified by the special education teacher and job coaches as a student exhibiting low-level (e.g., frequently off-task, non-complaint, disruptive) problem behaviors that impacted work performance at the job site; and (e) were engaging in moderate levels of low-level problem behaviors during initial pre-baseline screening observations on the job.
2.1.1Students
Two 11th grade students with disabilities (pseudonyms used below) participating in a high school, integrated work-based program to gain employment skills participated in this study. Students spent three to four days per week at the job site. Both students participated in a regional high school program that offered students opportunities to gain career readiness skills via WBLEs.
Judah: Judah was a 16-year-old, white male identified with autism spectrum disorder. Based on Judah’s Individualized Education Program (IEP), he was below grade level in math and exhibited deficits in reading comprehension, communication, and social interactions. At the work setting, Judah struggled to stay on-task and often engaged in prolonged conversations with adults.
Baxter: Baxter was a 16-year-old, white male identified with a specific learning disability. Based on Baxter’s IEP, Baxter displayed difficulty in reading fluency, comprehension, written expression, and exhibited social anxiety. At the work setting, he displayed difficulty following directions, engaged in inappropriate contact (e.g., touching, leaning) with job coaches, was easily distracted, and would leave or stop working on his job tasks.
Special education teacher and job coaches: One special education teacher and two job coaches participated in this study. The special education teacher and job coaches were asked to nominate students who would benefit from additional support in their WBLEs. The special education teacher was a white female and had a Master’s degree in Special Education. She had nine years of experience working with students with disabilities (i.e., five as an Educational Assistant, one as a student teacher, three as a classroom teacher). She communicated with employers and determined work-based placements in collaboration with job coaches to ensure students were connected with WBLEs in the community throughout the school year.
Both job coaches were employed by the school district and were white female Educational Assistants responsible for providing work-related, job training supports to students participating in the regional high school program that offered students opportunities to gain career readiness skills via WBLEs. One job coach had 16 years of experience working with students with disabilities and had completed 3.5 years of a bachelor’s degree in physical and recreation therapy. The second job coach had two years of experience working with students with disabilities and a bachelor’s degree in film and television.
2.2Setting
Students were enrolled in a suburban public high school serving 1,139 students located in the Pacific Northwest United States. The school’s student population was 67% White, 15% Hispanic/Latinx, 9% two or more races, 4% Black, 2% Native American, and 1% Asian/Pacific Islander. Student participants took coursework focused on career readiness, which included opportunities to gain employment skill training while in high school. Data collection took place at the high school (i.e., FBA interviews) and job site (i.e., behavior observations, intervention, social validity questionnaires). All training and intervention sessions were conducted at the job site. The job site was a food bank warehouse that partnered with the local high school to provide students with disabilities opportunities to gain experience completing job tasks with employees and volunteers without disabilities in an integrated work environment. The warehouse included a receiving and inspection area for food donated by the community, a stocking dry storage area, and a large walk-in refrigerator and freezer. Students completed job tasks, such as sorting bread, recycling food boxes, labeling food items, reviewing expiration dates, stacking crates, and inspecting produce for quality in the dry storage and refrigerator areas. Three to four days per week (depending on school schedules) between 1:15 p.m. and 3:00 p.m., the two student participants engaged in work-related activities at the job site with both job coaches. Data collection took place during 15 min direct observation sessions at the job site.
2.3Materials
Instructional materials for the intervention included three components. First, a goal-setting lesson (adapted from Rowe et al., 2017) was used prior to intervention to support student participants in setting a short-term behavior goal for work and long-term employment goal. Next, two MotivAiders® (i.e., one per participant; Behavioral Dynamics, 2018) were used during intervention as self-management devices to prompt students to ask job coaches for job-performance feedback. MotivAiders® are lightweight, prompting devices worn on a waistband, belt, or carried in a pant pocket that provide an “adjustable pulsing vibration signal” to prompt an individual to engage in a trained behavior to reach a behavior change goal (Behavioral Dynamics, 2018). Finally, job coaches used an implementation script/treatment fidelity checklist to monitor students during intervention, facilitate meetings at the beginning and end of each work shift, and ensure students correctly used the MotivAider®.
Data collection materials included the Functional Assessment Checklist for Teachers and Staff (FACTS; March et al., 2000). The FACTS (March et al., 2000) is a semi-structured, functional assessment interview measure intended to be completed with school staff and/or parents by interviewers knowledgeable about the functional assessment process. From an analysis of 10 studies using the FACTS with 41 students, McIntosh and colleagues (2008) found the FACTS to have moderate-to-strong psychometric properties, including: (a) reliability for individual or total summary statements (e.g., setting events, antecedents; rs = 0.62 –0.92) and interrater reliability (rs = 0.50–0.88); (b) strong interobserver agreement (100%) and convergent validity with direct observations (90%); and (c) evidence of strong treatment utility and social validity. Additional data collection materials included an adapted Treatment Acceptability Rating Form-Revised (TARF-R; Reimers & Wacker, 1988) to assess job coaches’ perceptions of the intervention, a brief 7-item social validity measure used to assess students’ perceptions of the intervention, and a 10 s partial interval recording form used for direct observation and interrater reliability data collection during baseline and intervention.
2.4Data collection
Two researchers (first and second author) served as primary data collectors for the study. One researcher (first author) held a PhD in special education with an emphasis on secondary transition and students with high incidence disabilities. The second researcher (second author) was a fourth-year doctoral student in special education with an emphasis on secondary students with disabilities and problem behaviors and had over seven years of experience conducting FBAs for students in public school settings. Prior to baseline data collection, data collectors were trained by (a) collecting pre-baseline data on both students to confirm results of the indirect FBA hypothesis statements and to decrease possible student reactivity to data collectors, and (b) developing operational definitions of off-task behaviors developed for both students based on direct observations and indirect FBA information. During observations, observers practiced taking observational data using the 10 s partial interval recording form during 15 min observations, compared data sheets, discussed discrepancies until agreement was 100%, and refined operational definitions of off-task behavior.
2.4.1Dependent variable
The dependent variable was students’ off-task, work-based problem behaviors at the work site. Off-task behaviors were defined as: (a) stopping work tasks to engage in unprompted conversations unrelated to work or tasks on the job; (b) not engaging in assigned work tasks (i.e., attending to co-workers that distract student participants from their work task, walking around looking at items within the warehouse); (c) engaging in inappropriate physical contact with peer workers and/or adult employees or job coaches (i.e., leaning on peer co-worker, touching peer co-workers, grabbing materials from co-workers); and (d) not complying with supervisor directions (e.g., not attending to work task when prompted by supervisor/job coach). Off-task behaviors were measured during 15 min direct observation sessions using a 10 s partial interval recording form. Using the 10 s partial interval recording form, data collectors recorded whether off-task behaviors occurred during any interval during the 15 min direct observation period (Cooper et al., 2007). Direct observation behavior data were collected three to four days per week at the work site across baseline and intervention phases.
Interobserver agreement (IOA): IOA data were collected on the dependent variable for 40.6% of sessions across all phases of the study. To collect IOA, the second observer collected data using the 10 s partial interval recording form at the same time as the primary observer for each student participant. Exact item agreement was recorded if both observers identically scored the interval as off-task. Interval-by-interval comparison was used to determine agreements and disagreements. A disagreement was recorded if intervals were not scored identically. Percentage agreement was calculated by dividing the number of agreements by the number of agreements plus disagreements multiplied by 100. IOA ranged from 77.8% to 100.0% with a mean of 95.2% across all phases. If IOA dropped below 80% during baseline or intervention phases, observers met to discuss disagreements, reviewed operational definitions, and came to consensus on disagreements prior to the next observation.
2.5Social validity
Following the last intervention phase, social acceptability of the intervention was assessed. Job coaches were asked to complete an adapted version of the TARF-R (Reimers & Wacker, 1988). Reimers and Wacker (1988) found the TARF-R to have strong psychometric properties, including high intercorrelations between individual items (e.g., willingness, acceptability). Items were adapted to capture information related to the multi-component intervention in the job setting. The TARF-R assessed job coach perceptions of the social importance of students’ behavior change and the social acceptability of the intervention. The TARF-R is a 15-item, 5-point Likert-type (e.g., 1 = not at all acceptable, 3 = neutral, 5 = very acceptable) social validity measure used to self-evaluate the acceptability, feasibility, effectiveness, and continued maintenance of the multi-component intervention in the job setting. Student participants completed a researcher developed 7-item questionnaire that used a 4-point Likert-type rating scale (i.e., 1 = strongly disagree to 4 = strongly agree). Questions assessed how effective and useable students perceived the intervention to be on the job.
2.6Data analysis
This study used a single-case A-B-A-B withdrawal design (Cooper et al., 2009) to examine the effects of intervention on two student participants’ off-task, work-based problem behaviors. Single-case designs are particularly useful for examining experimental effects for small n studies, where an individual or a small number of individuals are the unit of analysis (Kratochwill et al., 2013). According to What Works Clearinghouse (WWC) single-case design standards (Kratochwill et al., 2013), a withdrawal design requires a minimum of four phases and five data points (e.g., sessions) with each of the four phases meeting rigorous single-case experimental design standards. In addition, six key features of visual analysis (i.e., level, trend, variability, immediacy of effect, overlapping data, consistency of data across phases) were used to (a) assess whether a functional relation occurred and (b) determine strength of the functional relation (Kratochwill et al., 2013).
2.6.1Nonparametric analysis of intervention effectiveness
To examine the statistical impact of the intervention on students’ off-task behaviors, baseline corrected Tau analyses were conducted (Tarlow, 2017). Tau is a nonparametric, nonoverlap approach which controls for positive data trends and assesses within-phase differences across phases (Tarlow, 2017). Within-phase trends and across-phase differences for A-B phase comparisons for each students’ off-task behaviors were calculated. To compute the baseline corrected Tau, a web-based calculator was used (see Tarlow, 2016). If significant positive data trend existed in a baseline phase, a baseline trend correction procedure was used prior to computing Tau estimates. According to Tarlow (2017), using this approach, Tau effect sizes are bounded between – 1 and +1 (depending on the intended direction of the effect). Tau estimates closer to – 1 or +1 indicate a larger Tau effect size (Brossart et al., 2018).
2.7Procedures
2.7.1General procedures
The study was conducted in three phases. First, FBA interviews were conducted with the teacher, job coaches, and students and results were confirmed through direct observations across all work activities at the job setting. Second, a function-based, self-determined multi-component intervention was developed based on the function of each students’ attention-maintained problem behaviors. Third, the intervention was implemented across all work activities at the job setting. Effects of the intervention were documented using a single-case experimental design.
Phase 1 - FBA: During Phase 1, researchers conducted group interviews using the FACTS (March et al., 2000) with the special education teacher and at least one job coach for each student participant. Interviews were also completed with each student participant using the FACTS. The FACTS provided information about: (a) problem behaviors; (b) antecedent triggers; (c) possible setting events (e.g., problematic distant events that may have occurred prior to the student entering the job site); and (d) maintaining consequences to understand the functions or possible reasons why students were engaging in problem behaviors at the job site. The FACTS took approximately 30 min to complete and was conducted prior to direct observations in baseline. For Judah, answers from the FACTS interviews, between the special education teacher and job coaches and the student, were consistent for antecedents, problem behaviors, and maintaining consequences. No setting events were identified across all interviews. Direct observations (pre-baseline) confirmed information collected from the interviews for Judah. Student information collected from the FACTS interviews between the student, job coaches, and teacher interviews for Baxter were consistent for antecedents, behaviors, and setting events information. For consequences, the special education teacher and job coaches indicated that Baxter engaged in problem behaviors to obtain adult attention; however, information collected from the student interview also indicated that he engaged in problem behaviors to obtain peer attention. Pre-baseline observations confirmed the results of the FACTS interviews for Baxter, but also indicated that he engaged in off-task behaviors to attain adult attention and escape work demands (e.g., not moving or participating for an extended period of time in work-related tasks).
2.7.2Expert confirmation
Researchers identified one expert to confirm the hypothesized function of each student’s problem behavior based on information collected from interviews and initial pre-baseline direct observations. Expert verification was completed by a researcher with expertise in conducting secondary transition research and implementing function-based interventions. Researchers solicited expert verification to ensure agreement between authors concerning hypothesized functions of each students’ problem behaviors.
2.7.3Hypothesized function
Based on data collected from the indirect FBA interviews and direct observations (pre-baseline) of student participants engaging in work-based problem behaviors, it was hypothesized both students were engaging in off-task behaviors to obtain adult attention from job coaches and employees. For example, Judah would often stop working on his assigned task and leave his designated work area to talk with adult employees, and Baxter often stopped working to talk with job coaches about sports and engage in inappropriate hugging or touching. Given that results for Baxter indicated both attention and escape maintained behaviors, researchers made the decision to focus specifically on Baxter’s attention-maintained behaviors for the purposes of this study. During FBA interviews, job coaches indicated several strategies had been tried to address the problem behaviors, including redirecting students back to work tasks and offering small breaks during the work shift. Neither strategy resulted in increased long-term, on-task work behaviors.
Phase 2 - Development of function-based, self-determined multi-component intervention: During Phase 2, the intervention was developed by the researchers to address the hypothesized function of both students’ off-task work-based problem behaviors (i.e., obtain adult attention). After direct observations in the first baseline phase, the second author met individually with each student to identify short- and long-term behavior goals for employment using a goal-setting lesson (Rowe et al., 2017). The goal-setting lesson included: (a) reviewing the FACTS with each student; (b) discussing the steps to goal-setting; and (c) defining short and long-term goals. The lesson explicitly walked students through the process of goal-setting, including providing students examples of measurable goals, descriptions of short- and long-term goals, followed by providing students the opportunity to set short- and long-term behavior goals for work. Both students were aware of their lack of engagement in the work setting because both the job coaches and their classroom teacher had discussed their productivity on the job specifically related to off-task behavior. Therefore, both students set personal short-term goals focused on staying on-task during work. Judah’s long-term employment goal was to obtain employment at the foodbank warehouse after high school graduation. Baxter’s long-term employment goal was to attend the local community college after high school graduation to get training for a job. While Baxter’s post-school goal was not directly related to obtaining a specific job after high school, he understood that he would need additional training at the community college to gain work skills. Both students agreed accomplishing their short-term employment goals would better help them accomplish their long-term employment goals after high school.
Following direct observations in the first baseline phase and prior to the first intervention session, researchers trained job coaches during one, 30 min session by (a) reviewing the intervention components, (b) describing and providing examples of specific and positive praise statements, and (c) providing job coaches with implementation scripts/treatment fidelity checklists and describing how to complete them. Researchers trained students individually by (a) describing intervention components and reviewing the daily implementation plan, and (b) having each student role play with job coaches on how to recruit positive behavior-specific feedback from job coaches on their job performance after being prompted by the MotivAider®. Training sessions lasted approximately 20 to 30 min.
To assist students in reaching short-term employment goals, students used a MotivAider®, a self-management device, during each intervention phase. To allow students to self-manage their behavior on the job, the MotivAider® provided a prompt vibration using a fixed interval schedule of reinforcement (i.e., 5 min fixed interval) that prompted students to request feedback from job coaches about their work-based behaviors. Job coaches were instructed to provide brief, behavior-specific feedback and praise to students when prompted. If students did not request feedback when prompted by the MotivAider®, job coaches reminded students to ask for feedback. If students engaged in off-task behaviors during intervention phases, job coaches were instructed to engage in planned ignoring when appropriate to avoid providing adult attention. Additionally, job coaches conducted a brief (1–2 min) meeting at the beginning and end of each daily work shift. At the beginning of the shift, one job coach briefly met with one or both students (i.e., depending on whether the student was in the intervention condition) to review short-term employment goals and provide one to two examples of appropriate on-task behaviors at the job site. At the end of each shift, job coaches briefly met with students to review job performance and provide at least one positive example of how students engaged in appropriate on-task behaviors.
Phase 3 - Documenting intervention effects: During Phase 3, researchers collected behavior observation data for each participant across all study phases. After baseline data collection, the intervention was implemented for each student in a staggered fashion at the work site to ensure change in one student’s behavior was not attributed to change in the other student’s behaviors.
2.7.4Baseline phases
During the first and second baseline phases, observers recorded each student’s work-based problem behaviors to determine level of problem behavior prior to introduction and re-introduction of the intervention. Baseline data were collected at participants’ work site during 15 min direct behavior observations for a minimum of five sessions across baseline phase one and two. During baseline, researchers directly observed each participant completing work tasks and collected data on each participants’ work-based problem behaviors.
2.7.5Intervention phases
Once a stable level and trend was established during baseline phase one, the function-based, self-determined multi-component intervention was implemented. Following a reduction in off-task behaviors below initial baseline levels and collection of a minimum of five data points, the intervention was withdrawn and baseline conditions were re-established. Baseline phase two was re-introduced under the same conditions as baseline phase one to allow for verification of problem behaviors made during baseline one. Once higher levels and an increasing trend of off-task behaviors was established for the students over a minimum of five data points during baseline phase two, the function-based, self-determined multi-component intervention was re-introduced to determine replication of intervention effect.
2.8Treatment fidelity
To ensure intervention procedures were implemented as planned, treatment fidelity data were collected using the implementation script/treatment fidelity checklists. The implementation fidelity script/treatment fidelity checklist was used by job coaches (a) as a self-monitoring measure and (b) to improve fidelity of the implementation of the intervention (Pinkelman & Horner, 2017). This was key, as previous research indicates the accuracy of self-monitoring fidelity does not necessarily correspond to the ability to effectively self-monitor implementation of an intervention to improve fidelity (Broden et al., 1971; Pinkelmen & Horner, 2019). Implementation script/treatment fidelity checklists were completed daily by job coaches during intervention as a method to ensure intervention procedures were followed with fidelity. Steps included: (a) meeting with students prior to each work shift to remind students of short-term employment goals and define on-task behaviors at work (e.g., follow directions, work on assigned task at-hand without getting distracted); (b) providing positive feedback and specific praise when prompted by students during the work shift; and (d) briefly meeting with students at the end of each work shift to provide positive feedback and one example of an on-task, work-related behavior students did well during the work shift. Treatment fidelity data were self-report data completed by job coaches during intervention conditions. Treatment fidelity data were collected across 100% of intervention sessions for both students. Judah’s treatment fidelity data ranged from 75.0% to 100.0% with a mean of 97.0%. Treatment fidelity data for Baxter ranged from 75.0% to 100.0% with a mean of 90.1%.
3Results
Results of off-task, work-based problem behaviors that occurred at the job site over the duration of the study for Judah and Baxter are presented in Fig. 1. Data presented in both A-B-A-B withdrawal designs for each student meet the single-case design standards put forth by the WWC (Kratochwill et al., 2013). Each graph includes student participant data across baseline and intervention phases. Based on visual analysis of graphed data, results indicated there was a functional relation between the function-based, self-determined multi-component intervention and decrease in off-task, work-based problem behaviors for each student participant. Tau estimates indicated positive effects of the intervention for both student participants.
Fig. 1
Percentage of student participants’ off-task, work-based problem behaviors at the work site across baseline and multi-component intervention phases.
3.1Visual analysis
Baseline: Across baseline and intervention phases for both students, the number of data collection sessions ranged from 5 to 11 (M = 7.6). During initial baseline phases, both students engaged in moderate to high levels of off-task behaviors. Judah’s off-task behaviors during first baseline phase ranged from 41.0% to 51.1% (M = 46.7%) and were stable with minimal trend. Judah’s data in the second baseline phase showed a similar pattern in level, in comparison to the first baseline phase; however, data were more variable ranging from 17.8% to 54.4% (M = 38.1%) with an increasing, upward trend. Baxter’s data in the first baseline phase showed more variability and ranged from 20.0% to 64.4% (M = 48.6%) with an increasing, upward trend in off-task, work-based problem behaviors. Compared to Baxter’s first baseline phase, data in the second baseline phase were also variable with an increasing trend and ranged from 31.1% to 97.8% (M = 52.6%).
Intervention: Judah exhibited an immediate decrease in off-task problem behaviors (immediacy effect) when the first and second intervention sessions were introduced. Visual analysis of each intervention phase indicate data were consistent and showed a decreasing trend and low levels of off-task behaviors with limited variability during both intervention phases. Decrease in off-task behavior for the first intervention phase ranged from 34.4% to 4.4% (M = 16.6%) and second intervention phase ranged from 21.1% to 9.0% (M = 12.6%). Similar to Judah, there was a decrease in off-task behaviors for Baxter during the first and second intervention sessions. Baxter’s off-task behavior data across both intervention conditions were more variable than Judah’s. The first intervention phase ranged from 13.3% to 53.3% (M = 31.4%), and second intervention phase ranged from 10.0% to 45.5% (M = 23.7%).
3.2Nonparametric analysis
In addition to visual analysis, results from Tau analyses for Judah indicated both baseline trends in off-task behaviors were non-significant (Tau = 0.33 for first A-B phase and 0.39 for second A-B phase, ps > 0.05). Tau estimates for each A-B phase for Judah indicated that across-phase differences were moderate-to-large (Tau = –0.70 for first A-B phase and –0.70 for second A-B phase, ps < 0.001). For Baxter, results of the Tau analyses indicated baseline trends in off-task behaviors were also non-significant for both baseline phases (Tau = 0.28 for first A-B phase and 0.47 for second A-B phase, ps > 0.05). Results from the simple A-B phase comparisons indicated that the across-phase differences had moderate effects for Baxter (Tau = –0.50 for first A-B phase and –0.52 for second A-B phase, p < 0.05).
3.3Social validity
Social validity questionnaires were provided to job coaches and students following the last intervention phase. Job coaches completed the TARF-R to assess acceptability, feasibility, effectiveness, and continued maintenance of the intervention in the job setting. Overall, job coaches felt the intervention was socially important, acceptable, and feasible. Results of the adapted TARF-R are provided in Table 1. Finally, student participants completed a researcher-developed social validity measure. Results indicated students strongly agreed or agreed the intervention (a) helped them improve their behavior at work, (b) was easy to use during work, and (c) provided a way to focus on a specific job task to improve their work behavior. Additionally, students strongly agreed or agreed they liked learning how to use the MotivAider® to improve their behavior on the job, and overall, the intervention helped them achieve their short-term goals and work towards their long-term employment goals.
Table 1
Results of Modified TARF-R for Job Coaches
| TARF-R Items | Job Coach 1 Score | Job Coach 2 Score | |
| 1. | Given this student’s behavior problems, how acceptable did you find the intervention? (1 = not at all acceptable, 3 = neutral, 5 = very acceptable) | 5 | 5 |
| 2. | How willing are you to carry out this intervention in the future? (1 = not at all willing, 3 = neutral, 5 = very willing) | 5 | 5 |
| 3. | To what extent do you think there might be disadvantages in using this intervention? (1 = none, 3 = neutral, 5 = many) | 2 | 2 |
| 4. | How much time will be needed each day for you to carry out this intervention? (1 = little time will be needed, 3 = neutral, 5 = much time will be needed) | 1 | 1 |
| 5. | How confident are you that the intervention will be effective for this student in the future? (1 = not at all confident, 3 = neutral, 5 = very confident) | 4 | 4 |
| 6. | How likely is this intervention to make permanent improvements in this student’s behavior? (1 = unlikely, 3 = neutral, 5 = very likely) | 4 | 5 |
| 7. | How disruptive was it be to carry out this intervention? (1 = not at all disruptive, 3 = neutral, 5 = very disruptive) | 1 | 2 |
| 8. | How much do you like the procedures used in the intervention? (1 = do not like them at all, 3 = neutral, 5 = like them very much) | 5 | 4 |
| 9. | How willing will other staff members be to help carry out this intervention? (1 = not at all willing, 3 = neutral, 5 = very willing) | 5 | 3 |
| 10. | To what extent are undesirable side-effects likely to result from this intervention? (1 = no side-effects likely, 3 = neutral, 5 = many side-effects likely) | 1 | 2 |
| 11. | How much discomfort is the student likely to experience during this intervention? (1 = no discomfort at all, 3 = neutral, 5 = very much discomfort) | 4 | 2 |
| 12. | How willing would you be to change your routines to carry out this intervention? (1 = not at all willing, 3 = neutral, 5 = very willing) | 5 | 5 |
| 13. | How well did carrying out this intervention fit into the existing routine? (1 = not at all well, 3 = neutral, 5 = very well) | 5 | 5 |
| 14. | How effective was the intervention in teaching your student appropriate behavior? (1 = not at all effective, 3 = neutral, 5 = very effective) | 5 | 4 |
| 15. | How well did the goal of the intervention fit with the team’s goals to improve the student’s behavior? (1 = not at all, 3 = neutral, 5 = very much) | 5 | 5 |
4Discussion
This experimental, single-case study evaluated the effects of a function-based, self-determined multi-component intervention implemented by job coaches to decrease off-task, work-based problem behaviors on the job for two secondary students with disabilities. This study primarily focused on teaching students to request feedback from job coaches on their work performance. Job coaches conducted brief (1–2 min) meetings at the beginning and end of each daily work shift and were trained to deliver specific feedback to students on their behaviors during work shifts. Results demonstrated there was a functional relation between implementation of the intervention and decreased off-task, work-based problem behaviors for both students. Tau estimates indicated the intervention had moderate to large effects for Judah and moderate effects for Baxter. Students and job coaches rated the intervention as socially appropriate on the job and easy to use.
This single-case study adds to the special education and secondary transition literature in several important ways. First, to our knowledge, there have only been a small number of reported experimental studies examining the effects of using FBA procedures to address student problem behaviors in integrated, work-based settings (i.e., Hughes et al., 2006; Nittrouer et al., 2016). This experimental, single-case study adds to the limited literature on using FBA procedures to develop and examine the effects of a function-based, self-determined multi-component intervention to address student problem behaviors in work settings and extends the literature on training and using job coaches as interventionists.
Next, this study provides empirical evidence to support implementation of using FBAs and self-determination interventions in combination in WBLEs to decrease work-based problem behaviors for secondary students with disabilities. Mueller et al. (2012) highlighted the need to integrate self-determination in the FBA process including (a) providing students opportunities to participate in FBA and intervention implementation and (b) self-regulate their behavior. This study provided students the opportunity to participate in student-guided FBA interviews, set short- and long-term employment goals, and self-manage/regulate their behavior using the MotivAider® during WBLEs. Additionally, this study provides empirical evidence to support the need for including self-determination skill development in WBLEs to help ensure students attain positive employment outcomes (Shogren et al., 2018).
4.1Limitations and future research
Despite the positive findings, there are several limitations that should be addressed through future research. First, this study included two students and two job coaches. Although a functional relation was found between the implementation of the intervention by job coaches and decreases in off-task, work-based problem behaviors, additional replication across students, job coaches, researchers, and worksites are warranted (Kratochwill et al., 2013). Future research is needed to examine whether this intervention is effective for decreasing off-task, work-based problem behaviors for other students at-risk for, or with, disabilities in other work settings. Also, this study only reported data on student off-task behavior. Future studies should focus on examining the impact of interventions to increase positive behaviors, such as staying on-task or engaging/increasing work productivity.
Second, as part of intervention, students were taught to recruit feedback from the job coaches on their performance when prompted by their MotivAiders® (i.e., FI5; prompting students every 5 min). This study did not focus on fading the use of the MotivAiders® for the students in the job setting. Future research should examine how to incorporate a stimulus fading procedure to gradually fade (e.g., variable schedule of reinforcement) the use of the device over time (Cooper et al., 2007). Fading strategies could include systematically decreasing the number of prompts delivered to the students over time in the workplace (e.g., FI of 10 or 15 min) or adjusting the device to deliver prompts on a variable interval schedule.
Third, although the intervention was effective in reducing off-task, work-based problem behaviors for both students, Baxter’s off-task behaviors were more variable compared to Judah’s. This was likely due to Baxter’s off-task behaviors being maintained by attention from preferred peers and job coaches in addition to escaping certain work demands. Although the job coaches had previously used breaks as an intervention strategy, this strategy did not decrease off-task behaviors. It is likely the intervention would have been more effective for Baxter, had the intervention included an additional strategy for Baxter to escape or buy-out of certain job demands. Future research should examine whether this intervention would be effective for deceasing other types of problem behaviors at the job site, such as disruptive or non-compliant behaviors that are maintained by adult or peer attention. Future research should also examine whether job coaches can implement this intervention in different types of WBLEs (e.g., grocery stores, university settings, automotive repair shops).
Fourth, the authors of this study recognize the food bank warehouse may not have been a preferred job site for the student participants. Students engage in a variety of WBLEs in a range of job sites to develop preferences as suggested by Cease-Cook et al. (2015). This is a limitation in that students’ problem behavior could have been a result of disinterest in the job. Future research should consider student preference and interest related to selecting a preferred job site (Ninci et al., 2017).
A fifth limitation includes the lapse of time in data collection. Direct observations began in the spring and ended in fall after both participants returned to school and the worksite following summer break. Although implementation of the intervention resulted in decreases in off-task behaviors before and after summer, it is possible maturation effects could have affected the internal validity of the study. For example, over the summer, student participants may have learned other work-related social skills that helped them stay more on-task at the worksite. However, due to high baseline levels of off-task behaviors during the second A-B phase, potential maturation effects did not appear to impact the integrity of the study.
Next, this study demonstrates that training job coaches to implement an effective multi-component intervention can improve on-task, work-related behaviors for secondary students with disabilities participating during WBLEs. In addition, the researchers found including job coaches and students with disabilities in the development of the intervention (i.e., conducting student-guided FBA interviews, setting short- and long-term goals) may have increased buy-in related to implementing the intervention; however, more research in this area is warranted. Future research should investigate job coach implementation and behavior in work-based learning environments.
Another potential limitation of this study was related to social validity data collection. Social validity data were only measured indirectly using the modified TARF-R for job coaches and a researcher-developed measure for student participants. Although student participants and job coaches rated the function-based, self-determined multi-component intervention as being socially acceptable and effective, other more direct indices of social acceptability could have been used. For example, future research should examine whether job coaches continued to use the function-based, self-determined multi-component intervention following completion of the study. In addition, future research should also examine if students are able to generalize the newly learned work-based social skills to other job sites, and whether job coaches would be willing to use the multi-component intervention with other students engaging in similar off-task, work-based problem behaviors.
Finally, an additional limitation worth noting is the authors of this study collected the direct observational data on students’ behavior. Although training job coaches, or other naïve observers, to collect direct observational data would have limited some potential biases, there was a lack of available staff at the job site who could have been trained to be observers. In addition, because students in this study met with the job coaches briefly at the beginning of each session during the intervention conditions and wore MotivAiders®, training and blinding naïve observers to the treatment conditions of the study would have likely not been feasible.
4.2Implications for practice
Findings from this study provide several important implications for supporting secondary students with disabilities participating in WBLEs. This study provides initial empirical support for a low-response effort, multi-component intervention job coaches can use without extensive training or coaching. In addition, prior to this study, both job coaches were not trained in behavior analysis and had limited experience conducting FBAs or implementing function-based interventions. Researchers found that job coaches were receptive to the intervention and able to implement the function-based, self-determined multi-component intervention with high fidelity. Including job coaches into the FBA process and training job coaches to implement function-based interventions may be an effective strategy to support students with disabilities participating in WBLEs.
With WIOA (2014), vocational rehabilitation personnel work with employers to support hiring individuals with disabilities for competitive integrated employment. In an effort to ensure these individuals gain competitive integrated employment, vocational rehabilitation personnel attempt to match jobseekers with high demand jobs according to their knowledge, skills, and abilities. WIOA also mandates that secondary schools and vocational rehabilitation personnel collaborate to ensure secondary students with disabilities receive high-quality, pre-employment transition services, which includes training in WBLEs to prepare students with disabilities for competitive integrated employment. However, when secondary students with disabilities engage in work-based problem behaviors on the job, it presents barriers for job coaches to provide effective instruction and job coaching. Further, job coaches typically do not have extensive training or experience using evidence-based instructional strategies in daily practice (Brock et al., 2016). Therefore, local education agencies, schools, and vocational rehabilitation personnel should identify methods for providing professional development to personnel providing WBLEs to students with disabilities to ensure these personnel are effectively prepared to support students on the job.
Conflict of interest
The authors declare that they have no conflict of interest.
Funding
This project was not grant funded.
References
1 | Agran, M. , Hughes, C. , Thoma, C. A. , & LaRon, S. A. ((2016) ). Employment social skills: What skills are really valued?. Career Development and Transition for Exceptional Individuals 39: (2), 111–120. Doi: 10.1177/2165143414546741 |
2 | Anderson, C. M. , Rodriguez, B. J. , & Campbell, A. ((2015) ). Functional behavior assessment in schools: Current statues and future directions. Journal of Behavioral Education 24: (3), 338–371. Doi: 10.1007/s10864-015-9226-z |
3 | Behavioral Dynamics. (2018). MotivAider. Retrieved from http://habitchange.com/ |
4 | Broden, M. , Hall, R. V. , & Mitts, B. ((1971) ). The effect of self-recording on the classroom behavior of two eighth-grade students. Journal of Applied Behavior Analysis 4: (3), 191–199. Doi: 10.1901/jaba.1971.4-191 |
5 | Brock, M. E. , Cannella-Malone, H. I. , Schaefer, J. M. , Page, E. J. , Andzik, N. R. , & Seaman, R. L. ((2016) ). Efficacy of training job coaches to implement evidence-based instructional strategies. Journal of Vocational Rehabilitation 45: (3), 351–364. Doi: 10.3233/JVR-160835 |
6 | Brossart, D. F. , Laird, V. C. , & Armstrong, T. W. ((2018) ). Interpreting Kendall’s tau and tau-} for single-case experimental designs. Cogent Psychology 5: , 1518687. Doi: 10.1080/23311908.2018.1518687 |
7 | Cease-Cook, J. , Fowler, C. , & Test, D. W. ((2015) ). Strategies for creating work-based learning experiences in schools for secondary students with disabilities. TEACHING Exceptional Children 47: (6), 352–358. https://doi.org/:10.1177/0040059915580033 |
8 | Chapin, M. H. , & Holbert, D. ((2010) ). Employment at closure is associated with enhanced quality of life and subjective well-being for persons with spinal cord injuries. Rehabilitation Counseling Bulletin 54: (1), 6–14. Doi: 10.1177/0034355210367685 |
9 | Cooper, J. O. , Heron, T. E. , & Heward, W. L. ((2007) ). Applied behavior analysis (2nd ed.). Pearson. |
10 | Erickson, W. , Lee, C. , & von Schrader, S. ((2016) ). 2015 Disability Statues Report: United States. Cornell University Yang Tan Institute on Employment and Disability (YTI). |
11 | Hall, J. P. , Kurth, N. K. , & Hunt, S. L. ((2013) ). Employment as a health determinant for working-age, dually-eligible people with disabilities. Disability and Health Journal 6: (2), 100–106. Doi: 10.1016/j.dhjo.2012.11.001 |
12 | Hughes, M. A. , Alberto, P. A. , & Fredrick, L. L. ((2006) ). Self-operated auditory prompting systems as a function-based intervention in public community settings. Journal of Positive Behavior Interventions 8: (4), 230–243. Doi: 10.1177/10983007060080040501 |
13 | Kittelman, A. , Bromley, K. W. , & Mazzotti, V. L. ((2016) ). Functional behavioral assessments and behavior support plans for work-based learning. Career Development for Exceptional Individuals 39: (2), 121–127. Doi: 10.1177/2165143416633682 |
14 | Kratochwill, T. R. , Hitchcock, J. H. , Horner, R. H. , Levin, J. R. , Odom, S. L. , Rindskopf, D. M. , & Shadish, W. R. ((2013) ). Single-case intervention research design standards. Remedial and Special Education 34: (1), 26–38. Doi: 10.1177/0741932512452794 |
15 | Lindsay, S. , Hartman, L. , & Fellin. M. ((2016) ). A systematic review of mentorship programs to facilitate transition to post-secondary education and employment for youth and young adults with disabilities. Disability and Rehabilitation 38: (14), 1329–1349. Doi: 10.3109/09638288.2015.1092174 |
16 | Lindstrom, L. , Hirano, K. , & Thomas, R. ((2016) ). Career development for individuals with disabilities: Examining issues of equity, access and opportunity. In Broadbridge A. & Fielden S.L. (Eds.). Impact of Diversity on Career Development. USE Edward Elgar Publishing. |
17 | Luecking, D. M. , & Luecking, R. G. ((2015) ). Translating research into a seamless transition model. Career Development and Transition for Exceptional Individuals 38: (1), 4–13. Doi: 10.1177/2165143413508978 |
18 | March, R. E. , Horner, R. H. , Lewis-Palmer, T. , Brown, D. , Crone, D. , Todd, A. W. , et al. ((2000) ). Functional Assessment Checklist for Teachers and Staff (FACTS). Educational and Community Supports, University of Oregon. |
19 | Mazzotti, V. L. , Rowe, D. A. , Sinclair, J. , Poppen, M. , Woods, W.E. , & Shearer, M. ((2016) ). Predictors of post-school success: A systematic review of NLTS2 secondary analyses. Career Development and Transition for Exceptional Individuals 39: (4), 196–215. Doi: 10.1177/2165143415588047 |
20 | Mazzotti, V. L. , Rowe, D. , Kwiatek, S. , Voggt, A. , Chang, W. , Fowler, C. H. , Poppen, M. , Sinclair, J. , & Test, D. W. ((2020) ). Secondary transition predictors of post-school success: An update for the field. Manuscript under review. |
21 | McIntosh, K. , Borgmeier, C. , Anderson, C. M. , Horner, R. H. , Rodriguez, B. J. , & Tobin, T. J. ((2008) ). Technical adequacy of the Functional Assessment Checklist: Teachers and Staff (FACTS) FBA interview measures. Journal of Positive Behavior Interventions 10: (1), 33–45. Doi: 10.1177/1098300707311619 |
22 | Mueller, T. G. , Bassett, D. S. , & Brewer, R. D. ((2012) ). Planning for the future: A model for using the principles of transition to guide the development of behavior intervention plans. Intervention in School and Clinic 48: (1), 38–46. http://doi.org/10.1177/1053451212443130 |
23 | Ninci, J. , Gerow, S. , Rispoli, M. , & Boles, M. ((2017) ). Systematic Review of Vocational Preferences on Behavioral Outcomes of Individuals with Disabilities. Journal of Developmental & Physical Disabilities 29: , 875–894. https://doi.or/10.1007/s10882-0179560-2 |
24 | Nittrouer, C. L. , Shogren, K. A. , & Pickens, J. L. ((2016) ). Using a collaborative process to development goals and self-management interventions to support young adults with disabilities at work. Rehabilitation Research, Policy, and Education 30: (2), 110–128. Doi: 10.1891/2168-6653.30.2.110 |
25 | Pinkelman, S. E. , & Horner, R. H. ((2017) ). Improving implementation of function-based interventions: Self-monitoring, data collection, and data review. Journal of Positive Behavior Interventions 19: (4), 228–238. Doi: 10.1177/1098300716683634 |
26 | Ra, Y.-A. , & Kim, W. H. ((2016) ). Impact of employment and age on quality of life of individuals with disabilities: A multilevel analysis. Rehabilitation Counseling Bulletin 59: (2), 112–120. Doi: 10.1177/0034355215573538 |
27 | Reichard, A. , Stransky, M. , Brucker, D. , & Houtenville, A. ((2019) ). The relationship between employment and health and health care among working-age adults with and without disabilities in the United States. Disability and Rehabilitation 41: (2), 2299–2307. Doi: 10.1080/09638288.2018.1465131 |
28 | Reimers, T. M. , & Wacker, D. P. ((1988) ). Parents’ rating of the acceptability of behavioral treatment recommendations made in an outpatient clinic: A preliminary analysis of the influence of treatment effectiveness. Behavioral Disorders 14: (1), 7–15. Doi: 10.1177/019874298801400104 |
29 | Rowe, D. A. , Mazzotti, V. L. , Ingram, A. , & Lee, S. ((2017) ). The effects of goal-setting instruction on academic engagement for students at-risk. Career Development and Transition for Exceptional Individuals 40: (1), 25–35. Doi: 10.1177/2165143416678175 |
30 | Shogren, K. A. , Burke, K. M. , Anderson, M. H. , Antosh, A. A. , Wehmeyer, M. L. , LaPlante, T. , & Shaw, L. A. ((2018) ). Evaluating the differential impact of interventions to promote self-determination and goal attainment for transition-age youth with intellectual disability. Research and Practice for Persons with Severe Disabilities 43: (3), 165–180. Doi: 10.1177/1540796918779775 |
31 | Tarlow, K. R. ((2017) ). An improved rank correlation effect size statistic for single-case designs: Baseline corrected tau. Behavior Modification 41: (4), 427–467. Doi: 10.1177/0145445516676750 |
32 | Tarlow, K. R. ((2016) ). Baseline Corrected Tau Calculator. http://www.ktarlow.com/stats/tau |
33 | U.S. Department of Labor. (2017). Persons with disability: Labor force characteristics- 2016 (USDL-17-0857). https://www.bls.gov/news.release/disabl.nr0.htm |
34 | United State Department of Labor, Office of Disability Employment Policy. (USDOL, ODEP; 2019). November 2019 disability employment statistics: Ages 16 years and over. https://www.dol.gov/odep/ |
35 | Wehmeyer, M. L. , Palmer, S. B. , Shogren, K. , Williams-Diehm, K. , & Soukup, J. H. ((2013) ). Establishing a causal relationship between intervention to promote self-determination and enhanced student self-determination. The Journal of Special Education 46: (4), 195–210. Doi: 10.1177/0022466910392377 |
36 | Workforce Innovation and Opportunity Act, 29 USC § 3101-3361 (2014). |
