Employment supports in early work experiences for transition-age youth with disabilities who receive Supplemental Security Income (SSI)

1.1PROMISE federal initiative

In a response to the low employment rates for youth receiving SSI, the federal initiative known as PROMISE was created. In FY2013, ED funded six standard double-blind model demonstration projects to improve education and employment outcomes of youth SSI recipients and their families. ED awarded cooperative agreements across 11 states to implement the projects, while SSA contracted with Mathematica Policy Research (MPR) to conduct the national evaluation. PROMISE asked the projects to identify innovative methods of providing services and supports to improve outcomes for youth SSI recipients and their families.

1.2AR PROMISE Model Demonstration Project

The Arkansas PROMISE Model Demonstration Project (AR PROMISE) recruited and enrolled 2000 participants in Arkansas between September 2014 and April 2016. At the time of enrollment, youth were 14 to 16 years old and currently receiving SSI benefits from SSA. A random assignment system created and operated by MPR assigned 1027 youth to the program (treatment) group that received the interventions and 973 youth to the usual services (control) group that only received a list of non-AR PROMISE services available to them.

AR PROMISE utilized three primary interventions to improve educational and employment outcomes: intensive case management, supplemental training, and paid work experiences. The focus of this study was the paid work experience intervention specifically, though elements of intensive case management and supplemental training are discussed as well.

AR PROMISE provided at least two work experiences to each program group participant. A work experience (a) took place in an integrated community setting, (b) paid participants at local prevailing wages and (c) consisted of up to 200 hours. The vast majority of these experiences were completed in the summer between 2015 and 2018. All program group youth were scheduled to work in cohorts (400–500) in at least two of the four summers. Over four years, 779 youth recipients of SSI participated in this intervention.

Transition Specialists employed by Arkansas Rehabilitation Services created interest and vocational profiles and assessed the need for supported employment services. The Arkansas Department of Workforce Services and local workforce investment boards provided job readiness training, job development, job placement, and worksite monitoring. Job readiness training consisted of 10–20 hours of instruction in a classroom setting, utilizing a standardized curriculum provided by the local workforce investment boards. AR PROMISE secured employment supports (personal care, job coaching, accommodations, technology, and transportation) utilizing established providers of rehabilitation services and Sources (a Center for Independent Living). AR PROMISE Connectors (case managers) provided coordination and support before, during, and after the summer work experience. Transportation was only offered in summer 2018.

In order to better understand the factors that influence early work experiences for transition-age youth with disabilities, the present study investigated the effects of various employment supports, practices, and youth perceptions on early work experience outcomes for transition-age youth recipients of SSI enrolled in the AR PROMISE model demonstration project.

2.1Participants and procedures

The data for this study was extracted from the PROMISE Management Information System (MIS) and a survey administered at the end of the summer paid job work experience. The PROMISE MIS contained information on duration of employment (hours worked), hourly wages, attendance at job readiness training, and employer information. Staff of the local workforce investment boards entered the data into the MIS at the end of each summer work experience. Additionally, the self-report survey assessed participant perception of the PROMISE work experience including job satisfaction, job coaching, and transportation related experiences.

The AR PROMISE formative evaluation staff administered the survey to participants at 11 sites in August 2018 during scheduled end-of-summer recognition events hosted by local workforce investment boards. Two sites in each region were chosen. Upon entering the site and prior to the start each event, participants were given a paper copy of the survey and a pencil and were instructed to complete the survey prior to the start of the event. Evaluators were on hand to assist with reading comprehension as needed. Participants were asked to answer honestly and given privacy to complete the survey. Surveys were collected from all sites and the data were combined for analysis.

Overall, there were 291 participants enrolled in the AR PROMISE in 2018. However, only 126 participants (43.3%) completed the work experience survey. Other inclusive criteria included participants who (a) enrolled in the research project between August 2014 and April 2016, (b) were 14 to 16 years old at the time of enrollment, (c) were currently receiving SSI benefits at the time of enrollment, (d) participated in the summer work experience intervention in 2018, and (e) completed the work experience survey in August 2018.

Of the total 126 participants who had a paid work experience in the summer of 2018 and took the summer work experience survey, five of them (3.97%) did not respond honestly (i.e., marked the same extreme choices for the entire survey) and 30 (23.81%) had missing value(s) on one or more items. The final sample was comprised of 91 participants (32 females and 59 males) from six geographical regions in Arkansas: central (15.4%), eastern (40.7%), northwest (18.7%), Pulaski (7.7%) and southern (17.6%). At the time of the 2018 summer work experience, the age of the participants ranged between 16 and 20 years with a mean age of 17.75 (SD = 1.05). The demographic information is summarized inTable 1.

2.2Outcome and predictor variables

The outcome variable in this study was the level of hours worked. AR PROMISE set a goal for participants to work at least 200 hours during each summer work experience. Although the working hours form a continuous variable, the general cutting scores were applied due to project’s presetting goal and existing heterogeneous variances. Participants’ working hours were divided into three categories: “1 = 0 – 99 hours” (below expected), “2 = 100 – 199 hours” (approaching), and “3 = 200 and more hours” (met expectations).

Three sets of predictor variables were used in this study. Available data for the 2018 work experience were examined to identify appropriate indicators of job preparation and training, job experiences, and transportation related items. First, there were three items on the dichotomous scale (1 = Yes and 0 = No) related to job preparation and training. Participants were asked to respond to the statements “My Connector assisted me with my first face to face visit with the local work force office”, “I attended a ‘work readiness’ or ‘job club’ class before I started working”, and “I received additional support on the job for my disability”.

Second, two items asked about overall perceptions of the PROMISE work experience (i.e., “My PROMISE job was related to my interests and preferences” and “My PROMISE job gave me experience(s) that will help me in the future”). Originally, both items were on a 5-point Likert-type scale. Data from these items were converted into a dichotomous scale (1 = agree and 0 = neutral or disagree) for consistency with the previous three items.

Finally, to assess participant perception of transportation services, the following items were included in the survey: “In the future, I would be MORE willing to work if transportation was provided”, “I enjoyed working MORE this year because transportation was provided for me”, “Provided transportation was easier than finding my own transportation”, and “Having transportation provided made my working easier on my family”. Respondents indicated the degree to which they agreed with those items on a 5-point Likert-type scale from 1 = strongly disagree to 5 = strongly agree. All items were summed to obtain an overall transportation perception score with possible values ranging from 4 to 20.

2.3Data analyses

SAS version 9.4 was used for all statistical analyses. Given that the outcome of working hours was divided into three ordinal categories (i.e., below expected, approaching, and met expectations), ordinal logistic regression was applied to model the association between summer working hours and potential influence factors. Following Hosmer, Lemeshow, and Sturdivant’s (2013) recommended procedures, first, we conducted univariate logistic regression analyses to examine whether each given indicator was associated with working hours. In addition, alpha level was set to.25 which recommended by Hosmer et al. (2013) in order to account for important variables at the model building stage. A cumulative logit link function was used to model cumulative probabilities of working hours based on the dividing points. To conduct this analysis properly, a score test was used to examine whether all variables met the proportional odds assumption, that is, if the relationship between each pair of outcome groups (i.e., “below expected” vs. “approaching and met expectations combined” and “below expected and approaching combined” vs. “met expectations”) remained the same. Second, multiple logistic regression analysis (i.e., including two or more predictors simultaneously in a single model) was applied to examine effects of each indicator while controlling for other variables in the model. Odds ratios were computed to aid in interpreting relationships between each predictor and the level of working hours.

3.1Univariate associations with summer working hours

Table 2 presents the univariate associations between each individual predictor and the level of summer working hours. Although only one predictor was statistically significant at. 05 level, we still considered including all predictors for the multivariate analysis because the traditional alpha level (i.e.,. 05) might be too conservative and fail to identify important variables (Bendel & Afifi, 1977; Mickey & Greenland, 1989). All individual predictors have met the proportional odds assumption examined by score test. Overall, the level of working hours was positively associated with “connector assistance”, “attended work readiness”, and “PROMISE job was related to my interests”. In contrast, the level of working hours was negatively associated with “received additional support for my disability”, “job gave me experience(s) that will help me in the future”, and perception of transportation.

Odds ratios are also reported to aid in interpreting relationships between each predictor and the level of working hours. Using the item asked about “connector assistance” (i.e., Yes/No) as an example, an odds ratio (e^b) can be described as the probability of “met expectations” versus “approaching or below expected” for those who received assistance (Yes) compared to those who received no assistance (No). Moreover, as constrained by the proportional odds assumption, this estimated odds ratio is also the same for “met expectations or approaching” versus “below expected”.

An odds ratio of one indicates that either dummy coding in the predictor does not yield any change in the likelihood of the level of working hours. Odds ratios above one indicate a positive relationship between the predictor and the level of working hours, while odds ratios below one indicate a negative relationship. For example, the odds ratio for the connector assistance indicator is 2.84, indicating that participants assisted by their connector in the first face to face visit with the local work force office are 2.84 times as likely to reach the 200-hour goal of summer work hours as are participants not assisted by their connector. This is true for both for “met expectations” versus “approaching or below expected” and “met expectations or approaching” versus “below expected”. Additionally, it is important to note that the odds ratios presented in Table 2 are univariate effects and do not control for other predictors in the model.

3.2Multivariate associations with summer working hours

An ordinal logistic regression model was then estimated that included all the potential predictors of summer working hours. Parameter estimates, standard errors, significance test results, and odds ratios for the multivariate model are displayed in Table 3. Most predictors were statistically significant except perception of transportation and “My PROMISE job gave me experience(s) that will help me in the future.” Compared with the univariate analysis, the multivariate analysis does control for other predictors in the model. For example, the indicator of “My PROMISE job was related to my interests and preferences” has an odds ratio of 3.54. As one moves from Neutral/Disagree to Agree in the equation, the odds of “met expectations” versus the combined effect of “approaching or below expected” is 254% higher given all the other predictors are held constant. Similarly, as one moves from Agree to Neutral/Disagree, the odds of the combined effect of “met expectations or approaching” versus “below expected” is also 254% higher given all the other predictors are held constant.

4.1Limitations

This study focused on transportation services, which were only provided in 2018, this narrowed the subject pool down to 291. After examining the data quality, the usable data decreased to 91 participants, which limited statistical power. As the project was stipulated by the funding agency as a standard double-blind research model, AR PROMISE project did not gather data on those participants in the control group. As such, no comparative data was available for this use.

Given these limitations, findings from this study indicate that in-person, initial meetings with service providers by case managers, attendance at work readiness training, and the perception that a job placement was interest-based significantly predicted employment outcomes. Most notably, youth who perceived their job placement as related to their interest and preference were 3.54 times as likely to meet the 200-hour goal as those who were not. Youth whose Connector assisted them with their first face-to-face visit with the local workforce office were 3.73 times as likely to meet the 200-hour goal as youth who did not feel receiving assistance. In addition, youth who ranked higher score on attended a “work readiness” or “job club” class before starting work were 3.3 times as likely to meet the 200-hour goal as who ranked it in lower scores.

Unexpectedly, youth who received additional support on the job for their disability were less likely to meet the 200-hour goal. While included in the results, a youth’s perception that their job gave them experiences that would help them in the future was not a significant predictor. Also, the youth’s perception of transportation services was expected to predict employment success but was not significant in the final model.

Interestingly, while lack of access to reliable transportation was reported by participants in the project, as well as noted in the Introduction from previous research, providing those services did not show to predict better outcomes in the summer work experience. While additional on-the-job support was a negative predictor in the study, further studies using financial and administrative documents would provide continuous variables of job coaching support. This data would provide a larger sample over several years to further explore this variable and its relationship to the summer work experience outcomes.

Despite these lingering questions, these finding do underscore the effectiveness of some services. An initial face-to-face meeting with the local workforce office, participant, and the AR PROMISE Connector increases the likelihood that a participant will successfully complete the work experience intervention. As part of the intensive case management, AR PROMSIE Connector assisted youth in navigating disparate service agencies and coordinating activities.

The individualized job placement chosen collaboratively and informed by vocational assessments and personal goals also increases the likelihood that a participant will successfully complete the work experience. This underscores earlier findings of Customized Employment (Certo & Luecking, 2006; Rogers et al, 2008) and importance of youth involvement in job development and job placement. Finally, work readiness training was shown to increase the likelihood that a participant will reach the 200-hour goal.