Intervention fidelity in a large scale model demonstration project: Lessons from Maryland PROMISE
Abstract
BACKGROUND:
Maryland Promoting Readiness of Minors in Supplemental Security Income (PROMISE) model demonstration project has implemented and is evaluating a collaborative, integrated community-based intervention. PROMISE is designed to increase the likelihood that youth who receive Supplemental Security Income (SSI) in Maryland, and their families, will experience better employment outcomes, increased earnings, and decreased public income support. The intervention was conceptualized from extant research identifying factors, which promote competitive integrated employment outcomes for youth with disabilities.
OBJECTIVE:
In order to effectively assess the impact of the intervention on the participating youth and families it is necessary to ensure that the large-scale statewide project upholds a strong measure of fidelity across implementation sites. That is, does the project deliver the intervention as intended and is it implemented the same way no matter the individual characteristics of the participants and no matter where they live?
RESULTS:
The authors present a descriptive report on the implementation of Maryland PROMISE, illustrate measures of fidelity achieved in serving 997 transition age youth receiving SSI across the state, show preliminary data on early impact, and offer implications for ongoing PROMISE evaluation and its potential influence on policy and practice.
1Introduction
The federal Supplemental Security Income (SSI) program provides a monthly income benefit and medical coverage to youth who are deemed to have a disability and be members of low-income families. That is, youth receiving SSI face the dual challenges of disability and poverty, both of which are associated with high rates of unemployment (Harris & Associates, 2010; Hemmeter, Donovan, Cobb & Asbury, 2015). It is well documented that these youth face several challenges in successfully transitioning to adult life, such as high unemployment (Hemmeter, Kauff, & Wittenburg, 2009) and high drop-out rates from secondary school (Mamun, Carter, Fraker, & Timmins, 2017). Significantly, individuals receiving SSI, either as youth or as adults, have a longstanding and intractably low rate of achieving successful employment outcomes (Davies, Rupp & Wittenburg, 2009; Hemmeter, Donovan, Cobb & Asbury, 2015).
These circumstances stand in sharp contrast to increased expectation of employment outcomes for youth with disabilities that have occurred due to policy shifts as well as changes in special education and employment supports, services and practices. One condition in particular has been identified as contributing to post school employment outcomes for youth with disabilities: work experience, especially paid work experience, during the secondary school years (Carter, Austin & Trainor, 2012; Wehman et al., 2014). In addition, expectations of adult employment for youth with disabilities by their family members also is associated with improved post-school employment success for these youth (Wehman et al., 2014). These developments have coincided with advocacy initiatives to promote the presumption of employability for all people with disabilities, regardless of the nature or significance of the disability (Martinez, 2013). Concomitantly, recent legislative initiatives have in effect endorsed the notion of presumed employability, notably, the Workforce Investment Opportunity Act of 2014 (WIOA, PL 113-128) which requires vocational rehabilitation to presume that anyone who applies, regardless of severity of disability, will benefit from an employment outcome.
Among the public policy challenges is that, on the one hand, there is increasing recognition that presumed employability should guide disability employment service provision. On the other hand, the number of adult SSI recipients, who in order to establish eligibility have paradoxically had to establish their likely inability to become employed, has been rapidly and consistently rising. In the ten-year period from 2005 to 2015, the federal allocation for the SSI program rose by nearly 70 percent as the number of SSI recipients steadily increased to over 8 million (SSA, 2017). The results for individuals with significant disabilities are obvious as long-term dependency on public income support is the very definition of poverty. The consequences to the public at large are represented in the size of the SSI annual allocation which now exceeds $55 billion (SSA, 2017). This allocation will likely grow without efforts to bolster employment prospects of individuals receiving SSI, particularly youth, the majority of whom either remain eligible for SSI upon reaching age 18, or successfully re-apply later (Hemmeter & Gilby, 2009).
The federal response to these issues as they pertain to youth on SSI includes two large-scale randomized control studies intended to rigorously test the applications of specific interventions believed to have potential to address employment and income related challenges of youth SSI recipients. The first, the National Youth Transition Demonstration (YTD), tested a package of interventions that reflected the framework offered by the Guideposts for Success developed by the National Collaborative on Workforce and Disability for Youth (NCWD/Y) (NCWD/Y, 2005). The Guideposts are based on a comprehensive review of research, demonstration projects, and recognized effective practices. The YTD program model included many of the components identified in Guideposts, although these were customized to better meet the needs of the YTD target population (Luecking & Wittenburg, 2009). Foremost among the components were individualized work experiences, including worksite tours; volunteer work; subsidized jobs; and, most notably, competitive paid employment in integrated settings, where youth with disabilities work alongside individuals without disabilities. YTD, which concluded in 2010, demonstrated that, where there were strong applications of employment related services, participants achieved significantly higher employment and earnings outcomes (Fraker et al., 2018). However, results in the six sites were notably mixed. In four of the YTD implementation sites, participants in the treatment group did not achieve employment outcomes different from the control group (Fraker et al., final report). One logical inference from these differential results is that implementation fidelity varied among the six sites. In other words, better outcomes were achieved where YTD service delivery more closely approximated the prescribed intervention.
While the intent of broad scale projects such as YTD is to focus on identifying, implementing and testing interventions across multiple locations and with multiple implementers, a concern often arises about applying the designed intervention with consistent fidelity. Important deviations from ideal protocols commonly occur and can impact outcomes in such studies (Duan et al., 2001). Assessing deviations from these protocols is considered essential to properly and confidently interpret outcomes. For example, Bond et al. (2009) identified two key features of intervention fidelity. One is making structural changes in intervention delivery though administrative action, and another is measuring key process indicators to move toward desired changes in the intervention delivery. That is to say, it is key to fidelity maintenance to measure the intervention delivery and make changes as measurements show deviations. Other studies have confirmed the importance of fidelity adherence, the value of highly skilled consultants and trainers to assist implementers deliver high fidelity interventions, and midstream modifications and improvements (Becker et al., 2001; Becker et al., 2006; Orwin, 2000).
Implementation fidelity is a factor in the second federal large-scale initiative to study ways in which youth SSI recipients can improve adult employment prospects. Promoting the Readiness of Minors in Supplemental Security Income (PROMISE) is a joint federal research demonstration of the U.S. Departments of Education (USDOE), Health and Human Services, and Labor, with evaluation support for the demonstration from the Social Security Administration. As the lead federal partner, the USDOE funded model demonstration projects, one to each of five states and one to a consortium of six states (SSA, 2018). Maryland is one of the six recipients of a PROMISE grant to implement and test a collaborative, integrated community-based intervention designed to increase the likelihood that youth who receive SSI, and their families, will experience better employment outcomes, increased earnings, and decreased public income support. Maryland PROMISE, managed by the Maryland Department of Disabilities, includes a rigorous evaluation through a randomized assignment of youth and their families to either a treatment group receiving the intervention or to a control group which receives those services ordinarily available.
The broad scale implementation of a project the scope of the Maryland PROMISE (statewide), its large number of participants (at least 2,000), and the rigorous research parameters of a randomized control trial study, pose particular operational challenges. In order to effectively assess the impact of the intervention on the participating youth and families, it is necessary to insure that the project upholds a strong measure of fidelity under conditions of statewide implementation. That is, does the project deliver the intervention as intended and is it implemented the same way no matter the individual characteristics of the participants and no matter where they live in the state? This article addresses how well Maryland PROMISE has adhered to this ideal over the course of the five project years (2013–2018) of project operation, presents early service impact data, offers implications for the ongoing evaluation of participant outcomes, and poses considerations for potential policy and practice directions.
2Methods
2.1PROMISE participants
Maryland PROMISE recruited and enrolled 2,007 Maryland youth between the ages 14–16 receiving Supplemental Security Income (SSI) benefits and their families during April 2014 through February 2016. Eligible youth and their families who consented to participate were randomized into one of two groups: 1) enhanced services group (n = 997) and 2) usual services group (n = 1,010). The intervention services received by the enhanced services group are the subject of this article which covers services delivered through September 30, 2018.
2.2The PROMISE intervention
Maryland PROMISE designed an intervention that was guided by both the original federal Request for Proposal (RFP) which required that services be delivered to both eligible youth and their families, and extant research that identifies optimal transition services for youth with disabilities. Emerging research points to family engagement in youth transition and employment pursuits and work experiences during secondary school as strong predictors of post-school employment outcomes (Wehman et al., 2014). In particular, there is a growing body of contemporary evidence that work experience and paid integrated employment during secondary school years predicts successful post-school employment (Carter, Austin & Trainor, 2011; Gold, Fabian & Luecking, 2013; Test et al., 2009; Wehman, et al., 2014). Additionally, there is documentation of the need for low income families to develop financial literacy as they begin to increase earnings (Amsbaugh, 2007). And finally, connections to, and navigation of, the multiple services that may benefit youth with disabilities and their families has been identified as a frequent need (Wehman, et al., 2014). The Maryland PROMISE intervention featured components that recognize the need for research based comprehensive services to both youth and to their families and includes the following:
• Assertive case management: This entails proactive and ongoing coordination of services for the youth and their families so that they can navigate through the services and supports available through the project and through the larger service delivery system for which they are eligible and which may be required to achieve desired outcomes.
• Career and work-based learning experiences: These work experiences include a range of experiences in community-based workplaces such as job shadowing, work sampling, volunteer work, and service learning. However, the chief aspect of this intervention is at least one paid work experience in an integrated setting before leaving high school for participating youth.
• Benefits counseling and financial literacy services: These services include provision of information and counseling on SSA work incentives, eligibility requirements of various programs, earnings rules, as well as financial coaching, counseling and planning.
• Family training and information: This includes supporting family members to have the skills and knowledge to support and to engage in transition planning and other related activities. In Maryland PROMISE, this is not a distinct service, but rather it is integral to and integrated through the assertive case management approach identified above.
The Maryland PROMISE intervention was organized through a team of professionals including a case manager, family employment specialist, benefits specialist, and, when necessary, school personnel who work cooperatively and intensively with youth and families to deliver PROMISE services. Teams operated in each of five state regions: Baltimore, Eastern Shore, Northern Maryland, Southern Maryland, and Western Maryland. Statewide, a total of 27 teams delivered the intervention. In partnership with the Maryland Department of Disabilities, the PROMISE treatment intervention was delivered by Way Station, Inc., an organization with a statewide community and employment service program for individuals with disabilities. TransCen, Inc., an organization experienced in transition interventions and broad scale research projects, provided technical assistance to support program fidelity.
2.3Intervention fidelity
In order to reinforce intervention fidelity, a key feature of Maryland PROMISE was a continuum of technical assistance (TA) designed to build the capacity of the intervention teams to effectively implement the PROMISE service components. TransCen assigned experienced professionals to provide regular in-the-field training and assistance in implementing all aspects of the intervention. Way Station staff attended regular training events and participated in other learning opportunities to gain the knowledge and skills necessary to deliver intervention components. Each region was assigned a TA liaison from TransCen. The liaison provided guidance and support to the project team to promote intervention fidelity and resolve implementation issues as they occurred including field-based TA and one-on-one troubleshooting by the TA liaison with individual staff members.
The delivery of project services are documented by Way Station staff in a management information system customized for this project. Thus, TA was informed by data from the project’s management information system which documented service delivery activity and youth/family participation in components of the intervention. For example, documentation of staff contact with employers to facilitate work experiences might show a need to increase staff efforts and skill in this area of project services.
2.4Measuring fidelity
At the start of the Maryland PROMISE project, six program services or components were identified as essential fidelity indicators measuring the extent to which the intervention was accurately and reliably implemented in adherence to the proposed model. They include:
1. Family Plan which is a regularly updated plan for identifying and coordinating services to meet both short and long term goals for participating youth and family members, including employment and education;
2. Positive Personal Profile which provides a way to establish work preferences, work skills, and the need for accommodation and supports so that the work experience and job development activities for youth participants are individualized;
3. Job Development Plan which is based on the Positive Personal Profile and is the basis for pursuing employer contacts and opportunities for paid work experiences;
4. Benefits Counseling Services to help the youth and family understand work incentives and income management as well as to navigate the SSA requirements for reporting earned and non-SSI income;
5. Unpaid work experience which may include work sampling, job shadowing, and service learning activities; and
6. Paid work experience where the youth receives a wage for work performed.
Recognizing that various factors could inhibit perfect service intervention delivery, we derived a ratio positing that receipt of three to five of these intervention components constitutes partial fidelity. That is, participants received a moderately strong, but not the full, dose of the intervention. Receipt of all six intervention components constituted ideal fidelity. Maryland PROMISE established overall goals for delivering each of the six intervention components as both a measurement for calibration of the dose of intervention delivery and as a management tool for monitoring delivery of service components by project staff so that technical assistance and support could be targeted accordingly.
To determine if there were significant differences in the fidelity of the intervention by region and by disability categories, we used a chi-square test of goodness-of-fit along with effect sizes (Cramer’s V). A non-significant result on for this test suggests that any differences by subgroups are due to chance, while significant results suggest that there may be non-random differences in the proportion of youth who receive the fidelity services by region or disability.
3Results
3.1Service fidelity by component and state region
From the initiation of Maryland PROMISE service delivery in April 2014 through the cessation of PROMISE service delivery on September 30, 2018, fidelity measures reveal an overall consistent level of service delivery. Family Plans were completed for 94% of all PROMISE youth and their families. A Positive Personal Profile was completed for 87% of all PROMISE youth, Job Development Plans were created for 87%, 81% received Benefits Counseling services, 81% participated in an Unpaid Work Experience, and 73% achieved at least one paid employment experience.
A chi-square test of goodness-of-fit was performed to determine whether the youths’ receipt of fidelity services was equally distributed by region for each service. The number of youths receiving Family Plans, Positive Personal Profiles, and Unpaid Work Experiences, was approximately equally distributed by region, X2 (4) = 6.03, 7.85, and 7.04; all p > .05. Meanwhile, the number of youth receiving Job Development Plans, Benefits Counseling, and Paid Work Experiences was not evenly distributed by region, X2 (4) = 13.37, 13.39, and 17.69; p < .05,.05,.01. However, the effect sizes for these differences were in the small range (Cramer’s V < .15), suggesting that any differences in fidelity were marginal. Table 1 below illustrates the descriptive results and the chi-square analysis. The target goals for each component, which were established for project management purposes, are provided at the bottom of the Table.
Table 1
Service Delivery by State Region
| Region | Fidelity Component (n/% total enrolled) | ||||||
| Enrollment (n/% total enrolled) | Family Plans | Positive Personal Profile | Job Development Plan | Benefits Counseling Services | Unpaid Work Experience | Paid Employment | |
| Baltimore | 233 | 214 | 194 | 190 | 185 | 190 | 180 |
| 23% | 92% | 83% | 82% | 79% | 82% | 77% | |
| Eastern | 150 | 143 | 136 | 137 | 116 | 121 | 111 |
| 15% | 95% | 91% | 91% | 77% | 81% | 74% | |
| Northern | 194 | 180 | 175 | 177 | 165 | 165 | 154 |
| 20% | 93% | 90% | 91% | 85% | 85% | 79% | |
| Southern | 158 | 145 | 132 | 132 | 118 | 117 | 98 |
| 16% | 92% | 84% | 84% | 75% | 74% | 62% | |
| Western | 262 | 252 | 227 | 230 | 227 | 213 | 181 |
| 26% | 96% | 87% | 88% | 87% | 81% | 69% | |
| Total | 997 | 934 | 864 | 866 | 811 | 806 | 724 |
| 94% | 87% | 87% | 81% | 81% | 73% | ||
| Project Goals | 100% | 100% | 100% | 95% | 80% | 70% | |
| χ2 | χ2 (4) = 6.03, p = .197 | χ2 (4) = 7.85, p = .097 | χ2 (4) = 13.37, p < .05 | χ2 (4) = 13.39, p < .05 | χ2 (4) = 7.04, p = .13 | χ2 (4) = 17.69, p < .01 | |
3.2Service delivery by disability category
Given the number of disability categories with very low incidence, and for ease of comparison, we condensed the twenty-three impairment categories used by the Social Security Administration into six general disability categories. For example, participants with musculoskeletal disorders (total n = 3) circulatory disorders (n = 3), diseases of the blood (n = 11) and other medical disorders were combined into the single category of “medical disorders.” The six disability categories are reflected in Table 2 below. Across disability categories, the percentage that experienced each of the six service components was at or near the established target with two exceptions. Sixty-two (62) percent of participants with autism spectrum disorders and 63% of those with sensory disability have experienced paid work, compared to 73% of all participants. A chi-square test of goodness-of-fit was performed to determine whether the youths’ receipt of fidelity services was equally distributed across all disability categories for each service. The number of youth receiving services was unequally distributed only for Paid Work, X2 (4) = 12.63, p < .05, and the effect size was in the small range (Cramer’s V < .13), suggesting that any differences in fidelity were marginal. Table 2 illustrates the results across each disability category.
Table 2
Service Delivery by Disability
| Fidelity Component (n/% total enrolled) | |||||||
| Primary Disability | Enrollment (n/% total enrolled) | Family Plans | Positive Personal Profile | Job Development Plan | Benefits Counseling Services | Unpaid Work Experience | Paid Employment |
| Autism Spectrum Disorders | 98 | 94 | 81 | 82 | 79 | 79 | 61 |
| 10% | 96% | 83% | 84% | 81% | 81% | 62% | |
| Sensory Disabilities | 51 | 48 | 41 | 42 | 38 | 39 | 32 |
| 5% | 94% | 80% | 82% | 75% | 76% | 63% | |
| Intellectual/Developmental Disabilities | 262 | 249 | 235 | 232 | 218 | 212 | 201 |
| 26% | 95% | 90% | 89% | 83% | 81% | 77% | |
| Medical Disorders | 60 | 54 | 50 | 49 | 49 | 47 | 42 |
| 6% | 90% | 83% | 82% | 82% | 78% | 70% | |
| Mental Health/Behavioral Disorder | 484 | 451 | 423 | 428 | 384 | 396 | 361 |
| 49% | 93% | 87% | 88% | 79% | 82% | 75% | |
| Other | 42 | 38 | 34 | 34 | 31 | 33 | 27 |
| 4% | 90% | 81% | 81% | 74% | 79% | 64% | |
| Total | 997 | 934 | 864 | 867 | 799 | 806 | 724 |
| 100% | 94% | 87% | 87% | 80% | 81% | 73% | |
| χ2 | χ2 (5) = 3.97, p = .55 | χ2 (5) = 7.17, p = .21 | χ2 (5) = 6.21, p = .29 | χ2 (5) = 3.92, p = .56 | χ2 (5) = 1.35, p = .93 | χ2 (5) = 12.63, p < .05 | |
3.3Service delivery by component and region
Finally, the levels of fidelity achieved were consistent across state regions as illustrated in Table 3. Ideal fidelity, that is, meeting all six fidelity measures, was achieved for 66% of all participants. Eleven percent (11%) received five of the six service components. The percentage of participants who received four or three service components was 7% and 4% respectively. Thus, at least partial fidelity was achieved for 88% of the participants, that is, they received at least three of the six service components. A chi-square test of goodness-of-fit was performed to determine whether the number of service components delivered was equally distributed by region. Only the number of youth receiving six service components and one service component was unequally distributed by region, X2 (4) = 14.12, 13.77, both p < .01, and the effect sizes were both in the small range (Cramer’s V < .15), suggesting that any differences in fidelity were minimal.
Table 3
Service Fidelity by Component and Region
| # Service Components Delivered | State Region (n/% total enrolled) | ||||||
| Baltimore | Eastern | Northern | Southern | Western | Total | χ2 | |
| Six | 146 | 102 | 147 | 92 | 169 | 656 | χ2 (4) = 14.12, p < .01 |
| 63% | 68% | 76% | 58% | 65% | 66% | ||
| Five | 31 | 12 | 14 | 21 | 33 | 111 | χ2 (4) = 6.92, p = .14 |
| 13% | 8% | 7% | 13% | 13% | 11% | ||
| Four | 16 | 13 | 9 | 9 | 26 | 73 | χ2 (4) = 5.76, p = .21 |
| 7% | 9% | 5% | 6% | 10% | 7% | ||
| Three | 9 | 10 | 6 | 10 | 6 | 41 | χ2 (4) = 7.21, p = .125 |
| 4% | 7% | 3% | 6% | 2% | 4% | ||
| Two | 8 | 3 | 4 | 3 | 9 | 27 | χ2 (4) = 1.98, p = .74 |
| 3% | 2% | 2% | 2% | 3% | 3% | ||
| One | 15 | 4 | 2 | 13 | 11 | 45 | χ2 (4) = 13.77, p < .01 |
| 6% | 3% | 1% | 8% | 4% | 5% | ||
| None | 8 | 6 | 12 | 10 | 8 | 44 | χ2 (4) = 4.56, p = .34 |
| 3% | 4% | 6% | 6% | 3% | 4% | ||
| Total | 233 | 150 | 194 | 158 | 262 | 997 | |
4Discussion
Randomized control trials (RCTs), such as used in Maryland PROMISE, are the most certain approach for producing rigorous evidence for the efficacy of the trial intervention. Because participants are randomly assigned either to a treatment group (that receives the intervention) or a control group (that receives either no services or services as usual), any differences between the two groups are a result of the random assignment. If the treatment intervention services are delivered with fidelity, then any differences between the two groups represent the impact of the intervention. Thus, intervention fidelity is essential to effectively gauge the impact of the project.
These results indicate that, overall, the Maryland PROMISE intervention has been delivered with a consistent degree of fidelity. There were few differences in service delivery between regions and across disability category, and those few were statistically marginal differences. Since ideal fidelity was achieved by 66% of participants and since at least partial fidelity was achieved by a total of 88%, there will be a solid basis on which to compare their outcomes with the cohort of participants who did not receive the PROMISE intervention but instead received services as usual.
As the tables above indicate, services were delivered proportionately, with only marginal differences, to the youth and their families regardless of what region of the state they lived and regardless of the primary disability reported. This means that inferences can be made in the final outcome analysis that will not be skewed by disability type and by locations within the state where PROMISE was implemented. In other words, neither disability nor location affected how completely the intervention was delivered.
Further, these findings suggest that it will be possible and advantageous to analyze outcome data not only as an aggregate of services delivered, but also by individual and combined service components. That is, we will be able to analyze outcomes as they relate to the receipt of all six components, a combination of components, or individual components. For example, an analysis will be possible of outcomes of participants who received paid work experiences alone or in combination with other components received.
4.1Limitations
This analysis utilized data provided by in-the-field staff who entered service information at various points. They also uploaded the actual documents related to service delivery, such as the Family Service Plan, the Positive Personal Profile, and the Job Development Plan which provided direct evidence of their completion. Additionally, they recorded the duration of the unpaid and paid work experiences. However, the data used here only counted the incidence, not the duration or any other aspect, of these two services. Similarly, participants received Benefits Counseling Services which ranged from basic information about the existence of work incentives to direct help in calculating and reporting earnings to the Social Security Administration. Thus, we can draw conclusions only from the occurrence of these services rather than their relative intensity. Subsequent analysis of participant outcomes relative to both specific and overall intervention fidelity will need to take this into consideration.
4.2Implications
A well-structured intervention is critical to examining its effects in an RTC study. However, the intervention alone, without assurances of fidelity, will not be sufficient to gauge achievement of project intent and outcomes. Setting project targets and using quantitative data reports to monitor fidelity of the intervention can serve as powerful management tools to suitably conduct the delivery of services consistent with project purposes. It also might be inferred that any transition service, irrespective of association with a research initiative, would benefit from the use of performance management systems that would support similar quantitative reporting on staff activities, along with consequent professional development activities for transition and employment staff who are tasked with delivering program services.
Ultimately, since we will be able to state how closely the intervention in Maryland PROMISE was delivered as intended, we can determine how well the intervention has addressed the challenges faced by youth who, by virtue of SSI eligibility, simultaneously experience significant disability and low family income levels.
Conflict of interest
None to report.
Acknowledgements
The contents of this manuscript were developed under a grant from the U.S. Department of Education (cooperative agreement #H418P130005). However, these contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal government.
References
1 | Amsbaugh, T. ((2007) ). Smart Start: Enhancing access, equity, and opportunity for youth with disabilities. Retrieved October 14, 2008 from http://wid.org. |
2 | Becker, D., Smith, J., Tanzman, B., Dreake, R., & Trembley, T. ((2001) ). Fidelity of supported employment programs and employment outcomes. Psychiatric Services, 52: , 834–836. |
3 | Carter, E. , Austin, D. , & Trainor, A. ((2012) ). Predictors of postschool employment outcomes for young adults with severe disabilities. Journal of Disability Policy Studies, 23: , 1–14. |
4 | Davies, P. ; Rupp, K. ; & Wittenburg, D. ((2009) ). A life-cycle perspective on the ransition to adulthood among children receiving supplemental security income payments. Journal of Vocational Rehabilitation, 30: , 133–151. |
5 | Duan, N., Braslow, J., Weisz, J. , & Wells, K. ((2001) ). Fidelity, adherence, and robustness of interventions. Psychiatric Services, 52: , 413–413. |
6 | Fraker, T. , Cobb, J , Hemmeter, J. , Luecking, R. , & Mamun, A. (in press). Three-Year Impacts of the Youth Transition Demonstration Project. Social Security Bulletin. |
7 | Fraker, T. , Crane, K , Honeycutt, T. , Luecking, R. , Mamun, A. , & O’Day, B. ((2018) ). The Youth Transition Demonstration Project in Miami, Florida: Design, Implementation, and 3-Year Impacts. Journal of Vocational Rehabilitation, 48: , 79–91. |
8 | Fraker, T. , Luecking, R. , Mamun, A. , Martinez, J. , Reed, D. , & Wittenburg, D. ((2016) ). An analysis of 1-Year impacts of Youth Transition Demonstration Projects. Career Development and Transition for Exceptional Individuals, 39: , 34–46. |
9 | Gold, P. , Fabian, E. , & Luecking, R. ((2013) ). Job acquisition by urban youth with disabilities transitioning from school to work. Rehabilitation Counseling Bulletin, 57: , 31–45. |
10 | Harris,L. , & Associates, Inc. ((2010) ). The N.O.D./Harris survey on employment of people with disabilities. New York: Author. |
11 | Hemmeter, J. , Kauff, J. , & Wittenburg, D.C. ((2009) ). Changing circumstances : Experiences of child SSI recipients before and after their age-18 redetermination for adult benefits. Journal of Vocational Rehabilitation, 30: , 201–221. |
12 | Hemmeter, J. , Donovan, M ; Cobb, J. ; & Asbury, T. ((2015) ). Long term earnings and disability program participation outcomes of the bridges transition program. Journal of Vocational Rehabilitation, 42: (1),1–15. |
13 | Hemmeter, J. , & Gilby, E. ((2009) ). The age-18 redetermination and post-redetermination participation in SSI. Social Security Bulletin, Vol. 69: , No. 4. |
14 | Honeycutt, T. ; Wittenburg, D. ; Luecking, R. ; Crane, K. ; & Mann, D. ((2018) ). Potential Strategies to improve the employment outcomes of youth SSI recipients. Final report submitted to the Office of Disability Employment Policy. Washington, DC: Mathematica Policy Research. |
15 | Luecking, R. , & Wittenberg, D. ((2009) ). Providing supports to youth with disabilities transitioning to adulthood: Case descriptions from the Youth Transition Demonstration. Journal of Vocational Rehabilitation, 30: , 241–251. |
16 | Mamun, A. ; Carter, E. ; Fraker, T. , & Timmins, L. , ((2017) ). Impact of early work experiences of subsequent paid employment for young adults with disabilities. Career Development and Transition for Exceptional Individuals, 39: , 154–163. |
17 | NCWD/Y (National Collaborative on Workforce and Disability for Youth). Guideposts for Success, 2nd ed. Washington, DC: Institute for Educational Leadership, (2009) . Available at http://www.ncwd-youth.info/guideposts. Accessed January 4, 2018. |
18 | Orwin, R. ((2000) ). Methodological challenges in study design and implementation: Assessing program fidelity in substance abuse health services research. Addiction, 95: , 309–327. |
19 | Social Security Administration (SSA) ((2018) ). Promoting Readiness of Minors In SSI (PROMISE). Retrieved from https://www.ssa.gov/disabilityresearch/promise.htm. |
20 | Social Security Administration. ((2017) ). SSI annual statistical report, 2014. Publication No. 13-11827. Tables 2 and 4. Washington, DC: SSA. Retrieved from https://www.ssa.gov/policy/ docs/statcomps/ssi_asr/2014/index.html |
21 | Wehman, P. , Sima, A. , Ketchum, J. , West, M. , Chan, F. & Luecking, R. ((2014) ). Predictors of successful transition from school to employment for youth with disabilities. Journal of Occupational Rehabilitation, 25: , 223–234. |
