Centering communities, constellations and networks of practice to improve youth post-school outcomes through PROMISE

1Introduction

Practices in transition to adulthood for youth with disabilities continue to evolve, and recommendations are emerging for specific services to help youth and young adults mitigate barriers, develop human and social capital, and enhance their employability to match the needs of the contemporary labor market (Mazzotti et al., 2016; Rowe et al., 2013; Test et al., 2009). There are continued efforts to identify and promote effective practices to improve outcomes for youth with disabilities at the group level, and studies have continued to demonstrate the need for local-level adaptation of these practices to fit to the living and learning context for these young people. Local-level adaptation of effective practices is especially relevant when programs are scaled up.

Providing developmentally appropriate services for transition age youth with a focus on achieving meaningful and relevant outcomes results in program adaptation during scale-up.. However, substantive local customization may include deviations that can induce heterogeneity in program parameters (Blakely et al., 1987; Quin & Kim, 2017). The landscape of service delivery systems for youth and young adults is evolving in response to the Individuals with Disabilities Education Act (IDEA) and the recently enacted Workforce Innovation and Opportunity Act (WIOA), necessitating micro-level adaptation of transition to adulthood programs. In this context, scaling-up transition to adulthood programs and best practices requires a complementary learning system that advances local knowledge, as well as implementation of strategies equipping key stakeholders in education, workforce development, community-based mental health and other pertinent services for youth with disabilities.

Several approaches to customized and continual learning have recently been explored and promoted to engage diverse community-based practitioners to enhance their ability to adapt broad sets of practices to their local contexts (Povenmire-Kirk et al., 2015). In some instances, programs leveraging technology have been examined to understand their ability to create ‘learning systems’ in transition to adulthood practices. The theory of situated learning and communities of practices is the common underpinning of these approaches. This paper describes a learning system that was established not only to implement evidence-based practices in transition to adulthood for vulnerable youth with disabilities, but also to help support implementation of NYS PROMISE (New York State Promoting the Readiness of Minors in Supplemental Security Income), a large-scale randomized controlled trial measuring efficacy and effectiveness of a model demonstration program. By explicating the theoretical components of ‘center of practice’ and connecting them to program quality improvement efforts, this paper provides a unique perspective that has potential to enable scale-up of transition to adulthood programs at the county, region, state, or national level. Further, the paper also explores the concept of the “Center of Practice” as a feasible approach in implementation of large-scale randomized control trial model demonstration program for measuring its efficacy and effectiveness.

2Introduction to situated learning, communities of practice and PROMISE

Lave and Wenger (1991) proposed that the social and cultural context of the learner, i.e., the learner’s lived experience on a day-to-day basis, is a foundational element of learning, and that learners become involved in a community of practice (CoP) where there is a shared commitment to develop or improve behaviors needed to effect change. Wenger (1998) further refined the process of CoP into three distinct phases – mutual engagement, where members interact establishing norms and relationships; member engagement, to establish joint enterprise through shared commitment; and production of shared repertoires, in form of artifacts that help reify negotiated meanings or practices.

Viewed from this perspective, CoPs have potential applications in the area of transition to adulthood for youth with disabilities. The increasing emphasis on improving the growth and stability in youth employability via collaboration between education, labor programs, businesses, health care and social safety-net programs calls on practitioners from these agencies to have a shared understanding of youth needs and experiences, and to support them with customized services. Practitioners and institutions serving youth face challenges in creating this shared understanding. These challenges emerge not only from their need for information, but also from their institutional cultural environments that can impact the process of sharing and interpreting information across agencies leading to fragmentation of services for youth served. CoPs have the potential to create ‘negotiated’ understandings to address these challenges, which are crucial to overcoming systemic inertia by practitioners and institutions in improving service coordination and case management. CoPs have shown limited impact in fostering negotiated understandings, mostly resulting from their conceptual limitations and a lack creativity of their application.

Roberts (2011) proposed a typology of CoPs based on a critical review of CoP literature. This typology included situated practice involving mutual engagement (community of practice); situated practice achieved through boundary spanners/brokers (constellations of practice); and, dislocated practice (networks of practice which can occur at the local, regional, national, global or virtual level). This typology of CoPs (Roberts, 2011) is influenced by the understanding that social interactions among CoP members that are spatially distributed often occur without a direct impact on practice. Further, Brown and Dugid (2001) describe a term called “network of practice” to suggest existence of multiple CoPs within a large framework of practice-based communities, creating local communities with access to others ‘in-network’ and thus creating constellations of practices. However, these interactions may present opportunities for learning across other CoPs through boundary-level processes of harmonizing situated experiences to generalizable principles (i.e., methods standardization) and may create boundary objects – materials and resources that summarize specific as well as generalizable information – that percolate across the CoPs within networks or constellations of practices (Star & Griesemer, 1989). Differentiation of communities of practice recognizes the variance in size, spatial scaling, and nature of practice.

Based on this discourse in the field of distributed CoPs, we describe the multidimensional CoP of the NYS PROMISE program as consisting three primary domains: 1) community of practice focused on the local implementation of a program requiring intensive collaborations across education, vocational rehabilitation and pertinent social welfare programs; 2) a broad, constellation of practice formed through regional networks and dislocated practice federated at state-levels; and 3) these parts together forming a multi-faceted center of practice.

Figure 1 illustrates a construct that bounds the learning domains of the center of practice that needed to be considered.

Fig.1

Nested NYS Learning Construct.

The deeper understanding gained through this construct of the complexity of the NYS PROMISE initiative aided the learning community architects in understanding that this was not a singular community of practice, constellation of practice or dislocated practice, but rather something new; a collective ‘center of practice’ purposely formed around a large-scale social issue, “how do we improve the provision and coordination of services and supports for child SSI recipients and their families to enable them to achieve improved outcomes.” [PROMISE RFP, 2013]

Next, we discuss how a “Center of Practice” approach was utilized in New York State to re-calibrate the context for systems and organizational learning in a statewide experimental research demonstration project: Promoting the Opportunities of Minors Receiving Supplemental Security Income (PROMISE).

3The NYS PROMISE model

Promoting the Readiness of Minors in Supplemental Security Income (PROMISE) is a joint federal research demonstration launched in 2013 by the I.S. Departments of Education (USDOE), Health and Human Services, and Labor, with evaluation support for the demonstration from the Social Security Administration. As the lead federal partner, the USDOE funded six model demonstration projects to address barriers and obstacles to economic independence and promote successful education and employment post-school outcomes for youth who receive SSI. Youth SSI recipients between the ages of 14 to 16 were eligible to enroll in the PROMISE demonstration. The USDOE provided approximately $230 million over a five to six year period to the demonstration projects in Arkansas; California; Maryland; New York; Wisconsin; and a six-state consortium which included Arizona, Colorado, Montana, North Dakota, South Dakota andUtah.

The NYS PROMISE intervention model is designed to support schools and communities in providing high quality transition services to youth who receive SSI, while recognizing that the needs of each community and student are unique (Karpur, Brewer, & Golden, 2013). The NYS PROMISE project engaged existing local, regional and state transition stakeholders in the implementation of the demonstration, including: local school-based and community-based case managers; Parent Center family coaches; and a network of community service providers. Across three diverse regions of NY State (i.e., Western NY, the NY Capital Region, and New York City), the following program features are core to the NYS PROMISE:

In addition to this complicated structural arrangement, the case managers, family coaches and service providers developed contextually grounded strategies for communications to ensure youth are engaged in the program, and services and ancillary services meet families’ expectations. The Center of Practice approach supported the advance brokering of local solutions, enhanced cross-system learning among practitioners embedded across systems and facilitated the development of strategies at regional- and state-level to standardize implementation of the ‘core’ program components of the NYS PROMISE. Figure 2 outlines the activities that make up the NYS PROMISE intervention model and describes the specific domains and practices for the NYS PROMISE center of practice.

Fig.2

NYS PROMISE Intervention Domains.

Beyond managing the inherent geographical distances among key implementation partners, the NYS PROMISE also had to knit together the youth and adult disability systems that have direct bearing on not only the quality of services provided, but also the quality of outcomes achieved. The entitlement-driven approach to service provision within schools suddenly shifts to an individual responsibility-driven approach in the adult serving system. The WIOA and its requirements to serve youth while in-school has required state vocational rehabilitation agencies to realign practices and approaches in order to serve people within school systems. These differences in approach create a mosaic of services delivered in ways that affect the overall quality of services and resulting outcomes. A recent needs assessment conducted by Cornell University for the New York State Vocational Rehabilitation agency (ACCES-VR) identified substantial regional variations in the ability of state vocational rehabilitation agencies to be responsive to youth and young adults (Chang, Cook, Golden, Karpur, Malzer, Saleh & VanLooy, 2018). Regions that were most effective in delivering services to youth were more likely to have higher levels of experience in serving diverse population of youth and to have an open organizational climate that allowed for customization in delivery mechanisms. Therefore, the meso-level regional collaborating units of the NYS PROMISE, while rooted in similar systems, are sufficiently heterogeneous in terms of their practices in offering seamless transition to adulthood services.

4NYS PROMISE Center of Practice

The NYS PROMISE represents a multi-faceted center of practice at the macro, meso, and micro levels. This includes the state-level learning community at a broad macro or systems level, as well as the project as a whole across the three regional demonstration sites. The three regional demonstration sites, representing geographic communities of practice engaged in regionally-based issues and activities, are the meso-level of the community of practice. The final facet of this community of practice includes the specific, or micro, practice domains within the demonstration sites – case managers, family coaches, services providers, etc.

To address challenges and limitations associated with the community of practice model and to embrace the collective of communities of practice needed to achieve the intended outcomes, the NYS PROMISE Center of Practice developed and implemented the following strategies:

5Standardization methods

5.1Group concept mapping (GCM) for articulating challenges and opportunities

Concept mapping is a structured conceptualization method where a set of diverse stakeholders develop a conceptual model or map that can be used for a variety of strategic planning, problem solving, and evaluation purposes. This project employed Group Concept Mapping (GCM), a type of structured conceptualization developed by William Trochim of Cornell University (Novak 1998, Trochim 1989a). Employing the Concept System^® typically involves six defined steps: (a) brainstorming by key stakeholders around a focus prompt to generate a list of statements, (c) unstructured sorting of similar statements and rating of statements (d) computation of the map utilizing multidimensional scaling, (e) interpretation of the cognitive maps, and (f) implementation of the cognitive insight gained from the maps and process. A full step-by-step description of this method is out of the scope for this article, but can be found in Trochim & Trochim (2007).

The project conducted multiple GCMs to support continuous quality improvement across policy, practice, and partnership in the NYS PROMISE initiative. At the micro level, the GCMs annually helped to identify the facilitators of and challenges to achieving student, family, and project success across the project community. They also supported the state (macro) level Steering Committee in identifying opportunities for enhanced partnership, policy, and practice across state agency partners and developing a strategic plan for continuous quality improvement at the regional (meso) and state (macro) level. The GCMs were implemented at three-levels:

• • NYS PROMISE Steering Committee (macro)

• • NYS PROMISE Stakeholders (Learning Community) (meso)

• • NYS PROMISE Service Providers (micro)

Figures 3 and 4 illustrate the strategic planning and partnership framework outputs from the GCM.

Fig.3

Current Presence, importance and Viability Rating Data Comparison. NYS PROMISE (2016). Discovery and strategic partnership group concept mapping: 2014–2015 progress report. Concept Systems Inc., and Cornell University: Ithaca, NY, p. 5.

Fig.4

Relative Pattern Match Comparing Importance and Current Presence: All Participants. NYS PROMISE (2016). Discovery and strategic partnership group concept mapping: 2014-2015 progress report. Concept Systems Inc., and Cornell University: Ithaca, NY. p. 5.

6Boundary objects

6.1OTAC web-resource, project manuals, and structured coaching including expert inputs for modelling

The NYS PROMISE developed an Online Technical Assistance Center (OTAC) www.nyspromise.org/ secure was designed to track data and provide ongoing support to the Center of Practice by serving as a repository for boundary objects developed by the CoP. OTAC is a password protected site used by all project staff. It provides varying access to materials depending on the individual’s project role and responsibilities, and is an archive of project-related materials, tools, and created trainings. The OTAC supported sharing of information through summary reports across all three regions.

The NYS PROMISE project was divided into four distinct, yet overlapping, phases: Outreach and Recruitment, Case Management and Service Delivery, Retention and Engagement, and Quality and Assessment. Each phase was represented in OTAC as a dial with five categories; Discussion, Tools, Evidence, Innovations, and Training. As resources were created, they were added to the appropriate dial and category, and made available for use by the NYS PROMISE project community (Figure 5).

Fig.5

NYS PROMISE Online Technical Assistance Dashboard Source: http://www.nyspromise.org/secure/lcDials.cfm.

The implementation of this model was supported by instructional scaffolding, such as manuals and tools, and implemented across the center of practice to promote effective demonstration implementation. These manuals and tools included: a Policy and Procedures Manual detailing specific policies and procedures associated with the intervention; an Outreach and Recruitment Manual describing the recruitment and outreach process and practical guidance; an Intervention and Implementation Guide providing information about New York State’s PROMISE program and field implementation; and a Program Fidelity Rubric and Site Visit Protocol to track the degree to which the NYS PROMISE interventions were implemented by partners as prescribed, the confidence and satisfaction of partners in implementation, and regional and cultural differentiation. The Policy and Procedure Manual clearly articulated processes for implementation of the program and was a reference source commonly referred to and accessed frequently across trainingand capacity-building efforts. The Intervention and Implementation Guide became a field resource for practitioners across all levels of the CoP for ensuring that treatment and dosage held to specific guidelines—ensuring continuity across the state, regions and disciplines. Finally, a Case Manager Field Guide was created to standardize practice by case managers and family coaches in navigating day-to-day roles, functions, and expectations, while adhering to high quality standards. The Field Guide ensured that all case managers and family coaches were equipped to perform their job in a manner that led to the expected youth and family outcomes of the NYS PROMISE intervention—holding to a common measure of fidelity.

6.2Management by data and reflexive engagement for quality improvement

The NYS PROMISE project is data intensive, with a focus on collecting data on program implementation that include the dimensions of adherence, quality of delivery of program services, exposure and engagement of participants in services, and treatment differentiation by comparing the treatment and control conditions. Multiple methods were deployed to collect this data and Table 1 summarizes various data collection processes in the program.

Table 1

Summary of NYS PROMISE Data Collection Activities

New York State Employment Services System (NYESS)	Documenting individual program activities and services and for making PROMISE service referrals
Youth and Family Survey (YFS)	Gathers first-hand information and impressions from both the youth participating in PROMISE and their parents
Program Fidelity Qualitative Site Visits	Ascertain program structure, completeness in recordkeeping and adherence to program processes, as well as to provide critical on-site and in-person technical assistance for parent centers (PC) and Research Demonstration Sites (RDS)
OMH Audit of Service Providers	Document individual participant program activities and services and for making PROMISE service referrals
Focus Group – PROMISE Youth and Families	Gathers first-hand information and impressions from both the youth participating in PROMISE and their parents
Parent Center Core Training Evaluations	Includes attendance, usefulness and uptake of training,

These strategies were incorporated across the core elements for the center of practice and are described in further detail below. Outputs from the analysis of various sources of data allowed for a reflective engagement through the PROMISE Learning Communities leading to action planning and change. Figure 6 illustrates the longitudinal change in the patterns of one aspect of the programs –student recruitment.

Fig.6

Longitudinal changes in student recruitment.

This figure illustrates the way in which periodic reflection at bi-annual learning communities, coupled with continual sharing of this information led to overall project achieving its recruitment target ahead of the schedule time. Similar outputs continue to be developed to enable key program implementation activities.

7Discussion and implications

The purpose of this paper was to describe an approach to the Center of Practice concept, grounded in the theory of Communities of Practice, utilized in the implementation of a large-scale experimental design program to support vulnerable youth and young adults in transition to adulthood. While the overall purpose of NYS PROMISE was to identify effective practices in transition to adulthood for SSI youth, our utilization of the Center of Practice concept for cross-site program adaptation and implementation is unique in ensuring the fidelity of our program to key programmatic elements while allowing the customization necessary for achieving full implementation. The concept of a Center of Practice further facilitated standardizing knowledge about solutions needed for common problems across systems and structuring communications to navigate the mosaic of different organizational mission, cultures, and practices among agencies striving toward a common purpose of serving vulnerable youth and young adults receiving supplemental security income.

While such strengths make a Center of Practice a viable approach to solving complex problems of adolescent transition, it is also a resource intensive approach, requiring significant expertise in strategic cross system planning, and a backbone organization with the capability to generate cross system data for continued reflection by program participants on program achievement. The NYS PROMISE Center of Practice was made possible not only by the PROMISE funding support, but by decades of prior history and experience in the New York State systems in rehabilitation. Without those key ingredients, the proposed model would be in jeopardy of being less effective as a cross system implementation of complex intervention.

The methods employed by the NYS PROMISE leadership and project community present a viable roadmap and design for other large scale research demonstrations seeking to address a common problem or issue across multiple stakeholders at the macro systems level, meso organizational or regional level, and micro level of practice and individual agency. In establishing a Center of Practice, several priorities must be attended to:

The data on impact of implementation of this intervention and its outcome is currently being compiled, with initial evidence suggesting positive impact of intervention. Future research will examine the impact of the overall program on improving employment, post-secondary education, health and economic wellbeing of vulnerable youth and their family members. Finally, the impact assessment research will provide evidence on what works, for whom, and in which circumstances to allow for further replications and scale-up transition to adulthood interventions for minors receiving supplemental security income.

Conflict of interest

None to report.