Interagency collaboration in transition to adulthood: A mixed methods approach to identifying promising practices and processes in the NYS PROMISE project

1Introduction

Interagency collaboration is often cited as a best practice in transition to adulthood for youth with disabilities (Oertle & Trach, 2007; Noonan, Morningstar, Erickson, 2008; Wehman, Schall, & Carr, 2014; Kohler, Gothberg, Fowler, & Coyle, 2016). Emphasis usually falls on relations between secondary education, and adult rehabilitation and independent living systems, where systems coordination can mitigate service gaps (Oertle, Trach, & Plotner, 2013). Practices that systematically involve parents and families in the transition process are commonly supported (Henninger & Taylor, 2014; Biswas, Tickle, Golijani-Moghaddam, & Almack, 2016). Service coordination in transition planning is also a legal requirement under the Rehabilitation Act of 1973 (29 U.S.C. §723(b)(7)) and the 2004 Individuals with Disabilities Education Act (IDEA) reauthorization (20 U.S.C. §1401(34)). Although service system collaboration is one of the most highly-recommended practices in transition, it is also one of the least empirically understood (Fabian, Dong, Simonsen, Luecking, & Deschamps, 2016; National Council on Disability, 2008). Recent literature emphasizes the need for further empirical work to identify effective practices, clarify collaboration as a construct, and supplement research on antecedents and outcomes with focus on systems-level processes that increase collaboration (Morningstar & Mazzotti, 2014; Fabian et al., 2016). To contribute to these need areas, we used a mixed methods, exploratory approach with evaluation data collected from agencies, public schools, and non-profits engaged in New York State Promoting the Readiness of Minors in Supplemental Security Income (NYS PROMISE), a five-year project providing case management and transition services to youth with disabilities receiving Supplemental Security Income.

Promoting the Readiness of Minors in Supplemental Security Income (PROMISE) is a joint federal research demonstration of the U.S. Departments of Education (USDOE), Health and Human Services, and Labor, with evaluation support for the demonstration from the Social Security Administration. PROMISE funded six model demonstration projects to address barriers and obstacles to economic independence and promote successful education and employment post-school outcomes for youth who receive SSI, providing approximately $230 million to demonstration projects in Arkansas; California; Maryland; New York; Wisconsin; and a six-state consortium which included Arizona, Colorado, Montana, North Dakota, South Dakota and Utah.

One objective of NYS PROMISE was to strengthen existing systems to better support youth during the transition to adulthood, through formal cross-agency service delivery and coordination practices, supplemented by professional development, organizational capacity, and cross-training opportunities at state and regional levels. The formal coordination framework of NYS PROMISE increased the required minimum level of interagency coordination and communication above pre-existing levels. To increase formal, task-oriented interagency service coordination, the project utilized an existing centralized data collection and information sharing system (New York State Employment Services System, or NYESS), embedded within regional networks of local employment service providers. During the project, use of the NYESS system for recordkeeping, service referral, and payment was expanded to include regional networks of participating public schools, community service providers, and parent resource centers. Using this centralized data system, school, provider, and parent center staff had contractual responsibilities for service planning, delivery, and reporting. School-based case managers and parent center family coaches met with intervention group youth and families quarterly to determine appropriate referrals geared towards financial, employment, educational, and/or independent living outcomes, after which providers carried out and recorded services, initiating review and approval by the case manager or family coach. Contracts with schools, service agencies, and parent centers varied in scope, service requirements, and payment-for-service method, as field conditions warranted. Individual contracts were modified, cancelled, or initiated with new service providers as the intervention evolved. Rather than viewing the project as a discrete, externally-derived intervention, in this paper we focus on systems-level change within pre-existing regional service networks as a result of required frequency and formality of contact. In identifying themes, we consider both practices and processes that were included in the project intervention, as well as those developed contextually by project staff.

2Background

In the field of transition to adulthood for youth with disabilities, interagency collaboration entails both formal and informal relationships between youth and adult systems, in which communications, information, and resource sharing help achieve joint transition goals and service coordination (Noonan, Morningstar, & Erickson, 2008; Test, Fowler, White, Richter, & Walker, 2009). Different theoretical and operational definitions focus alternatively on synergistic elements (e.g., leadership, trust, mutuality; Weis, Anderson, & Lasker, 2002; Thomson, Perry, & Miller, 2007) and more instrumental elements such as frequency of communication, resource sharing, formality, shared decision making, and role clarity (Frey, Lohmeier, Lee & Tollefson, 2006; Noonan, McCall, Zheng, & Erickson, 2012; Povenmire-Kirk et al., 2015). The assumptions and constructs that underlie collaboration as a best practice still need refinement. For example, Fabian et al. (2016) utilized two different measures of collaboration among interagency teams, finding that instrumental or task-oriented perceptions of collaboration had a positive effect on vocational rehabilitation outcomes, while perceptions of team “synergy” had slight, negative effects. Lack of agreement as to how to define systems collaboration is a potential weakness in a number of human services fields, and raises concerns that poorly calibrated approaches to collaboration can have undesired effects, like diffusion of responsibility (Longoria, 2005). In collaboration research more broadly, a recent systematic review revealed a tendency to focus more on antecedents and outcomes of collaboration, and less on the processes exhibited in strong collaborations (Gazley & Guo, 2017). Even task-oriented measures of collaboration often lack granular detail about effective interventions in practice, such as what minimal level of interagency communication is “frequent” or how best to structure formal collaborative agreements (such as memoranda of understanding/agreement) and resource sharing practices. This is a barrier to making specific, evidence-based recommendations.

Two psychometrically-validated instruments, the Interagency Collaboration Activities Scale (IACAS) (Greenbaum & Dedrick, n.d.; Dedrick & Greenbaum, 2011) and the Levels of Collaboration Survey (LCS) (Frey, et al., 2006) are commonly used in transition research (Trach, 2012). Both conceive of collaboration as progressing through a continuum of stages or degrees, and utilize different measures for task-oriented collaborations. In comparison to the LCS, the IACAS uses a Likert-type measure of the extent to which an agency shares in financial and physical resources, program development and evaluation, client services activities, and collaboration policies (e.g., informal/formal agreements; Greenbaum & Dedrick, n.d.).

Building on the pre-existing literature on interagency collaboration in transition, this mixed method analysis aims to answer two related research questions: (1) Is there evidence that the NYS PROMISE model improved collaborations between schools and providers? (2) What attributes of successful collaboration practices and processes were described by program staff? Thus, the first question was evaluative, and the second was exploratory. We answered these questions with three studies. To answer the first question, we used Frey et al.’s (2006) LCS, which provided data on regional network changes over time (Study 1) and an organizational attitudes and experiences (OAE) survey which measured satisfaction of partner communications (Study 2). To answer the second research question, we conducted a qualitative analysis of biannual on-site interviews at project research demonstration sites (Study 3), using a priori codes developed based on the constructs in Frey et al.’s (2006) LCS. This allowed for a deeper dive into the LCS scale findings, and an opportunity to narrow recommendations along the LCS’s continuum. This data allowed a useful opportunity to evaluate systems change during a statewide grant project focused on cross-systems collaboration, while also exploring stakeholder feedback about attributes of successful partnerships, as a means to investigating, revising, and narrowing recommended best practices in collaboration.

3Methods and results

The mixed methods approach combines three studies. Sources of data included the LCS (Study 1), a survey instrument capturing project staffs’ self-reported organizational attitudes and experiences (OAE, Study 2), and feedback from project case managers as to the quality and nature of their collaborations with service providers and parent resource centers (Study 3). Table 1, below, summarizes the instruments, frequency of administration, and type of data retrieved. In this section, we provide: (i) a description of the study participants, (ii) an overview of the project implementation, which differed by region, and (iii) a description of methods and results for each study. Quantitative analyses in Studies 1 and 2 are presented first with a focus on describing change in collaborations and communications among project staff, followed by a qualitative analysis (Study 3) to identify practices and processes that staff found effective at improving collaboration and service coordination efforts.

3.5Study 3: Qualitative site visit analysis

4Discussion

This paper has several key findings. First, longitudinal analyses of levels of collaboration (LCS, Study 1) and partner communications (OAE, Study 2) provide evidence that task-oriented interagency agreements can meaningfully change collaborations between schools and providers. Second, qualitative analysis (Study 3) allowed a deeper dive into the attributes of successful partnerships, as described by project staff, including practices and processes that were part of the project intervention, as well some that were developed contextually by staff. Third, the differences observed in the “indigenous” (CAP and WNY) and “hybrid” (NYC) models of service coordination implicate program design considerations, and the importance of tailoring models for interagency collaboration to localized bureaucratic, demographic, and geographic realities. Finally, the predominance of practical and organizational barriers to collaboration in the qualitative findings, and accompanying strain on partnerships, reinforces the importance of setting realistic goals and expectations for partners at the macro (state), meso (regional) and micro (agency) levels (see also Golden, Karpur, & Podolec, pre-press). By mapping qualitative analysis to the psychometrically-validated constructs utilized in Frey et al.’s (2006) LCS, these findings may help to refine understandings of task-oriented constructs within interagency collaboration, including contextual understandings of what formality of service coordination, resource sharing, and frequency of communication might look like in practice. Importantly, the findings also highlight areas where these constructs need further clarification and explication.

Studies 1 and 2 provided a quantitative perspective as to whether there were meaningful changes in the PROMISE transition network. Study 1 indicated that, in the two upstate regions, overall collaboration ratings increased each year until the project’s penultimate year, with a small decline in the final year. At their highest, mean ratings were between coordination and cooperation (between 2 and 3 on the LCS scale). In New York City, with its continually evolving network model, mean ratings were between networking and coordination (1 and 2 on the LCS). Thus, in the upstate, “indigenous” networks, there were tendencies towards reporting more integrated communication, role clarity, decision making, and resource/information sharing. Study 2 helps contextualize how communications did, and did not, change during the project. According to OAE data, the frequency of communication did not change during the project, but satisfaction with provider communications did increase over time, again with regional differences observed. In the existing transition literature, there is minimal discussion of the meaning of frequent or effective communication within task-oriented continuums of collaboration. Future studies should explore what minimum level of weekly/monthly contacts lead to meaningful changes in service coordination. For instance, OAE analysis found that most partners communicated monthly or more, adhering to project requirements for increased, formal contact, but not tending to increase the frequency of contact as the project progressed.

Study 3 provided a qualitative perspective as to where and why meaningful changes were observed in the PROMISE transition network, as well as where they fell short. The characteristics of partnerships described as effective included: (a) joint objectives and clearly defined roles; (b) extensive sharing of information and resources; and (c) frequent communication (along an interesting formal-to-informal progression). In the upstate regions, early-stage, formally-planned meetings were gradually supplanted by more informal connections, including deepened interpersonal ties, as distinct from more formal, field-oriented agency-level connections, developed contextually to meet project needs. This maps well to Frey et al.’s (2006) conceptual framework, and the continuum from required/formal contact through more frequent contact characterized by mutual trust. These informal connections were not mentioned as frequently in NYC, where collaborations were, on average, described as weaker along Frey et al.’s (2006) continuum. Confirming regional differences in the OAE findings, the qualitative data evidenced infrequent or poor communication as a bigger problem in NYC, whereas communications in the other regions were more frequently described in neutral or positive terms. Interview responses routinely indicated that contextually-developed practices of resource and information sharing, according to project staff, entailed relatively low cost and effort once a level of familiarity and lines of communication were established. For instance, multiple respondents that went beyond formal requirements for communication cited successful practices as simple as sharing student class schedules with providers, and offering space for in-school provider “office hours.” The centralized case management data system also provided a direct, and contractually required, means of formally sharing information during the course of the project, with staff utilizing the system differently, from meeting minimal entry/reporting/billing requirements to using the system as a tool for shared decision-making and information sharing practices above project requirements.

However, another major theme from the qualitative findings was the impact on organizational capacity issues on effective collaborations. Staff turnover, organizational capacity, and failure to execute roles are predictable, yet difficult, challenges to reckon with. Qualitative analysis indicated that barriers to collaboration often resulted from practical, rather than synergistic, concerns. These barriers to successful network development may be managed at the project design or implementation phases through careful consideration and structuring of processes, policies, and contracts in order to support effective collaboration. Additionally, pre-assessment of potential network partners may help to mitigate strains of capacity, resource availability, and staffing by allowing projects to initiate collaborations with the strongest available partners. Issues of capacity and staff turnover were more acute in NYC, where familiarity and comfort with a partner agency was more likely to rely on a single point of contact. Larger, more diffuse networks likely require different models of collaboration than smaller networks with fewer partners and more cohesion. Future research should aim to clarify the contextual factors that impact the efficacy of recommended practices in interagency collaboration, and account for the processual and practical challenges that more complex networks (e.g., urban, high population, more complicated bureaucracy) may encounter. Finally, our findings implicate a continuing need for empirical research on the sustainability of collaborations beyond grant periods, and what types of strong partnerships tend to survive external funding or contractual end dates. Some findings from this analysis indicate losses in self-reported collaboration ratings as the project approaches closeout. It is evident that further work is needed to develop specific strategies that are sustainable, adapted to different network structures and types, and cognizant of challenges posed by staff turnover and capacity issues.

5Conclusion

Feedback from project staff provides clarification of the experiences of agency staff coordinating services, and deepens contextual understandings of systems collaboration in two different scenarios of transition-focused grant projects (hybrid and indigenous intervention frameworks). The findings present some conservative evidence that task-oriented measures of collaboration can be manipulated and improved through interventions, but there is still a deep need to further develop the constructs that are frequently used and measured in transition research on interagency collaboration. Qualitative analysis indicated that many of the elements of successful interagency collaboration mentioned in the literature and existing task-oriented collaboration measurement tools are also on the minds of practitioners in the field of transition. Certain opportunities and strategies emerge out of this analysis, including new areas of potential research about: (a) the dosage of formal and informal communication needed to increase interagency coordination; (b) strategies and processes for information and resource sharing across contextually different transition networks; (c) frameworks for assessing partnership capacity and initiating formal agreements with clear roles and responsibilities; and (d) the integration of capacity, staffing, and resource issues into models of collaboration in transition.

Conflict of interest

None to report