Usability and consistency in findings of the work support needs assessment tool
Abstract
BACKGROUND:
Structured work support needs assessment could facilitate professionals and increase assessment consistency.
OBJECTIVES:
Evaluating usability of the Work Support Needs Assessment Tool and test if professionals’ (labour experts, coaches) findings become more consistent after a tool training. The tool includes a 21 item checklist for assessing work support needs of people with disabilities.
METHODS:
Usability was explored through 28 interviews with professionals. Consistency was evaluated in an experimental pre-post study design, in which thirty-nine other professionals assessed work support needs of standardized clients before and after a protocolized training. Quantitative content analysis was conducted. Consistency of findings between professionals covered three categories: type (client-focused coaching), focus (topics to be addressed) and duration of support. An increase in consistency was defined as a decrease in the total number of different sub-categories of findings in each category.
RESULTS:
Nineteen professionals indicated that the tool was useful, as they gained relevant information and insights. Regarding consistency, the number of findings differed pre- and post-training for type of support (8 vs 9) and focus of support (18 vs 15 and 18 vs 17).
CONCLUSIONS:
Participants had positive experiences with the tool. Increased consistency in findings of professionals after the training was not demonstrated with the current study design.
1Introduction
Disability affects many people, with the global average prevalence rate being estimated at 16% of the adult population [1, 2]. These prevalence rates are combined with high levels of unemployment among people with disabilities leading to high social security costs for society [3]. Employment can have beneficial effects on health, including feeling more autonomous and having an increased sense of wellbeing [4]. Moreover, especially for people with a disability, having a job is associated with a higher level of functioning and a better quality of life [5, 6]. Therefore, it is important that people with disabilities are given the opportunity to enter and remain in the labour market.
People with disabilities encounter problems in finding and holding onto a job [7]. However, when they receive support, these problem can be alleviated [8, 9]. Frequently, vocational rehabilitation professionals are required to assess the type and intensity of support that a person with a disability will need in order to function at work. The type of support can vary from job search assistance (support in finding suitable jobs, writing resumes or job interviews) to job coaching (support when people start work), job training (teaching job-related tasks) or workplace adaptation [2, 3, 7, 10, 11] In order for a professional to make a decision about the necessary support during work, referred to below as ‘work support needs’, the professional must assess what kind of support will be needed in which situations.
Work support needs are often assessed before the person with a disability has a job, because the required amount and type of support plays an important role in the process of matching the right person with a suitable job and workplace. Therefore it is often not feasible to search for a job when work support needs are not known. This makes the assessment of work support needs complex, as the level of support needed is partly dependent upon the actual workplace and the specific job tasks. Additionally, the assessments are often made for young people with little or no work experience. In such cases the professional has to look for practical examples from a person’s daily life to determine the areas in which support is needed, translating these into work situations. A tool that facilitates these work support needs assessments by providing the professional with more structure and guidance could therefore be useful. Furthermore, an universal structured approach in assessing work support needs might increase transparency and consistency in assessments between professionals. To the best of our knowledge, currently no universal tool exists for assessing work support needs of people with disabilities when the future workplace is unknown.
Therefore, we developed the Work Support Needs Assessment Tool. This tool facilitates vocational rehabilitation professionals (labour experts, job coaches) in the process of assessing work support needs of people with disabilities when the future workplace is still unknown. The 21 items included in the tool cover four categories: pre-conditions for work; skills; personal characteristics; and cognitive and executive functions. The tool guides the professional in deciding which items are deemed relevant for the work support needs assessment and in making an overall decision on the work support needs. It is important to evaluate whether professionals feel the tool can be used to assess work support needs in practice (i.e. does it provide the professional with relevant information) and whether the tool can be incorporated in the work process of a professional. These views of professionals concerning the usability of the Work Support Needs Assessment Tool are still unclear. Furthermore, it is not known if professionals’ assessment of work support needs becomes more consistent after using the tool.
The first phase of the study explored the usability of the Work Support Needs Assessment Tool, while the second phase evaluated the consistency of work support needs assessments. Training programmes have been found to be effective in enhancing knowledge and skills as well as guideline adherence of professionals [12, 13]. It is likely that training in the use of our tool will lead to better knowledge and skills in using the tool. Therefore, participants in our study completed a training programme on how to use the Work Support Needs Assessment Tool. Subsequently, we evaluated the influence of this training programme on the consistency of the professional findings generated by using the work support needs tool. The following research questions were addressed: What are the views of professionals regarding usability of the Work Support Needs Assessment Tool?; Do the findings of professionals become more consistent after completing a training programme on the Work Support Needs Assessment Tool?
2Methods
2.1Design and procedures
The first phase of the study on the usability of the tool had a qualitative design, with Dutch vocational rehabilitation professionals participating in semi-structured interviews. Prior to the interviews, participants were asked to use the tool to assess work support needs with at least two clients from their caseload.
The consistency of findings on work support needs was evaluated in an experimental pre-post study design using two client vignettes played by an actor. Participants assessed the work support needs of one client before the start of the training programme and without using the tool. The participants then completed the training programme before assessing the work support needs of a second client with use of the tool. The order of the client vignettes alternated between the four training groups.
2.2Ethics
The research was conducted in accordance with the Helsinki declaration. The research proposals were submitted to and approved by the Medical Ethical Committee of the Academic Medical Center, which judged that a comprehensive evaluation was not required since these studies were not subject to the Medical Research Involving Human Subjects Act (reference numbers W17_235#17.273 and W17_ 298 # 17.351).
2.3Participants
Various types of vocational rehabilitation professionals (labour experts, consultants, job coaches) participated in the study. In the first phase, professionals were eligible to participate if the assessment of future work support needs of clients with disabilities was part of their job, and if clients in their caseload did not yet have a job. For the second phase of the study, professionals were eligible to participate if they had not been involved in prior studies in which the tool was developed and if they were experienced in assessing future work support needs of clients with disabilities.
Participants were recruited through flyers that were distributed on professional networks, websites, through a newsletter and by approaching professionals and organizations from the researchers’ network. The recruitment procedure was aimed at various types of vocational rehabilitation professionals from different organizations. All participants gave their written informed consent prior to study participation.
2.4The work support needs assessment tool
The tool development started in 2015, as part of another study (study not published). The aim of the study was to develop a checklist with items that were deemed relevant for assessing work support needs of people with disabilities, when the future workplace is still unknown. A concept mapping approach was used to collect, integrate and conceptualize views and appraisals of the consulted experts (i.e. vocational rehabilitation professionals) [14]. The checklist development followed three stages: (1) literature search (conducted by researchers), (2) a focus group and (3) prioritizing and organizing of items. The literature search resulted in a list of 66 items relevant for assessing work support needs. In the second stage 15 vocational rehabilitation professionals participated in a focus group. Prior to the focus group, participants could online submit relevant items, this resulted in a list of 55 items. All items from the literature search and the online submittal were discussed in the focus group and consensus was reached on a list with 50 items. These items were organized in clusters and prioritized by relevancy online, on a 0–5 scale (0 = not very important; 5 = very important). Items with a mean score of 3.5 or higher were included in the final list. This resulted in a checklist of 21 items. In a second focus group with the same professionals, consensus was reached about the definitions of these 21 items (study not published).
The checklist was further developed into the Work Support Needs Assessment Tool in 2017. The core of the tool is the 21-item checklist (see Table 1). The first step of the tool is to systematically assess every item by answering several questions. These recurring questions guide the professional in deciding whether the item is relevant to the assessment. The second step of the tool facilitates the professional in the process of making an overall work support needs judgement by obtaining answers to several concluding questions (see Table 1). Additionally, professionals can consult an appendix, which provides an overview of the items, corresponding definitions and suggestions on how to question clients on a certain item. See supplement material 1 and 2 for the paper-based (translated into English) and print screens of the digital version of the tool (in Dutch).
Table 1
21 items and the questions included in work support needs assessment tool
| Cluster | Item | Definition | |
| Pre-conditions for work | 1 | A place to live | A place to sleep, access to sanitary facilities and sufficient food |
| 2 | Ability to cope independently | Physical, intellectual and mental ability to manage situations independently in personal life | |
| 3 | Social network | The opinions, help and expectations of parents, brothers, sisters and friends that positively or negatively influence the choices that a person makes | |
| Skills | 4 | Perseverance | Having the physically and mentally capacity to persevere in job related tasks |
| 5 | Working independently | The ability to start, execute and finish job related activities without the help and support from others | |
| 6 | Work agreements | Being on time and adhering to work agreements | |
| 7 | Dealing with time limits | Being capable of achieving specific work tasks in a certain amount of time and dealing with time limits | |
| Personal characteristics | 8 | Dealing with other people | Being capable of showing appropriate behavior and attitude towards colleagues and costumers (i.e. implicit rules of conduct and behavior) |
| 9 | Dealing with authority | Being able to work for a boss and accept instructions or assignments from people with authority | |
| 10 | Conflict management | Dealing with conflicts with or between others | |
| 11 | Dealing with resistance | Being able to deal with feedback, criticism or resistance | |
| 12 | Motives and motivation for work | Motives in life and the intrinsic motivation to work | |
| 13 | Sense of reality | Being able to separate fantasy (including delusions and hallucinations) from reality | |
| 14 | Self-knowledge | Knowing your own strengths and weaknesses and boundaries | |
| 15 | Determination | The ability to carry on and not to give up, even when something is very difficult or does not work out as intended | |
| Cognitive and executive functions | 16 | Ability to concentrate | The ability to completely focus on processing of information and to block off distraction |
| 17 | Effectiveness | Being able to perform activities in a coordinated manner in order to reach a certain goal, by structuring and prioritizing tasks and by keeping oversight | |
| 18 | Information processing | Being able to filter an process what we perceive through our senses | |
| 19 | Apprehension | Being able to understand job tasks and job instructions | |
| 20 | Dealing with unexpected or new situations | Dealing with unexpected or new situations in such a manner that it does not affect the quality of work | |
| 21 | Memory | The ability to store, remember and reproduce information | |
| Answer options | |||
| Questions for assessing item | Does the person have capacities concerning this item that will help with functioning in work? | Yes, sufficient | |
| No, insufficient | |||
| Not yet clear at this moment | |||
| Is support and/or adjusting workplace/job tasks regarding this item necessary? | Yes support is necessary | ||
| Yes, adjustment workplace/job tasks is necessary | |||
| Yes support and adjustment to workplace/job tasks is necessary | |||
| No, cannot be solved by support or adjustment workplace/job tasks | |||
| What did you observe (or read in client’s file) that made you think support is necessary? | Open answer | ||
| What kind of adjustment in workplace/job tasks would be necessary? | Open answer | ||
| Concluding questions | What are the best chances for participation in work and what kind of support will enhance these chances? | 1-7 Open answers | |
| On which items is coaching/training/supervision necessary, by whom and with what aim? | |||
| Which areas where support is needed should be addressed first and why? | |||
| Duration of support: How many hours a week or month, during which period and why? | |||
| What is the view of the client on support? | |||
| When is re-evaluation of work support needed? Other remarks? | |||
2.5Phase one: Usability of the work support needs assessment tool
The interviews were audio-recorded and conducted by the first author (MS) and another member of the research team (SV). Interviews were conducted with the use of a topic list developed by the research team. The topic list included open-ended questions regarding demographic information, how the tool was used, usability of the tool (i.e. whether the tool assisted in obtaining the right information and generated sufficient insight regarding the support needs of the client) and whether the tool could be implemented in the assessment routines of the participant.
Box 1. Description of client vignettes
| Vignette 1. Client with autism | Vignette 2. Client with an intellectual disability |
| Gender: Male | Gender: Male |
| Age: 26 years old | Age: 21 years old |
| Education: diploma in intermediate vocational education (ICT specialization) | Education: special needs secondary education |
| Previous work experience: internship in catering (canteen) | |
| Previous work experience: various short ICT-related employment contracts | Living situation: assisted living |
| Main concerns: Low IQ and limited self-reliance in areas of living, school and work. Capable of carrying out simple and defined tasks; however, needs someone who will explain and show him how to execute the task. Difficulties with prioritizing and multi-tasking and working under time pressure. | |
| Living situation: lives on his own | |
| Main concerns: Encounters problems with maintaining a job due to limitations in social interaction, conflict management and taking orders from colleagues with authority. Cannot deal with unexpected changes in work tasks or in the workplace. |
2.6Phase two: Consistency of the findings regarding work support
2.6.1Training programme
The consistency of the findings of professionals on work support needs were evaluated after completing a protocoled training programme. The learning objectives of the training programme were: 1) the participant understands the 21 items and related definitions 2) the participant is able to formulate suitable questions to assess relevant items, 3) the participant can use the tool during work support needs assessment. The three-hour training consisted of the following components: 1) general introduction (stating aims and learning objectives), 2) short specific introduction on the tool development process, 3) interactive exercise in pairs (aimed at interiorizing several items and definitions), 4) general discussion about interactive exercise, 5) demonstration of how to use the tool. The training was supervised by the second author (KN), a professionally qualified and experienced teacher.
2.6.2Client vignettes
The two client vignettes were played by an actor (see Box 1 for short vignettes description). During the session, participants were given the opportunity, as a group, to ask the actor questions, so that they would be able to assess the work support needs. In order to minimize the possibility of differences in the roles played, the actor used a fixed script. After questioning the actor, the participants individually recorded their findings regarding the work support needs of the client.
2.6.3Outcome measure
The outcome measure was operationalized as the findings of the work support needs assessment and was measured with the following question: ‘Describe the necessary support for the client: include in your assessment the type of support, areas on which the support should focus and the level of support needed’.
2.7Analysis
2.7.1Phase one: Usability of the work support needs assessment tool
The interviews were transcribed by MS and SV and were randomly checked for inconsistencies by SA. Transcriptions were categorized according to views about usability and implementation in the work processes and summarized for each category by SV. The categorizing and summarizing of the data were checked and discussed by MS, KN and SA until inconsistencies were resolved and adjustments made accordingly. The final analysis was then discussed by the whole research team.
2.7.2Phase two: Consistency of the findings regarding work support needs
Assessors involved in analysing the work support needs assessments were blinded for group allocation and whether the work support finding was pre- or post-training. To analyse the differences in the consistency of the findings, a quantitative content analysis was conducted in three steps [15]. In Step 1, one assessor (MS) coded all of the findings independently, according to the predetermined codes (type, focus and duration of the support). If necessary, new codes were added. A second assessor (KN) independently coded 10% of the findings. Differences in coding and newly emerging codes were compared and discussed by both assessors until consensus was reached. This resulted in a final coding scheme that was discussed with the other authors (SA, HW, MD). Step 2 involved organizing the codes according to the main and sub-categories by MS and KN (for instance, coaching client or coaching colleagues/manager as sub-categories of the main category ‘type of support’). Lastly, the codes from the anonymous findings on work support needs were connected to the timing of measurement and to client A or B.
Step 3 comprised counting of the different type of findings within the three main categories: type of support (e.g. coaching client or coaching colleagues/manager), the focus of the support (e.g. support aimed at teaching certain skills or adjustments in task instruction) and the duration of the support (e.g. certain number of hours of coaching during a specific period). Within these three categories, the total number of different sub-categories of findings were counted. The total number of different sub-categories of findings in each of the three main categories were then compared across pre- and post-training measurement.
Greater consistency of findings on work support needs was defined as a decrease in the total number of different sub-categories of findings in the post-training measurement, as this would indicate fewer differences in findings with less variation among the participants. An increase in the total number of different findings would indicate the findings were becoming less consistent and more incongruous among the participants.
3Results
3.1Participants
3.1.1Phase one: Usability of the work support needs assessment tool
A total of 28 participants were interviewed between July-October 2017. Of these 28 participants, 13 were male and 14 were working as labour experts. Six participants were job coaches and eight worked as vocational rehabilitation consultants or coaches. The mean interview duration was 45 minutes.
3.1.2Phase 2: Consistency of the findings regarding work support needs
A total of 41 participants completed the training programme in September 2017. Two participants did not provide a pre- or post-training measurement, therefore only 39 participants were included in the analysis. Of these 39 participants, 17 were male and 19 were labour experts, 15 were vocational rehabilitation consultants and five were job coaches.
3.2Phase one: Usability of the Work Support Needs Assessment Tool
A small majority of the participants (n = 19) stated that the tool was useful for obtaining relevant information and gaining insights regarding the work support needs of their clients. It is necessary to gather in-depth information on a variety of subjects and settings (e.g. a client’s everyday life, previous work and education experiences) for a professional to make a judgement about work support needs. The tool was found to assist the professionals in collecting this information by addressing different subjects across the 21 items and by the optional in-depth questions for each item. Participants stated that by following the tool format, client conversations became more structured.
You are forced to touch on everything. That helps a lot and gives structure. You reach a deeper level and get a better insight into the person and where the guidance is needed. (Participant 22).
I find it helpful that the instrument provides insight and structure. That is an improvement on my current way of working. If someone is new to the job I can imagine that an instrument like this is very useful. It gives you something to hold on to. (Participant 9)
Participants mentioned that certain items from the tool were very important to address, something they were made aware of or reminded of by using the tool.
The tool addresses aspects that I wouldn’t normally ask about in an intake interview, but which are certainly important in the guidance. Concerning cognitive and executive functions, for example ability to concentrate and comprehension, those are certainly important things to take into account, or in employment reintegration mediation, to see what positions are suitable. (Participant 14)
Two participants did not consider the tool to be useful because it did not provide them with information they needed. They considered it overly focused on work support, or found that certain items which they normally would consider were lacking in the tool. One participant stated that building trust and developing a bond between the professional and the client was also an important element in their work and that the tool did not facilitate this. Another participant stated that the tool did not sufficiently focus on the living conditions or the nature of the disability of the client.
It’s very standardized. That means there’s no room for grey areas. (Participant 2)
It’s the sum of things, the total picture that there may be a need for guidance. Only the 21 aspects makes for a very analytical approach. (Participant 15)
The majority (n = 26) of the participants stated that the tool could be used to assess the work support needs. The participants also appreciated the optional item-related example questions that were included, as they gave guidance on how the tool items can be converted into questions during clients’ interviews. These example questions also led to more in-depth information about the client.
Normally intake interviews are more general (what kind of work would you like, why are you motivated to find work) and now the questions go much deeper, so you get far more information about the client. For example, dealing with criticism, that’s something that most clients say they have no problem with, but now you really get to grips with things through the questions (example questions, appendix) so you get a far more honest answer. (Participant 14)
A small majority of the participants (n = 20) believed that it was possible to incorporate the tool into their own work process. However, a few preconditions were mentioned; for example, some suggested that the use of the tool should not take too much time, while others said they would only use the tool if colleagues were also using it.
3.3Phase two: Consistency of the findings on work support needs
The results on consistency in the findings on work support needs are presented below according to three main categories: 1) type of support, 2) focus of support and 3) the duration of the support. The results regarding the second category are presented separately for Client A and Client B due to differences in the content of the vignettes.
3.3.1Work support findings regarding type of support
Before the training programme, the participants produced eight different sub-categories of findings regarding the type of work support, while after the programme, the participants provided findings that covered nine different sub-categories. This indicates that consistency in findings about the type of support did not improve after completing the programme. See Table 2 for an overview of the findings on the type of support.
Table 2
Different sub-categories of findings on the type of support
| Type of support | Pre-training N (%)* | Post-training N (%)* | |
| 1 | Solely coaching/support of client | 19 (27%) | 19 (26%) |
| 2 | Job coaching (not further specified) | 16 (23%) | 13 (18%) |
| 3 | Solely coaching of colleagues/employer | 10 (14%) | 17 (23%) |
| 4 | Sheltered work environment | 7 (10%) | 11 (15%) |
| 5 | Support with finding a job | 8 (11%) | 5 (7%) |
| 6 | Support with living/everyday life | 5 (7%) | 1 (1%) |
| 7 | Supervise client | 4 (6%) | 3 (4%) |
| 8 | Specified regulation 1 | 0 (0%) | 3 (4%) |
| 9 | Specified regulation 2 | 0 (0%) | 1 (1%) |
| 10 | Preparing client for work | 1 (1%) | 0 (0%) |
*Percentage of the total number of findings on the type of support (pre-training: N = 70; post-training: N = 73). The categories are not mutually exclusive.
3.3.2Findings regarding the focus of work support for Client A: Autism
In the pre-training measurement for Client A, there were a total of eighteen different sub-categories of findings regarding the focus of the support. In the post-training measurement, this number decreased to fifteen. This indicates a small increase in the consistency of the findings regarding the focus of support for Client A after the programme (see Table 3 for an overview).
Table 3
Client A: Different sub-categories of findings on the focus of support
| Focus of support | Pre-training N (%)* | Post-training N (%)* | |
| 1 | Adjusted task instruction | 11 (11%) | 16 (21%) |
| 2 | Defined/singular/repetitive tasks | 18 (18%) | 15 (20%) |
| 3 | Supervision at the workplace | 14 (14%) | 11 (14%) |
| 4 | Coaching/teaching skills | 11 (11%) | 3 (4%) |
| 5 | Teaching/increasing interpersonal competences | 7 (7%) | 0 (0%) |
| 6 | Teaching/increasing intrapersonal competences | 4 (4%) | 6 (8%) |
| 7 | Individual tasks | 3 (3%) | 0 (0%) |
| 8 | Facilitating client | 7 (7%) | 3 (4%) |
| 9 | Other advice about workplace | 0 (0%) | 2 (3%) |
| 10 | Fixed point of contact in the workplace | 6 (6%) | 3 (4%) |
| 11 | Further examination of client necessary | 3 (3%) | 1 (1%) |
| 12 | Informing and supporting colleagues and employers | 2 (2%) | 3 (4%) |
| 13 | Structured/quiet workplace | 3 (3%) | 4 (5%) |
| 14 | Accessibility between home and workplace | 3 (3%) | 3 (4%) |
| 15 | Support client in area of personal life/living/network | 4 (4%) | 2 (3%) |
| 16 | Characteristics of supervisor | 3 (3%) | 3 (4%) |
| 17 | Not having deadlines in work tasks | 1 (1%) | 0 (0%) |
| 18 | Own workplace | 1 (1%) | 0 (0%) |
| 19 | Other advice | 1 (1%) | 1 (1%) |
*Percentage of the total amount of findings on the focus of support (pre-training: N = 102; post-training: N = 76). The categories are not mutually exclusive.
3.3.3Findings regarding the focus of work support for Client B: Intellectual disability
Before the training programme, eighteen different sub-categories of findings concerning the focus of support were offered for Client B. This number decreased to seventeen after the programme. Since the difference is very small, an improvement in consistency in findings regarding focus of support for Client B cannot be assumed. See Table 4 for an overview of the findings for the focus of support for Client B.
Table 4
Client B: Different sub-categories of findings on the focus of support
| Focus of support | Pre-training N (%)* | Post-training N (%)* | |
| 1 | Teaching/increasing interpersonal competences | 12 (12%) | 17 (17%) |
| 2 | Informing/supporting colleagues and employer | 15 (15%) | 12 (13%) |
| 3 | Defined/singular/repetitive tasks | 12 (12%) | 11 (11%) |
| 4 | Teaching/increasing intrapersonal competences | 11 (11%) | 11 (11%) |
| 5 | Support with finding a job | 10 (10%) | 4 (4%) |
| 6 | Coaching/teaching skills | 1 (1%) | 6 (6%) |
| 7 | Fixed point of contact in the workplace | 7 (7%) | 3 (3%) |
| 8 | Individual tasks | 1 (1%) | 4 (4%) |
| 9 | Structured/quiet workplace | 8 (8%) | 6 (6%) |
| 10 | Support client in area of personal life/living/network | 0 (0%) | 2 (2%) |
| 11 | Own workplace | 4 (4%) | 5 (5%) |
| 12 | Facilitating client | 4 (4%) | 3 (3%) |
| 13 | Supervision at the workplace | 1 (1%) | 0 (0%) |
| 14 | Other advice about workplace | 1 (1%) | 0 (0%) |
| 15 | Adjusted task instruction | 5 (5%) | 5 (5%) |
| 16 | Further examination of client necessary | 2 (2%) | 2 (2%) |
| 17 | Other advice | 2 (2%) | 2 (2%) |
| 18 | Characteristics of supervisor | 2 (2%) | 2 (2%) |
| 19 | Not having deadlines in work tasks | 1 (1%) | 1 (1%) |
*Percentage of the total number of findings on the focus of support (pre-training: N = 99; post-training: N = 96). The categories are not mutually exclusive.
3.3.4Findings regarding duration of support
The participants described the duration of support in various ways. Due to the many different kinds of responses given on the duration of support, it was not possible to analyse and codify the majority of the findings and thus it was not possible to draw conclusions regarding the consistency of these findings.
4Discussion
This study aimed to explore professionals’ views about the usability of the Work Support Needs Assessment Tool. Most of the participants considered the tool to be useful for assessing the work support needs of clients with disabilities. By using the tool, they obtained useful insights and in-depth information to assist in their assessment of work support needs. The tool also gave them structure and guidance during consultations with clients. A few participants did not find the tool useful, due to its standardized and analytical format or because of missing items. The majority of the participants believed that the tool could be incorporated into their assessment routines.
An overall significant increase in consistency in the professionals’ work support needs judgements could not be demonstrated in this study. The results indicate that findings about the type of support did not become more consistent after the training programme, as the total number of sub-categories of findings on types of support increased. In relation to the findings on the focus of support, a very small increase in consistency was found.
Another similar work-related support needs assessment instrument is the Work-ability Support Scale. This tool has demonstrated acceptable levels of accuracy and reliability and is considered adequate for use in clinical practice [16]. The Work-ability Support Scale can be used in the assessment of vocational rehabilitation needs when a client already has a job or is expected to start in a job. This tool addresses specific items that are also included in our tool, for instance dealing with managers, clients and colleagues, motivation to work, personal support, planning and organizing work tasks and having realistic expectations regarding work or abilities. Differences between both tools relate to the timing of using the tool in the vocational rehabilitation process. Our Work Support Needs Assessment Tool is developed to be used at the start of the guiding process, when the future workplace of the client is still unknown. It therefore produces an initial and generic work support needs judgement that can be used to guide the job searching and matching process. When there is a potential workplace in sight, the initial support needs judgment should be specified and adjusted to the actual workplace. This revised support needs judgement can be conducted by using a more specific tool, such as the Work-ability Support Scale, in which the job tasks and workplace are taken into consideration [16]. Although the majority of consulted professionals in our study had positive views about the Work Support Needs Assessment Tool, caution should be exercised when using the tool as more research is required to increase the psychometric properties of the tool.
4.1Limitations
The Work Support Needs Assessment Tool seemed acceptable for assessing work support needs according to the majority of the professionals in our study. However, our study did not demonstrate an improvement in the consistency of findings regarding work support needs after a training programme. Explanations for this finding might be found in the study design and the way data were analysed. First, the training programme was focused on participants internalizing the items and related definitions included in the tool, along with a method of assessing these items during a client interview. For practical reasons, it was not feasible for participants to individually interview the two clients. This meant it was not possible to allow every participant to complete an individual work support needs assessment overview, as for which the tool was designed. A training programme that focused on judging the work support needs through such an individually generated overview of work support needs assessment might have had other effects on the consistency of the findings regarding work support needs. Second, consistency in work support findings was assessed through quantitative content analysis [15]. Although this type of analysis has a quantitative element (i.e. counting the times a certain word or phrase is used), in our study the sub-categories for type of support and focus of support were based on a qualitative analysis preceding the quantitative analysis. Another method of measuring differences in assessments might produce different results, for instance letting the participants tick fixed multiple choice answer categories and assessing inter-rater reliability through statistical analysis. Another limitation is that due to the small amount of findings in each sub-category of the assessments, it was not possible to conduct stratified analyses based on the type of professional. It is possible that a job coach could have assessed work support needs different from a labour expert.
One strength of this study is the large number of participants included in the entire study, and the variation in type of professions such as labour experts, job coaches or consultants. The participants appeared to be a representative sample of the different type of professionals who are involved in assessing work support needs. Furthermore, using an actor to play the role of the client in two vignettes was also a strength, as this strategy has been associated with the initiation of active learning processes, allowing trainees to experience and reflect through assignments with the actor. Active learning has been found to be an effective method for obtaining new skills or changing behaviour [17]. In addition, vocational rehabilitation professionals use all kinds of sources when collecting information about the functioning of a client. An interview with the client is the most important source of information, as it gives the professional the opportunity to ask questions and to observe, and sometimes test the clients’ functioning, responses and behaviour. A written case description of a client or a video of them would not make it possible for participants to ask questions and would therefore not be a realistic simulation of the use of the tool in practice.
Future research should focus on exploring other suitable and quantitative methods of assessing consistency of findings that follow from using the Work Support Needs Assessment Tool. An increase in consistency of work support needs findings would contribute to the reliability of work support needs assessments. Moreover, further development of the tool should also be considered; for example, the tool might be embedded in a mobile app. An app could possibly offer users more flexibility in relation to their needs and preferences and might also reduce the time it takes to fill out the questions included in the tool. This could further increase the usability of the tool.
5Conclusion
The Work Support Needs Assessment Tool offers valuable assistance in the structural assessment of work support needs of clients with disabilities at the start of the guiding process, when the future workplace is still unknown. The tool provides professionals with structure and guidance during client consultations. However, in the current study design we were not able to demonstrate an increase in the consistency of professionals’ findings regarding work support needs after a training programme in the use of the tool.
Conflict of interest
The authors declare that they have no conflict of interest.
Data availability
The datasets generated during and/or analysed during the current study are not publicly available as these include Dutch text data, but are available from the corresponding author on reasonable request.
Funding
This study has been funded by a SUWI grant from ‘The Dutch Social Security Institute: the Institute for Employee Benefits Schemes (UWV)’ Grant number: K1617a.
Acknowledgments
We would like to thank Stefan van der Veen for his contribution to the study.
References
[1] | World Health Organization. WHO global disability action plan 2014-2021: Better health for all people with disability: World Health Organization; 2015. |
[2] | World Health Organization. World report on disability 2011: World Health Organization; 2011. |
[3] | OECD Sickness, disability andwork: Breaking the barriers: Organisation for Economic Co-operation and Development; 2010. |
[4] | Modini M , Joyce S , Mykletun A , Christensen H , Bryant RA , Mitchell PB , et al. The mental health benefits of employment: Results of a systematic meta-review. Australasian Psychiatry. (2016) ;24: (4):331–6. |
[5] | Burns T , Catty J , White S , Becker T , Koletsi M , Fioritti A , et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophrenia Bulletin. (2008) ;35: (5):949–58. |
[6] | Eggleton I , Robertson S , Ryan J , Kober R . The impact of employment on the quality of life of people with an intellectual disability. Journal of Vocational Rehabilitation. (1999) ;13: (2):95–107. |
[7] | Greve B . The labour market situation of disabled people in European countries and implementation of employment policies: a summary of evidence from country reports and research studies. Academic Network of European Disability Experts (ANED). 2009. |
[8] | Kaya C , Hanley-Maxwell C , Chan F , Tansey T . Differential vocational rehabilitation service patterns and outcomes for transition-age youth with autism. Journal of Applied Research in Intellectual Disabilities: JARID. (2018) ;31: (5):862–72. |
[9] | Peterson D , Gordon S , Neale J . It can work: Open employment for people with experience of mental illness. Work. (2017) ;56: (3):443–54. |
[10] | Frank A . Vocational rehabilitation: the way forward. London: British Society of Rehabilitation Medicine; 2000. |
[11] | Sundar V . Operationalizing workplace accommodations for individuals with disabilities: A scoping review. Work. (2017) ;56: (1):135–55. |
[12] | Joosen MC , van Beurden KM , Terluin B , van Weeghel J , Brouwers EP , van der Klink JJ . Improving occupational physicians’ adherence to a practice guideline: feasibility and impact of a tailored implementation strategy. BMC Medical Education. (2015) ;15: , 82. |
[13] | Zwerver F , Schellart AJ , Anema JR , van der Beek AJ . Changes in insurance physicians’ attitudes, self-efficacy, intention, and knowledge and skills regarding the guidelines for depression, following an implementation strategy. Journal of Occupational Rehabilitation. (2013) ;23: (1):148–56. |
[14] | Trochim W , Kane M . Concept mapping: an introduction to structured conceptualization in health care. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care. (2005) ;17: (3):187–91. |
[15] | Hsieh HF , Shannon SE . Three approaches to qualitative content analysis. Qualitative Health Research. (2005) ;15: (9):1277–88. |
[16] | Fadyl JK , McPherson KM , Schluter PJ , Turner-Stokes L . Development of a new tool to evaluate work support needs and guide vocational rehabilitation: the work-ability support scale (WSS). Disability and Rehabilitation. (2015) ;37: (3):247–58. |
[17] | Beidas RS , Cross W , Dorsey S . Show Me, Don’t Tell Me: Behavioral Rehearsal as a Training and Analogue Fidelity Tool. Cognitive and Behavioral Practice. (2014) ;21: (1):1–11. |
