Introduction National Context The National Institute for Health and Care Excellence (NICE) states that those experiencing first episode psychosis must have ‘a maximum wait of two weeks from referral to start of treatment’ and receive ‘treatment in accordance with NICE guidelines and quality standards for psychosis and schizophrenia’ (NICE, 2016). The NICE guidelines require a range of intervention and psychological therapy to be offered to the client, including Cognitive Behavioural Therapy and Family Interventions. Service Context Within the Mid-Essex Early Intervention in Psychosis (EIP) Team, the Early Psychosis Intervention Clinic (EPIC) aims to provide individuals with quick access to psychological assessment, formulation, and intervention. Typically, Care Coordinators inform the clients in person about the service. Clients can book up to four EPIC sessions, one at a time, when they feel ready to access the service. After completing the sessions, they can be added to a list for any further psychological intervention should they want this. NICE guidelines state that clients should be offered at least 16 planned sessions of CBT (NICE, 2014). In the first session, clients are asked to complete a Clinical Outcome in Routine Evaluation – Outcome Measures (COREOM; Evans et al., 2009) questionnaire and once again upon finishing treatment with EPIC to assess their progress. The measure provides an overall clinical score and subsection scores (wellbeing, risk, functioning and symptoms). Prior Evaluation The previous service evaluation aimed to evaluate the effectiveness of EPIC within the Mid-Essex EIP Team from October 2017 to October 2020 by using 17 client’s CORE-OM questionnaire results. It was found that 35.29% of clients significantly improved on all four subsections and 52.94% of clients moved from the clinical to the non-clinical population in one or more subsections. The previous evaluation only looked at clients who had fully completed EPIC, therefore this data does not represent those who partially completed EPIC or were offered but did not complete any of EPIC. Aims To re-evaluate the effectiveness of the EPIC service by comparing clients’ pre and post CORE-OM questionnaire scores. To gain qualitative information about the effectiveness of the service by asking clients about their personal experiences (clients who completed EPIC, clients who partially completed EPIC, and clients who did not start EPIC). To provide recommendations for service improvement. Method The current evaluation incorporates both quantitative and qualitative data. The quantitative side of the evaluation follows a repeated measures design using continuous data. Prior to completing EPIC, clients are asked to complete a CORE-OM questionnaire. They complete a second CORE-OM questionnaire upon finishing EPIC. Clients’ pre and post CORE-OM questionnaires were scored and added to an excel spreadsheet on a shared drive. The data is kept within a password-protected computer to ensure confidentiality. Percentages demonstrating how many clients reliably improved or deteriorated in clinical score and individual subsections and how many clients moved from the clinical to the non-clinical population or from the non-clinical to clinical population were calculated. The CORE-OM is a self-report questionnaire consisting of 34 questions, leading to a maximum score of 136, regarding experiences during the last week. It is rated on a 5 point scale from 0 (not at all) to 4 (most or all of the time). The CORE-OM consists of 4 subscales (wellbeing, functioning, risk and symptoms) and an overall clinical score (the mean of all scores multiplied by 10). A clinically significant improvement is a decrease of 5 or more in clinical score and 0.5 or more in subsections. The clinical population scores 10 or more in clinical score and 1 or more in subsections. The quantitative data sample consists of 19 clients who completed EPIC between October 2020 and October 2021. In total, 30 clients completed the pre-measure, but 11 did not complete the post-measure, either due to not finishing the four sessions (10 clients) or not completing the post-measure (1 client), leading them to be discarded from the final data set. The qualitative side of the evaluation used thematic analysis to analyse clients’ responses on a feedback survey. A sample of clients was contacted by telephone and asked if they were willing to provide feedback on their experience of the EPIC service. Informed consent was sought and gained from all clients included in the sample. Clients were asked open-ended questions based on the following themes: overall experience, motivation to engage (or not), expectations, access and potential barriers, suggestions for improvements. Themes were identified amongst the answers, and answers between the categories were compared to find differences between those who did and did not complete EPIC. When a frequent answer to a question was found, percentages were derived from seeing how many clients answered the same. The sample of the qualitative evaluation consists of 13 clients. Five completed EPIC, three did not fully complete EPIC, and five did not start EPIC. Results CORE-OM results (quantitative) 31.6% of clients significantly improved in overall Clinical Score (n = 6) 52.6% of clients significantly improved in Wellbeing (n = 10) 52.6% of clients significantly improved in Symptoms (n = 10) 31.6% of clients significantly improved in Functioning - when controlling for clients who scored 0 at baseline and therefore could not improve (n = 2), this increased to 42.1% (n = 7) 5.3% of clients significantly improved in Risk - when controlling for clients who scored 0 at baseline (n = 13), this increased to 73.7% (n = 14) 5.3% of clients significantly improved in all four subsections – when controlling for clients who scored 0 at baseline (n = 4), this increased to 26.3% (n = 14). 31.6% of clients moved from a clinical to a non-clinical population in overall Clinical Score (n = 6) 47.4% of clients moved from a clinical to non-clinical population in one or more subsections (n = 9). 10.5% of clients significantly declined in overall Clinical Score (n = 2) 15.8% of clients significantly declined in Wellbeing (n = 3) 15.8% of clients significantly declined in Symptoms (n = 3) 10.5% of clients significantly declined in Functioning (n = 2) 15.8% of clients significantly declined in Risk (n = 3) 5.3% of clients significantly declined in all four subsections (n = 1). 10.5% of clients moved from a non-clinical to a clinical population in overall Clinical Score (n = 2) 15.8% of clients moved from a non-clinical to a clinical population in one or more subsections (n = 3). Feedback Survey (qualitative) Clients who completed EPIC (5 clients) 100% of clients said their overall experience was good and useful. When asked what they did/did not like, the majority (60%) said they liked the psychologist conducting their session and their abilities. Common themes were that the sessions were helpful, that it was good to speak to someone, and that the sessions helped them understand. ‘I did like the psychologist. She was very kind and patient with me, and I found it helpful to talk to her.’ ‘It was helpful. Helped me understand what I was going through.’ However, one client felt EPIC wasn’t advertised and promoted to them well, leading to little understanding about it and a lack of hope. ‘I didn’t have a good understanding of what was going on; I didn’t like that I wasn’t really ‘sold’ on this and did not have the confidence it would work’ Client’s motivations for completing EPIC were mixed. 40% of clients said that they took part in EPIC as it was recommended to them, 40% wanted to learn techniques and gain more understanding of their condition and one client felt they needed it. ‘They were offered to me, wanted to take everything that was offered in order to get better.’ ‘Getting more knowledge and understanding how voices work.’ ‘I had a need to have a psychology session.’ Out of those who started EPIC with expectations (60%), all expected EPIC to be about learning and understanding their condition. Out of the four clients that answered whether their expectations were met, one said yes, two reinforced that they had no expectations, and the last said their expectations were partly met as some of their expectations were incorrect. ‘I thought I will go through my memories and understand them from a different point of view.’ ‘Learning more about psychosis. Learning ways to manage it.’ ‘They were not met in that I thought the sessions would be more medical in nature… my overall expectations of trying to understand the mind were met.’ ‘I did not expect more than I was getting. But the experience was positive.’ 40% of clients said that they were concerned to meet a psychologist, one saying they were worried they wouldn’t learn anything. The other 60% had no concerns. ‘Worried that I won’t learn anything new or helpful.’ ‘Felt nervous about the experience.’ 60% of clients said they generally had no practical barriers. However, 40% of clients said travel was an issue. One client said the sessions were too early for them, and another said COVID was a barrier as they could not meet face to face. 75% of the clients that used the booking system said it was fine and easy to use, but one client suggested that a text confirmation/reminder would be helpful. Out of the two clients that provided suggestions on anything that would be helpful for clients to know before starting EPIC, one suggested that a leaflet with information and testimonies from service users would be good. Another client suggested it would be good to understand beforehand that the sessions are entirely in the client’s control and that the psychologist works alongside them. ‘A leaflet that is given out by Care Cos that has a few short sentences on what is offered and include some testimonials of feedback and helpfulness of the sessions.’ ‘It is not going to be anything out of control for the patient, and within patient’s control… the psychologist worked alongside me rather than worked on me.’ 40% of clients felt there wasn’t anything the service could have done differently, however three of the clients offered suggestions. One client said they wished for a shorter gap between sessions, one wished to meet the psychologist beforehand and the other wishes they had a fully qualified psychologist. ‘Maybe not have a gap of a month… at first there was a real impact, the acute period kind of passed, maybe I would have had more to talk about during the early stages.’ ‘I go on walks with my Care Co… if the psychologist was introduced to me by coming along to the walk, this would’ve been sold to me more easily and I would’ve had better expectations.’ ‘Had some with a trainee – maybe having EPIC with a fully qualified psychologist straight off the bat might be more effective.’ Clients who partially completed EPIC (3 clients) When clients were asked about their experience with EPIC, one client said it was good; another said they found it confusing, and a third said that nothing was wrong with it; it just was not suitable for them. The answers provided on what the client liked/did not like were mixed. One client said it was not a pleasant experience as they did not want to be there, one said there was nothing in particular, but it did not have much of an impact on them, and the third said that they liked the psychologist. ‘It was good, led by the person, was open, I could take it where I wanted.’ ‘At the time I found it confusing. Too many questions were asked about the past.’ ‘It wasn’t a nice experience. Can’t put a finger on it, I just did not want to be there.’ ‘Nothing in particular, just personal preference. Did not have much of an impact for me.’ 100% of the clients said they took part in EPIC because it was recommended, and one client said they did not do it for themselves. Of the two people who gave their expectations, 100% expected it to be talking about issues and concerns, with one client saying they expected it to be similar to their school counselling. One client said their expectations were met, but the other was unsure. ‘Dreading it a bit, I don’t like talking about things. Expected it to be similar to counselling I had at school which I didn’t like.’ ‘Was offered to me to talk about any concerns, so I thought I’d take up that opportunity’ The majority of clients (66.7%) said they were nervous to meet a psychologist, one because they were not sure what to expect and the other because they do not like opening up. ‘Little bit of nerves at the beginning. Didn’t know what to expect.’ ‘Just nerves, I’m a closed book.’ 100% of clients said they had no practical barriers, and 100% said they had no issues with the booking system. Each client had different reasons for not completing EPIC. One client said they did not like talking; another said they stopped due to the pandemic, and another said they felt they had finished. 100% of clients said there was nothing we could have done differently. ‘I’m not the best at talking… didn’t want to explain the situation over and over again.’ ‘We got to a point where I understood my thoughts/feeling.’ Clients who did not start EPIC (5 clients) 100% of the clients were aware of the service. 40% of clients expected EPIC to involve talking about issues and problems. Another 40% expected EPIC to help solve and understand their problems. One client said they expected to learn techniques to help with this. One client had no expectations. ‘Help understand everything that is happening. I want to learn techniques to help me manage stress and anxiety’ ‘Get my issues solved’ ‘Talking about issues’ 60% of clients said they had no concerns about meeting a psychologist; however, one said they did not want to talk, and another said they were nervous about what would come up. One client said they were concerned about meeting a psychologist as they did not want to open up. ‘Feeling a bit nervous about what will come up. Talking brings up emotions.’ ‘Didn’t really want to talk about stuff.’ ‘I initially had too much pride and didn’t feel like opening up about my mental health and didn’t know what to say.’ 100% of clients had no practical barriers; however, one client said they wanted to focus their time on work. 40% of clients said they did not want EPIC as they did not want to talk. One client said they had counselling before and did not find it helpful, and another client said they do not need it. The final client said they want the service but are waiting until they have finished university. ‘I don’t need it; I just want to move on with my life.’ Discussion This service evaluation aimed to re-evaluate the effectiveness of the EPIC service by comparing clients’ pre and post CORE-OM questionnaires and gathering qualitative information by asking clients about their personal experiences. Furthermore, it aims to provide recommendations for service improvement. The findings of the quantitative research suggest that EPIC is an effective intervention. 31.6% of clients improved in clinical score, 31.6% moved from a clinical to a non-clinical population, and 47.4% of clients moved from a clinical to a non-clinical population in one or more subsections. The current study’s data is similar to last year’s; therefore, EPIC continues to be successful. Some clients showed deterioration, 10.5% of clients significantly declined in clinical score, 10.5% moved from a non-clinical to a clinical population, and 15.8% of clients moved from a non-clinical to a clinical population in one or more subsections. However, it was typically the same clients displaying the deterioration in subsections and clinical score. The majority of clients displayed improvement or remained the same. The level of improvement in clients after only four sessions of EPIC is impressive as NICE guidelines require clients to be offered a minimum of 16 sessions of ongoing therapy. Suppose clients are expected to show strong levels of improvement after 16 sessions of therapy. In that case, clients showing strong levels of improvement after four sessions of EPIC demonstrates that the service is successful. Most clients who completed or partially completed EPIC had favourable opinions of the service, saying they found it good and liked their psychologist. However, there was a common theme amongst those who did not complete or did not finish EPIC of not liking/wanting to talk, and many clients were scared to open up. Clients who completed EPIC and partially completed EPIC had different motivations for using the service. Of those who fully completed it, over half did it because they said they needed it and wanted to learn. All clients who partially completed EPIC did it because it was recommended to them, one saying they did not do it for themselves. It appears that clients’ reasons for attending EPIC sessions affected how well they engaged. Those who sought help for their reasons completed the sessions, and those who were doing it for someone else tended to finish early. There were also different EPIC expectations between those who completed and partially completed/did not complete. Those who completed EPIC expected learning and understanding, whereas those who partially completed expected to discuss concerns. Those who did not complete EPIC also spoke about expecting it to involve talking about concerns and solving their issues. Potentially, clients who did not complete or did not fully complete did not understand EPIC well. Therefore, those who understand EPIC better and have correct expectations engage better. There was a common theme of clients being nervous about meeting a psychologist across all categories. Those who completed EPIC were less nervous about meeting a psychologist than those who partially completed EPIC. Those who did not complete EPIC had fewer nerves about meeting a psychologist; however, there was a common theme of being uncomfortable to open up and talk. Interestingly, more clients who completed EPIC said they had slight practical barriers than those who partially completed and did not complete it. This could demonstrate different levels of interest and motivation to complete EPIC. Those motivated to engage in the service will attend despite any barriers, whereas they are less likely to overcome barriers if the client is not motivated. Furthermore, all clients who did not complete EPIC were aware of the service, so promotion of the service seems to be good. Limitations One limitation of this evaluation is the limited amount of data gathered. Many clients finished the sessions early or did not complete the post-measure, leaving incomplete data sets. The quantitative sample only consists of 19 clients and the qualitative sample 13 clients, so it cannot truly reflect the service. It can provide insight into clients' experiences, but it is not possible to make solid generalisations. For future studies, it will be helpful to gain a larger sample size through more clients completing the service and the post measure. Ideally, the recommendations for improvement will assist with this. Furthermore, there could be extraneous variables affecting the outcomes of clients. EIP services also offer care coordination, medication, groups, support work visits and family interventions. Clients' post measures may be further improved due to the combination of EPIC and other care that they are accessing. Furthermore, clients may be naturally improving over time. It is, therefore, not possible to gauge a proper understanding of the solo impact of EPIC. Clients who decline EPIC could be asked to complete a CORE-OM form and another six months later. This will allow future research to compare clients who had EPIC and clients who did not have EPIC. A further limitation is the length of time after completing EPIC to completing the questionnaire. If the clients had a large gap from completing EPIC to completing the questionnaire, their answers might be affected due to forgetting or extra reflection. It could be helpful to ask clients to complete a feedback questionnaire alongside their post-measure CORE-OM form. Recommendations Some clients do not engage well with EPIC as they do not like talking; however, they still require help. Information booklets could be created covering areas of difficulty such as self-esteem and anxiety. To tailor this to clients’ needs further, a survey could be created where clients select which areas they need the most help. Clients who are more motivated to complete EPIC for themselves tend to have better outcomes. Clients need to understand the benefits of EPIC better and be motivated to help themselves. Clients who partially completed or did not complete EPIC seemed not to understand the service best. A leaflet could be created with more precise information about the service. The leaflet could also include testimonials of feedback from clients who have completed the service. This leaflet may assist clients’ motivation if it encourages and promotes the benefits. Many clients appeared to be concerned and nervous about meeting a psychologist. There is a lot of stigma and misconception surrounding psychologists. It could be helpful for clients to understand more about a psychologist’s job. This could be included in the leaflet. It may also be helpful for clients to know more about their allocated psychologist prior to starting sessions. Clients may be nervous about opening up as the psychologist is a stranger and potentially intimidating. They could meet their psychologists for a casual discussion before the sessions begin. Each psychologist could record a short video introducing themselves and a little about them that could be sent to clients. If the psychologist can appear more relatable and approachable, this could alleviate the client’s nerves before starting sessions. Most clients found the booking system easy to use. The booking system could be modernised by clients receiving a text confirmation and reminder of their appointment. Appendix Number of Clients How Many Clients in Clinical Range in Clinical Score pre and Post EPIC 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Pre EPIC Post EPIC Number of Clients How Many Clients in Clinical Range in Each Subsection pre and Post EPIC 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Risk Wellbeing Pre Symptoms Post Functioning References Implementing the Early Intervention in Psychosis Access and Waiting Time Standard: Guidance. NICE. (2016). Retrieved 19 January 2022, from https://www.nice.org.uk/guidance/qs80/resources/implementing-the-early-intervention-inpsychosis-access-and-waiting-time-standard-guidance-2487749725. Implementing the Early Intervention in Psychosis Access and Waiting Time Standard: Guidance. Nice.org.uk. (2016). Retrieved 19 January 2022, from https://www.nice.org.uk/guidance/qs80/resources/implementing-the-early-intervention-inpsychosis-access-and-waiting-time-standard-guidance-2487749725.
