The evaluation will access data from three groups of individuals: (i) women enrolled in the Exit Doors Here program; (ii) program staff and peers; (iii) partnering service providers.
All women enrolled in the Exit Doors Here program will be invited to take part in the evaluation study by completing a baseline and post-intervention questionnaire in-person or virtually (during the COVID-19 pandemic) with a trained research assistant and/or granting the evaluators access to their CTI charts and related documentation. We aim to recruit 25 to 30 women into the evaluation study each year, which is approximately 83–100% of all women enrolled in Exit Doors Here. We intend to recruit women within 2 weeks of their starting Phase 1 of Exit Doors Here. Enrolment in the program and the evaluation will occur on a rolling basis. All program staff and volunteers will be invited to take part in annual semi-directed interviews. Organizations providing services and support to program staff and clients will be invited to complete an annual online survey.
The study protocol has been approved by the St. Michael’s Research Ethics Board (approval #18–215). Prior to data collection women will provide written or verbal informed consent, service providers will provide written consent, and program staff will provide verbal consent. Women will be compensated for their time.
Data sources and tools
A number of data sources will be used as summarized in Table 1 and described in more details below.
Baseline client questionnaire
The baseline questionnaire collects socio-demographic data, information about legal issues, satisfaction, goals, and challenges related to housing, employment, education and training, and financial situations (Additional file 1). To assess participants’ level of readiness to make changes in their lives, we will use the validated University of Rhode Island Change Assessment Scale. It includes 32 statements rated on a five-point Likert scale [26, 27] which can be combined into an overall “readiness to make changes” stage. Participants’ level of social support will be assessed with the validated 12-item Multidimensional Scale of Perceived Social Support . The baseline questionnaire also includes six semi-structured interview questions pertaining to participants’ experience with, and reasons for, enrolling in Exit Doors Here, and what they hope to get out of the program.
Post-intervention client questionnaire
A post-intervention questionnaire will be administered approximately 1 month after women graduate from the program (Additional file 2). Similar questions as in the baseline questionnaire will be asked, in addition to nine questions assessing participants’ level of awareness and use of community services and supports after program completion . Women will be asked to describe their current level of involvement in sex work as “not involved at all”, “involved rarely” or “involved frequently”. Open-ended questions will assess changes made by women to their housing, employment, education and training, and financial situations, and goals remaining to be achieved. Ten semi-structured interview questions will explore participants’ overall experience in the Exit Doors Here program.
To assess participants’ progress on their selected CTI focus areas, we will rely on the Empowerment Star, an evidence-based tool originally developed for women who have experienced domestic violence but which can be adapted to other contexts . The Empowerment Star is designed to support and measure change, and is underpinned by a person-centred, strengths-based and co-production approach to service delivery . It is completed by clients together with their case manager, keeping clients’ perspectives and priorities front and center. At the beginning of each of the three CTI phases, clients rate the stage of change they feel they are at for all nine subscales of the Empowerment Star which overlap to some extent with CTI focus areas: safety, accommodation, support networks, legal issues, health and wellbeing, money, children, work and learning, and empowerment and self-esteem. This allows tracking of each individual’s journey of change through five stages: “Not ready for help”, “Accepting help”, “Believing”, “Learning and Rebuilding”, and “Independence and Choice”. The Empowerment Star is informed by the Transtheoretical Model (TTM) of Change which stipulates that intentional change in behaviour occurs as a process, not at a specific moment in time, and recognizes that individuals can transition back and forth between stages until they achieve stability [31, 32].
CTI client charts
Case managers complete several CTI charts documenting information about, and meetings with, their clients: (1) an intake form completed upon first meeting with the client to gather background and referral information; (2) an assessment form documenting clients’ strengths, risks, and needs; (3) a detailed phase treatment plan for each CTI phase based on clients’ goals; (4) progress notes documenting contacts between case managers, their clients and community and informal supports; (5) a contact information form; and (6) closing notes summarizing clients’ progress during the program. Data needed to answer evaluation questions will be systematically extracted from participants’ CTI charts.
CTI fidelity scale
The Dutch adaptation of the CTI fidelity scale will provide a quantitative assessment of two dimensions of program fidelity (i) compliance fidelity and (ii) competence fidelity . Compliance fidelity measures the degree to which staff are practicing the key elements of CTI. It is measured by assessing the extent to which the program is being delivered to its intended structure and with skill and attention to the CTI model. Competence fidelity is assessed by measuring chart quality and completeness. The Dutch CTI fidelity scale comprises 12 items with one to five criteria each which can be rated as “fulfilled” (score of 1) or “not fulfilled” (score of 0) using data from participants’ CTI charts. Fidelity will be assessed in the first year of the program to provide staff with recommendations for change and improvement, and in each subsequent year for quality control purposes .
Every year, Exit Doors Here staff and volunteers who provide direct services to clients will be invited to take part in a 20 to 30-min semi-structured interview conducted face-to-face or over the phone. The interviews will help identify factors which might influence program implementation, fidelity, and outcomes, and will inform the contribution analysis.
Service provider survey
A survey for service providers, jointly devised by the program and evaluation teams, will be used to assess whether service providers’ needs and expectations are fulfilled by the program and what could be done to improve on these fronts. The survey includes a mix of close-ended questions on 5-point Likert scales and open-ended questions.
Process evaluation questions
To assess whether program recruitment strategies are effectively reaching target clients (Table 1, Q1), data from CTI charts will be used to compare the demographic characteristics and types of sex workers (street-based, trafficked, and non-identifying sex workers) recruited into the program to those of the target population. Client retention (Table 1, Q2) will be assessed every year and at the end of the program duration. In fact, although CTI has a strict “no drop-out” policy, it may happen that women do not complete the program. We will compare the number of women who enrolled in the program to those who graduated using information from the program director’s yearly reports to the funding agency.
To assess the extent to which the program is being implemented with fidelity to the CTI model (Table 1, Q3), data from participants’ CTI charts will be used to complete the CTI fidelity scale. In the first year, the Dutch version of the fidelity scale will be validated against data from a subset of women (n = 8) and results will be discussed with the program team. The fidelity scale may be adapted accordingly to better reflect the context of working with women attempting to exit sex work. Following this validation step, the adapted fidelity scale will again be completed every year using data from five randomly selected participants. Item-level fidelity ratings will be interpreted in light of a 5-point rating scale: program not implemented in line with CTI model, poorly implemented, fairly implemented, well implemented, and ideally implemented . Each year, interviews with program staff and volunteers will complement the interpretation of fidelity scores to identify factors that may enable or constrain high fidelity in program delivery. Subsequent adjustments may be made to the program if needed . Finally to assess if the program is meeting service providers’ needs and expectations (Table 1, Q4), responses to the service provider survey will be described in frequency tables or qualitatively summarized.
Primary outcome evaluation questions
To assess the change in participants’ level of social support (Table 1, Q5), we will compute the difference in overall mean scores across the 12 items from the Multidimensional Scale of Perceived Social Support  between post-intervention and baseline, and report the proportion of participants with an increase, decrease, or no change in perceived social support.
To assess if participants advanced their readiness to make progress on their housing, pre-employment, and income-related goals (Table 1, Q6, 7, and 8 respectively), data will be systematically extracted from participants’ closing notes in which case managers document the changes made, and progress achieved, by each participant, for each of these areas. Outcomes may include “submitted subsidized housing application”, “signed up for MyStartUp or H&R Bloc training program” or “secured Ontario Disability Support Program (ODSP)”. With these data we will categorize women according to whether they have progressed towards their initial goals. This is will be complemented by post-intervention questionnaire data in which women discuss facilitators and barriers to making progress (or not) on their initial goals.
Since exiting sex work is a process, it is unrealistic to expect that all clients will attain their long-term goal of exiting sex work within the limited timeframe of CTI. In fact, the stated goal of CTI is to “successfully link clients to supports who will eventually take over helping clients meet their long-term goals” . To assess whether participants have reduced their involvement in sex work (Table 1, Q9), we will compute the proportion of women reporting being “involved rarely” and “not involved at all” in sex work in the post-intervention questionnaire.
Secondary outcome evaluation questions
Participants’ progress on their chosen CTI focus areas (Table 1, Q10) will be assessed by comparing their Empowerment Star ratings from the beginning of Phases 1 and 3 and reporting the proportion of women who, among those who will have chosen each of the nine Empowerment Star focus areas, will have “progressed at least one stage”, “remained at the same stage” or “regressed at least one stage” for each of the nine Empowerment Star areas.
To assess participants’ progress in terms of readiness to make changes in their lives (Table 1, Q11), we will follow the University of Rhode Island Assessment Scale guidelines to compute their overall “readiness to change” score and assign participants an overall stage of readiness to change: pre-contemplation, contemplation, action, and maintenance . Stages will be compared between baseline and post-intervention assessments. We will report the proportion of participants who have moved forward on the continuum of stages, those who have remained at the same stage, and those who have moved backwards on the continuum of stages. Finally, to explore participants’ awareness and use of support services at the end of the program (Table 1, Q12), we will compile the proportion of participants reporting agreement with each of the nine statements tapping into this concept.