Therapists play a crucial role in the dissemination of Evidence-Based Psychotherapies (EBPs) such as Cognitive Behavioral Therapy (CBT) [1] and the success of the implementation can depend on the proficiency of their performance [2-4]. However, therapists often have reservations about the implementation process, such as concerns about the validity of new treatments, overloaded work schedules, and limited time for training [5,6]. Even with training, clinicians may struggle to incorporate new skills into their therapeutic repertoire and may lack confidence in their ability to do so [7,8].

The effective implementation of EBPs in community mental health requires cognitive and behavioral changes in newly trained therapists [5-7]. This includes learning to re-conceptualize clients’ problems, understand and apply the underlying theoretical model, and act in a manner consistent with the treatment methods. These cognitive and behavioral shifts, particularly for therapists accustomed to other forms of treatment, require confidence in their abilities to learn and execute new skills.

According to social cognitive theory [9], one’s self-efficacy determines, in part, whether an action will be initiated, how much effort will be applied to the action, and how long the action will be sustained. Counseling self-efficacy has been defined as “one’s beliefs or judgments about her or his capabilities to effectively counsel a client in the near future” [10]. Greater counselor self-efficacy has been linked to lower counselor anxiety [11], higher counseling skill level and performance [10,12,13], and more congruence between perceptions of the counselor and the client regarding the quality of the counseling session [14]. The effective dissemination of CBT should include training that not only teaches therapists the clinical methods, but supports clinicians’ self-efficacy for delivering the intervention. CBT is a highly interactive therapy, and its success is highly dependent on the clients’ acceptance of the therapeutic model and willingness and ability to actively participate in the process [15]. Therapists that doubt their abilities and/or have little faith in the therapeutic model will have difficulty convincing their clients of the validity of the treatment.

Extended supervision is an important aspect of any dissemination effort. In a synthesis of the dissemination literature, Fixsen et al. (2005) [16] reviewed more than 1,000 articles published in a 35 year period and concluded that one of the core components of successful implementation is ongoing consultation and coaching. Just as clinical supervision was critical to initial skill acquisition during graduate training, evidence suggests that providing clinical supervision to experienced therapists in addition to didactic training improves the acquisition of new skills [17-19]. Further, it has been noted that if supervision is too brief, improvements in skills not only cease, but also tend to decline over time [20]. If clinical supervision improves skill acquisition and greater skill is associated with enhanced self-efficacy, then it is possible that including clinical supervision as a component of the treatment dissemination process could increase therapist self-efficacy for delivery of CBT.

To date, the empirical support for the role of supervision in improving therapist self-efficacy has been mixed. In university training clinics or laboratory settings with graduate student trainees, some studies have found an increase in self-efficacy related to participation in clinical supervision [21,22] while other studies have not found such an advantage [23,24]. The present study was aimed at shedding light on the link between participation in ongoing consultation and coaching and therapist self-efficacy in seasoned clinicians. Masters level clinicians were surveyed during a large scale dissemination of CBT for Major Depressive Disorder (MDD). The dissemination effort was part of a state-wide initiative in Texas from 2005 to 2011 to redesign assessment processes in the mental health service system. Within Texas, 39 Local Mental Health Authorities (LMHA) provide public mental health services for the government [25]. Therapists from fifteen LMHAs participated in the present study.

This introduction of psychotherapy into a system that primarily provided medication treatment required not only the introduction of new procedures, but also a paradigm shift away from the medical model of depression to a more biopsychosocial model of depression [26]. To accomplish this task, didactic training was provided for nearly 300 therapists throughout the state in groups of approximately 25 therapists. A subset of therapists volunteered for or was assigned to participate in six to nine months of telephone-based group supervision with an expert in CBT. It was hypothesized that therapists who received didactic training in conjunction with ongoing supervision would have greater self-efficacy for delivery of CBT and for their therapeutic skills in general, as well as greater acceptance of the CBT model than those who received only didactic training. Because of the collaborative nature of CBT, it is likely that therapist self-efficacy for CBT would be, in part, related to therapists’ beliefs in their clients’ abilities to do their part in executing the treatment. Therefore, it was also hypothesized that self-efficacy for implementation of CBT would be accompanied by increased confidence in clients’ abilities to participate in CBT.

Participants

Participants were 272 Master’s degree level therapists who received a four-day (26 hours) didactic workshop on CBT for MDD as part of a statewide initiative to disseminate CBT to community mental health centers in Texas. A subset of 30 therapists, who were assigned by superiors or volunteered themselves, participated in weekly telephone based supervision for six to nine months. Therapists were contacted to take part in a survey regarding their participation in supervision and their experiences with providing CBT for their clients with MDD. The Institutional Review Board of the University of Texas at Arlington approved the study, and all participants provided informed consent prior to participation.

Email addresses were provided by the therapists at the time of training. All were contacted; however, 80 emails were undeliverable due to change of employment. Forty-five therapists (16.5%) completed the survey. Those who previously participated in group supervision following the didactic training (n=13) constituted the supervision group, and the remaining respondents (n=32) constituted the no supervision group. Of the 45 participants who completed the survey, five of those from the no supervision group had experienced a change in job duties since the time of the didactic training and had subsequently not provided any type of psychotherapy. Their responses were excluded from all analyses (Figure 1).

Procedure

Didactic training

Didactic training included review of diagnostic criteria for MDD, discussion of the theoretical model underlying CBT, and skills training in the most commonly used cognitive and behavioral interventions for MDD. Training methods included lecture, demonstration, role-plays, practice of interventions among trainees, and observation of taped vignettes as well as full therapy sessions. To facilitate dissemination of CBT, the trainees were provided with an 18-session protocol adapted from Wright et al. [27] that included specific session structuring behaviors, suggested and alternate CBT skills to be taught at each session, and recommended homework assignments. Therapists were instructed to follow this protocol during delivery of CBT for MDD.

Figure 1: Flow chart of participant recruitment and completion of survey

Clinical supervision

Three groups of therapists, with six to eight therapists per group, received group supervision via teleconference. Participants either volunteered or were selected by their clinic supervisor to participate in this supervision. Therapists were provided with the opportunity to attend supervision up to twice weekly during which they presented audiotapes of therapy sessions on a rotating basis. Clinicians provided background clinical information on each case prior to review of the session and identified specific areas on which they wanted feedback. The supervisor (second author) identified and reinforced areas of strength in the therapists’ performances and opportunities for improvement. Supervisees were encouraged to brainstorm as a group on intervention strategies as a way of practicing case conceptualization.

Survey Construction

As a group exercise during two rounds of the didactic training, trainees were queried about their concerns regarding their implementation of CBT and their responses listed on a flip chart. This activity was audio and video taped for later review. The issues raised by the two groups of trainees were very similar and centered around concerns about their own abilities to execute CBT in accordance with the protocol, clients’ abilities to actively participate in and benefit from the treatment, and the applicability of the CBT model in their complex client population.

These themes were captured in the development of four groups of self-report items that tapped: (1) self-efficacy for CBT, which included therapists’ concerns about their abilities to conduct CBT, including following the suggested structure and executing the interventions; (2) client CBT ability, which included therapists’ predictions about their clients’ abilities to understand CBT activities and participate in each component of the session such as contributing to the agenda, recognizing automatic thoughts, and completing homework; (3) efficacy of CBT, which included the usefulness of CBT in meeting the needs of the client; and (4) self-efficacy as a therapist, which included therapists’ perceptions of their general therapeutic skills. Six questions were generated for each of the first three content areas (self-efficacy for CBT, confidence in client CBT ability, and efficacy of CBT) and seven questions were generated for the last area (self-efficacy as a therapist). Each item was measured using a five point Likert scale (1, “Very Limited” to 5, “Very High”). Responses to each question were combined within the four subscales with a potential score of six to 30 for the first three subscales and seven to 35 for the fourth subscale. See Table 1 for a complete list of questions.

The survey also assessed each therapist’s frequency of use of CBT with MDD clients since the training, participation in additional CBT training or supervision, and proportion of workload allocated to delivery of psychotherapy. The survey was pilot tested among psychology undergraduates for understandability and readability, and adjustments were made in accordance with their feedback.

Survey Administration

Clinicians were contacted by email and asked to complete the survey via SurveyMonkey.com regarding their experiences with CBT for depression. Respondents electronically provided informed consent to participate before they were presented with survey questions. As part of the consent procedure, respondents were notified that their identifying information would not be collected in order to protect their anonymity. Given the small number of clinicians who received supervision, the investigators’ familiarity with supervisees’ personal demographic data could have made it possible to link the respondents to his or her answers. Therefore, to protect their confidentiality, no demographic data were requested.

Analytic Strategy

Prior to analyzing survey results, data were screened for missing values. If a therapist did not answer any questions within a subscale, his or her data were excluded from the analyses for that scale. This resulted in excluding one therapist from the analysis on the efficacy of CBT scale. If the therapist did not answer one to two questions within a scale, group mean substitution was used to replace the case with the item mean for that supervision condition. This resulted in only eight substitutions across all responses. Reliability analyses were run on each of the four subscales and it was determined each subscale had adequate internal consistency (a=.89 or greater) (Table 1).

Table 1: Therapist self-efficacy survey: four subscales.

Four independent samples t tests were run to examine the extent to which being involved in clinical supervision was related to the four variables of interest: clinicians’ self-efficacy for providing CBT, clinicians’ self-efficacy for their basic therapy skills, confidence in their clients’ abilities to actively participate in all elements of the CBT process, and their belief in the CBT model’s ability to address the psychotherapeutic needs of their clients. Significance level was set at p < 0.0125 to account for family wise error associated with conducting four t tests with correlated data. A MANOVA, although preferable, was not utilized due to lack of power given the small sample size. Table 2 summarizes the means and standard deviations for the four survey scales of the full sample, the supervision group and the no supervision group.

Table 2: Therapists' self-efficacy scores according to participation in supervision (N = 40).

Therapist Activities

Given the complex nature of such a field study, some preliminary analyses were conducted on psychotherapy workload (e.g., percentage of job spent in therapy), protocol adherence (e.g., working within the confines of recent training), and experience with CBT since training (e.g., number of clients seen) in an effort to characterize the sample and provide context for the survey results.

Psychotherapy Workload

Therapists in community mental health clinics have several responsibilities in addition to providing psychotherapy, including diagnostic assessment, case management, and supervision of staff. Table 3 summarizes the portion of workload allocated to providing psychotherapy and number of clients treated since participating in CBT dissemination training. There was no significant difference between the supervision group and the non-supervision group with regard to number of clients seen (X2 (2, N = 40) = 1.52, p = .467, ns) or percentage of workload allocated to providing psychotherapy (t(39) = -.029, p = .845, ns).

Table 3: Therapists' workload: percentage of responsibilities and number of clients (N = 40).

Self-Reported Protocol Adherence

Therapists were asked to report the extent to which they followed the 18-session protocol for CBT provided to them during training. There was a significant association between supervision group and adherence to the protocol (X2 (1, N=40)=6.87, p=.010) such that those that did not participate in supervision were more likely to answer that they followed the protocol none, minimally, or some of the time and those that participated in supervision were more likely to answer that they followed the protocol mostly to all of the time.

Survey Results Supervision and Self-Efficacy

Results showed that the supervision group reported greater self-efficacy for delivering CBT skills than the no supervision group, t (38)=-2.61, p=.013, although the difference was marginal. In addition, the supervision group reported greater self-efficacy than the no supervision group for their basic counseling skills, t (38)=-3.00, p=.005.

Similarly, the supervision group reported significantly greater confidence than the no supervision group in their clients’ abilities to actively participate in the CBT process, t (38) = -4.274, p < .001. Finally, when responding to questions regarding the efficacy of CBT in addressing the depressive symptoms in their clients, the supervision group had significantly greater belief in CBT as an effective psychotherapeutic model than the no supervision group, t (37)=-4.274, p<.001.

Experience and Self-Efficacy

To explore the possibility that increased self-efficacy was a reflection of greater practice with CBT, a hierarchical regression analysis was conducted to explore the relationship between self-efficacy for CBT and number of clients treated with CBT. Self-Efficacy for CBT was regressed on supervision condition, while controlling for number of clients in psychotherapy since training. Supervision was still a significant predictor of Self-Efficacy for CBT, (F (2, 37) = 7.06, p < .01), accounting for 23.7% percent of the variance (Adjusted R2 = .237, ΔR2 = .18), indicating that while experience in delivering a new intervention was important, supervision was also a critical component.

The purpose of this survey was to gain a better understanding of therapist-centered outcomes associated with participation in clinical supervision to support psychotherapy dissemination. Specifically, the focus of this survey was on how useful supervision is in strengthening therapists’ confidence in their skills, the therapeutic model, and their clients’ abilities to engage in treatment. In the present sample, those that participated in clinical supervision after didactic training reported significantly greater confidence in their own overall abilities as a therapist, their client’s CBT abilities, and the cognitive behavioral approach to the treatment of depression than those who did not receive supervision. Greater confidence in their CBT abilities, did not reach statistical significance, though clearly showed a trend in the expected direction.

As the goal of dissemination of evidence-based psychotherapies becomes more widely accepted, a better understanding of the dynamics of that process is needed to inform such efforts. One key question is how much training is needed to support the dissemination process. As social cognitive theory would predict, dissemination is not only about the delivery of treatment, it is about enduring the complex behavior change and the social context and attitudinal changes that sustain it. The present findings are particularly interesting given the system-wide dissemination of CBT for MDD in a community mental health system that had historically focused on medication management rather than providing psychotherapy for depression. During the initial didactic training, prior to initiation of the supervision phase of the dissemination process, clinicians consistently expressed wariness. The dissemination changed the therapeutic culture of these clinics from a medical model of depression and clients as passive recipients of care to a more biopsychosocial model that emphasizes patient empowerment. This shift in the therapeutic culture was aimed at all clinicians within the system, regardless of participation in CBT training or supervision.

The importance of supervision has been supported as an important part of learning a new treatment method [18]. While prior research on the link between supervision and therapist self-efficacy had been mixed [21-24], the present study found very clear support for this relationship; this difference is perhaps attributable to the surveying of experienced Masters level clinicians in the present study as opposed to graduate student clinicians in prior work.

Counselor self-efficacy generally tends to improve with repeated practice [28]. However, in this sample, number of patients treated with CBT did not predict self-efficacy. In fact, taking part in supervision accounted for a three-fold proportion of the variance in self-efficacy compared to practice alone, underscoring its potentially important role in the process of training and treatment dissemination.

In addition to training in CBT specific skills, supervision provided opportunities to reinforce general therapy skills, discuss how to engage clients in the treatment process, and how to incorporate the new therapy into their existing repertoire of clinical skills. Perhaps it is for these reasons the survey showed that those who received supervision not only felt more empowered to deliver CBT than the comparison group, they also felt more confident in their abilities as therapists in general as well as in their clients’ abilities to handle the demands and complexities of this treatment approach. Greater confidence in patients’ abilities to engage in CBT and the observed usefulness in reducing symptoms have been viewed as keys to long-term dissemination and adoption of EBPs [5,7].

Of course, it is possible that improved self-efficacy is with a byproduct of developing stronger CBT skills as measured by objective assessment such as ratings on the Cognitive Therapy Scale (CTS) by trained experts. Lopez and Basco (2014) [25] examined therapist competency in a similar group of therapists from the Texas community mental health system who received clinical supervision in CBT for six months following comparable didactic training. They found that those who participated in clinical supervision following didactic training achieved a moderate level of competency in CBT with average CTS ratings of 34.3 (SD = 7.8). Although this is somewhat lower than the standard for randomized controlled trials of CBT conducted in university settings (CTS = 39 or greater), the clients receiving treatment tended to have co-morbid psychiatric, medical, and other psychosocial problems that would commonly exclude them from randomized controlled clinical trials.

The findings of this survey may be generalizable to therapists involved in dissemination efforts. However, those participating in the supervision group either self-selected or were asked to participate by their clinic supervisors. Therefore, their baseline levels of self-efficacy may have been greater than those not receiving CBT supervision. Additionally, many therapists could not be reached by email due to transitioning out of their jobs after initial training. This is not surprising, as a recent study found voluntary turnover rates as high as 80% in an evidence-based implementation effort [29]. It is possible that the four scales utilized in this study are not independent, but rather tapping into larger constructs of overall dissemination effectiveness in therapist practice. Due to a small sample size, the current study did not have enough power to analyze if these were, in fact, different factors, but they all did indeed show a consistent effect of supervision having a positive influence on self-efficacy for CBT. As previously mentioned, minimal personal information was collected from participants to protect their anonymity. Future studies should consider the influence of demographic or geographic factors on self-efficacy for CBT.

Therapists at mental health centers play a critical role in the dissemination of EBPs and their comfort in the delivery of new interventions is critical. Therefore, more work is needed in this area to be able to untenably declare the role of supervision in fostering self-efficacy. Future studies should focus on validating the scale and replicating its findings in other training efforts. Future work might also examine supervisory style and quality of relationship to determine if this is a factor in supervisee self-efficacy. The results of the current study suggest that while didactic training is helpful for therapists to be introduced to the structure of the EBP, ongoing clinical supervision can play an important role in influencing therapists’ confidence both in their skills and in the EBP as a valid treatment.