Which of our CBT skills affect outcomes?
I know many psychologists feel bombarded with professional development opportunities and with every new therapy name they see advertised, feel increasingly insecure about their knowledge and abilities. This is why I love big-picture studies of therapists in routine practice that aim to sort the wheat from the chaff regarding what’s important for us to know that makes a difference to client outcomes. This study was published last year but I haven’t seen a study like it since so I’m calling it current. Uhl and colleagues (2022) set out to see whether symptom improvement could be predicted by microskills and “correct application of techniques/strategies” (can you guess which country this study was conducted in? So German).
The authors collected self-reported symptom data from 398 clients of 48 therapists and had independent observers rate therapists’ use of microskills and specific cognitive behaviour therapy intervention skills from video recordings of sessions. Their Microskills measure covered ratings of the therapist’s: a) pacing and efficient use of time; b) clarity of communication; c) use of feedback and summaries; d) rationale; e) guided discovery; f) therapeutic relationship and collaboration; g) handling problems/questions/objections; h) empathic understanding; i) focusing on key cognitions and behaviors; j) strategy for change and k) application of techniques. The Interventions scale measured the extent to which specific techniques were delivered correctly, including: a) therapy goals; b) functional analysis; c) psychoeducation; d) suicidality/crisis interventions; e) behaviour modification; f) cognitive techniques; g) skills training; h) exposure or behavioural experiments; i) emotion regulation; j) mindfulness/acceptance; k) clarification of schemas/needs/motives/values/goals; l) emotion-focused techniques; m) interpersonal techniques; resource/solution-oriented techniques; o) homework; p) use of psychometric feedback; q) motivational clarification; r) problem actuation; s) mastery and t) resource activation. See Box for more detail about how high and low ratings were operationalized. Observers also rated the difficulty of the session (“how difficult was the treatment with this patient?”).
Before discussing the main results, a couple of other findings are worth highlighting. Microskills and intervention skills were highly correlated (r = 0.61 [0.57, 0.64]) which could reflect several fairly obvious things: good “intervention” delivery relies on good microskills, good intervention skills are good microskills applied toward well-chosen skill development tasks and people who are good at one set of psychotherapy skills are more likely to be better at another set. If you look at how intervention skill ratings were given (see Box), you’ll see that higher ratings would require use of good microskills in the optimum delivery of the technical interventions. More interestingly, intervention skill application was negatively correlated with symptom severity; the more severe the client’s psychopathology at a given session, the less well therapists implemented interventions. OK, maybe that’s not so surprising, but maybe more surprisingly, it was a fairly small effect (r = -1.1 [-0.17, -0.05]). Even so, lower intervention competence scores predicted (a small) deterioration in symptom severity in subsequent sessions (r = -.08 [-.14, -.01]). Symptom severity had no effect on microskill competence but conversely, microskill competence also did not predict subsequent symptom severity. So, overall, intervention skill mattered. A little bit.
If you want the complicated stats, read the main article. Here, I’ll assume life’s too short for most readers. The main goal of the study was to understand therapist effects (i.e., how much differences in client outcomes was attributable to differences between therapists). In this study, therapist effects explained approximately 4% variance in client outcomes. Interestingly, although within-therapist differences between sessions in intervention skill competence predicted client improvement, average differences between therapists did not. In other words, the variation in each clinician’s performance from session to session or client to client made a difference to how much clients in their care improved, but in this study at least, no therapists achieved consistently better aggregated outcomes than other therapists. Observer ratings of session difficulty also predicted whether clients improved, as did the interaction between treatment difficulty and intervention competence. This means that clients who were harder to treat achieved greater symptom reduction when therapists executed intervention skills better. Again, microskill competence did not predict client improvement.
This study took place in an outpatient university training clinic in Germany with therapists having obtained their Master’s degree in clinical psychology, 1.5 years’ experience and participating in 3-5 years postdoctoral training. Therapists had a mean age of 31 years. Most clients had mood disorders (46%), stress and adjustment disorders (21%) and anxiety disorders (21%). The clinic practices cognitive behaviour therapy in its broadest sense, including third wave approaches. Although manualized disorder-specific protocols are sometimes used, the clinic also encourages flexibly tailored formulation-driven treatment plans. Therapists were supervised individually or in groups after every 4 sessions with a client. Therapy courses were quite long, with an average of 34 sessions. So, how relevant these results are to your work depends on how similar these characteristics are to those of your workplace. Except for the experience of the therapists and typical length of treatment, there is much in common with the student clinics I’ve managed.
Whether differences in average effectiveness between therapists are found in a study will depend on how heterogeneous the experiences of the therapists in the sample are. In RCTs, between-therapist therapist effects can often be minimized by selecting therapists, careful client screening, consistent training, manualized procedures, and supervision. Uhl and colleagues’ training program appears to have achieved good consistency across students. Their results are also consistent with what we all know from experience and what research studies also increasingly demonstrate: no matter who we are, we have good sessions and not so good sessions, and we’re more likely to do a poorer job of more technical tasks with more difficult clients. Therefore, the most helpful things to practice are empirically supported interventions with peers simulating clients experiencing severe symptoms.
Clinician implications
Competence in CBT microskills such as session structuring, giving feedback and summaries, expressing empathy, using guided discovery and skills in building the collaborative relationship, by itself may not improve client outcomes.
Therapists should continue to develop competence in executing cognitive and behaviour change strategies.
CBT microskills are not trivial, they are necessary to implement interventions with good competence. Therefore, therapists should focus on applying their microskills in session in the context of delivering empirically supported interventions (and practice the same, outside of session).
The Inventory of Therapeutic Interventions and Skills (ITIS) is a comprehensive checklist which is more in keeping with contemporary practice than older rating scales like the Cognitive Therapy Rating Scale. Clinicians should experiment with video recording their sessions and evaluating themselves against it (or have a supervisor whom they trust and respect, rate their performance on it).
Supervisors should remember that client severity influences clinician performance. Supervisees may need assistance in responding to client disengagement.
Clinicians should resist abandoning sound change strategies when clients are experiencing severe symptoms and may present as disengaged; change strategies are more likely to relieve their distress than empathy and rapport building alone (do both!)
For the original article, go to: https://psycnet.apa.org/record/2022-63973-001
REFERENCES
Uhl, J., Schaffrath, J., Schwartz, B., Poster, K., & Lutz, W. (2022). Within and between associations of clinical microskills and correct application of techniques/strategies: A longitudinal multilevel approach. Journal of Consulting and Clinical Psychology, 90(6), 478–490. https://doi.org/10.1037/ccp0000738
Appendix : Therapist skill ratings in more detail
The study employed a rating scale the authors had previously developed called the Inventory of Therapeutic Interventions and Skills (ITIS). The rating scale definitions are available in the supplementary material of the published article but I summarise this here briefly.
Microskills
Poor ratings were usually given for the absence of the skill whereas excellent ratings were given for frequent and clear delivery, although some were more complex. Poor pacing and use of time could mean the session was aimless and the client completely dominated, or the therapist too rigidly structured the session; good time use and structure meant both topics were evenly distributed and responsive to the client’s abilities. Poor communication involved excessive jargon, incoherent communication or communication not suited to the client’s abilities, whereas good communication was clear. Excellent use of feedback involved the therapist attending and responding to client verbal and nonverbal feedback throughout the session as well as regularly checking out the client’s understanding. Poor ability to handle problems, questions and objections could mean the therapist didn’t notice problems, ignored them or tried not to answer questions. Excellent focus on key cognitions and behaviours ratings were given when the therapist identified beliefs and behaviours that seemed most relevant to the client’s problems. Excellent strategy for change ratings are given to therapists who selects interventions that seem to target changing the most relevant beliefs or behaviours for the client at that time. Application of techniques refers to how well the selected interventions were implemented, where poorer ratings are given for technical errors.
Intervention skills
Therapy goals: poor ratings were given for not specifying goals concretely enough; failing to write them down or (later in therapy) not using the client’s goals to direct the session.
Functional analysis: high ratings are given for comprehensively describing the problematic experience and its triggers and reinforcers (includes all relevant cognitive behavioural components) and the inter-relationships between components are made clear to the client.
Psychoeducation: High ratings are given for clear, understandable explanations and harnessing clients’ prior knowledge and experiences to make current issues clearer.
Suicidality/crisis interventions: Poor ratings are given when therapist doesn’t explore topic in enough detail to assess client’s risk level and if coping strategies are only discussed vaguely/generally and not written down.
Behaviour modification: These activities include behavioural activation, stimulus control, improving daily structure and arranging rewards contingent on increasing desirable behaviour or reducing undesirable behaviour. Poor ratings are given for vague discussions without identifying concrete behavioural changes. High ratings are given for illustrating behavioural principles with personally-relevant examples and planning changes based on evidence of past behaviour-mood relationships.
Cognitive techniques: Poor ratings are given for unclear identification of thoughts and beliefs and/or unsuccessful efforts to change them. High ratings are given for identification of specific automatic thoughts associated with unwanted emotions and effective efforts to change the impact of those thoughts.
Skills training: Examples include social skills training, communication, problem-solving, relationship training, sleep hygiene, relaxation and any guidance in developing new abilities. High ratings are given for in-session practice or making concrete plans for practice in the client’s life.
Exposure/behavioural experiments: Low ratings are given for errors (e.g., allowing client to use safety behaviours or other avoidance, not debriefing what was learnt) and high ratings for preparation, execution and debriefing that maximizes the client’s opportunity to learn from the experience.
Emotion regulation: Low ratings are given for not personally tailoring strategies or providing in-session practice opportunities. Higher ratings are given for tailoring strategies to suit the client’s needs and abilities, giving opportunities in session to practice and planning specifically when and how the client will use these outside of session.
Mindfulness/acceptance: Low marks are given for vague descriptions, no in-session practice or incorporation into therapy process. High marks are given for clear rationale, in-session practice and reflection to facilitate skill development.
Clarification of schemas/needs/motives/values/goals: Higher marks are given for helping clients better understand what motivates their current behaviour patterns and clarify what motives they wish to drive their future behaviour.
Emotion-focused techniques: This refers to any activity that focuses clients on paying attention to and experiencing their emotions and associated bodily sensations. Lower ratings are given when it’s unclear whether the client really felt their emotions and the exercise doesn’t result in a reduction in how distressing the client finds their emotions.
Interpersonal techniques: This includes understanding patterns of interacting with others and making plans to change the way the client interacts with others. Low ratings are given when the therapist fails to have the client reflect on their own contribution to interactions or gain greater understanding of historical and/or biological influences on their patterns of interaction. Higher ratings are given when the therapist helps the client identify their contribution to problematic interpersonal interactions, understand historical influences on the formation and perpetuation of the pattern and work toward new goals for interactions.
Resource/Solution-oriented techniques: This refers to interventions that teach the client to harness their own strengths and resources to solve their problems. Low ratings are given when it is unclear whether the client has the strengths the therapist discusses or recognizes them as such, or when the therapist focuses on discussing strengths to the extent the client feels their suffering is invalidated.
Homework: Ratings of excellence are given for: a) when reviewing homework, taking enough time to allow the client to reflect on what was learnt, or understand and address difficulties with implementation; and b) when setting homework, clearly explaining the rationale and procedure, checking the client’s willingness and making specific implementation plans.
Motivational clarification: Low ratings are given for failing to help the client develop further insight into their motives. High ratings are given for helping clients understand their motives, as shaped by biographical experiences which improves insight and self-acceptance.
Problem actuation: This refers to exercises intended to have the client vividly experience their problematic pattern to better appreciate its qualities or impact (e.g., some chairwork, psychodrama exercises). Low ratings are given when it is unclear whether the client has been emotionally stimulated or remains in a mainly detached or intellectual state. High ratings are given for evident emotional activation that facilitates new learning.
Mastery: This refers to work on strategies developed to target disorder-specific deficits. Low ratings are given when the therapist suggests strategies that are either too general (irrelevant) or too difficult (overwhelming). High ratings are given when the therapist suggests well-targeted strategies and prepares implementation by checking the client’s willingness, anticipates and overcomes barriers and plans specifically how and when the strategy will be implemented.
Resource Activation: This refers to the therapist’s ability to build the client’s self-efficacy by helping them be aware of their strengths and attribute improvements to their own efforts.
Use of psychometric feedback: This refers to using information like symptom questionnaires, working alliance inventories, or achievement of goals (e.g., Goal Attainment Scaling) to provide feedback on progress. Higher ratings are given when therapists give a clear explanation of the data and why they are presenting it.