The effect of therapist competence in psychological therapies on client outcomes

Of course, competence as a mental health professional is important. How could it not be? The Australian Psychological Society Code of Ethics even specifies that psychologists maintaining appropriate skills and learning, working within the limits of their experience and training and basing their practice on the science of psychology are key components of ethical practice. Master of Psychology courses in Australia are now being accredited according to whether they can demonstrate that their graduates are competent in a range of clinical skills and practices. So, you would expect there to be a robust literature demonstrating the role of clinician competence and its powerful effects on the mental health of clients served by clinicians. Right?

This week I summarise a scoping review published by Ottman and collegaues (2020) on therapist competency and its association with client outcomes in psychological interventions. Scoping reviews have crept into existence as the literature has become increasingly dense and constitute a kind of short-cut snapshot of the state of the literature by looking at publications – many of which themselves might be systematic reviews - within a relatively brief, defined recent period. Ottman et al revealed that there remain two seminal meta-analyses on the subject, which I discuss in some detail here.

Webb and colleagues (2010) conducted the most authoritative meta-analysis to date on adult clients receiving individual therapy in randomized controlled trials. It consisted of 17 studies which had to have at least 5 clients per treatment condition, using clinical samples (not university or analogue samples) and include a quantified measure of competence based on independent expert raters of either video or audio recordings or transcripts of sessions. The outcome measures included in the meta-analysis varied according to the study but independently rated measures like the Hamilton Rating Scale for Depression (HDRS) were preferred, then client self-report measures, with therapist-rated scales only used if one of the former two kinds wasn’t available.

The first key finding was that overall, the weighted average correlation between therapist competence and outcome was not significantly different to 0; z = .070 [-.069, .201]. In general, in RCTs of individual psychotherapy for adults, there is no demonstrable relationship between therapist competence and outcome. The second key finding was that there was significant variation in the strength of the competence-outcome correlation across studies, with approximately 60% of the variance in effect sizes due to true differences between the studies rather than sampling error. Differences in effect sizes could not be explained by differences in therapy protocol (e.g., CBT v interpersonal v emotion-focused v psychodynamic). Some of the variability between studies could be explained by the problem type; the strongest correlation between competence and outcome was evident for depression (r = .28 [.14, .41]) whereas there were no significant correlations for mixed diagnoses, drug use or child abuse trauma studies. It should be noted that these analyses are based on only 2-4 studies per problem type and so are hardly an exhaustive investigation of the relationship within these problem areas, let alone the breadth of problems for which clients present. Outcome-competence correlations were also higher when the study did not control for working alliance. Once working alliance was accounted for, competence was not related to outcome. One interpretation of these findings is that when competence improved therapy outcome in these studies it did so via the key aspects of working alliance: agreement on tasks and goals, and therapeutic bond.

The second key meta analyses was of psychosocial interventions for children and adolescents by Collyer and colleagues (2020) which identified 9 effect sizes for competence and 5 effect sizes of a composite competence-adherence measure from 35 studies described in 52 papers. The studies did not include interventions delivered by teachers, peers or unqualified school professionals.
The results are somewhat similar to those for adults: the weighted average correlation between competence and outcome was not significantly different from 0: r  = .026 [-.020, .073]. The results differ from adults in that there was no significant variability in the effect sizes; the lack of relationship is very consistent. The types of problems represented were quite diverse: ADHD, substance use, behavioural problems and anxiety disorder. Similarly, treatment approaches included community reinforcement approach, individual CBT with parent sessions, family-focused therapy and single session motivational interviewing. Nevertheless, the lack of association between competency and outcome remained.

A common and fair explanation for the lack of association between competence and outcome in RCTs is that RCTs successfully restrict the variance in competence through training and supervising therapists carefully to ensure they do not deliver low quality therapy. Arguably, it is in the public interest to begin investigating the competency-outcome relationship in more routine, real-world settings. However, before attempting to overcome the obstacles to therapists submitting to have their competence externally evaluated, we need valid measures of therapist competence that are clearly associated with therapy outcome. In furthering this pursuit, Ottman and colleagues provided their original contribution by reviewing existing studies of competency assessment tools.

Of 21 studies, 8 studies found a positive relationship between competence and client outcomes, 5 studies provided mixed support in that competence correlated with some client outcomes but not others, and 5 studies found no relation. The areas for which there is a measure of competence with demonstrated ability to predict improved client outcomes are listed in Box 1 below.

Despite its self-evident importance, the only technology we have for measuring competence which has any demonstrated relationship with client outcomes involves review of session recordings by independent experts. This is an expensive and sometimes inefficient technology; single session ratings often don’t provide enough context to judge the therapist’s performance across the course of therapy, a single session may not be representative of the course, and the rating scales have been criticised for being either too complex to use with validity or too coarse to be sufficiently sensitive. Ottman and colleagues conclude their review by calling for more research into measuring competence via standardized role plays with actors, and for more research into psychotherapeutic competence, generally.

Problems and therapy protocols for which there has been a competence measure demonstrated to predict changes in client outcomes

  • Community reinforcement approach for adolescent substance use disorders

  • Oregon model of parent management training

  • Motivational interviewing

  • Cognitive therapy for social anxiety disorder

  • Schema therapy for personality disorders

  • Mentalization therapy for borderline personality disorder with comorbid substance dependence

Partial support

  • Cognitive therapy for depression

  • Manual assisted cognitive therapy for prevention of parasuicide

  • Developmentally adapted cognitive processing therapy for adolescents with PTSD

  • Client reprocessing of child abuse memories

Clinical Implications

  • Many conscientious clinicians worry about their competence. Part of what makes psychotherapy difficult is that there really isn’t adequate technology to objectively give you feedback that your performance is good enough. In lieu of this, I recommend:

    • Supervision with someone you trust who has the expertise and humility (doesn’t get off on the supervisor-supervisee power imbalance) to both recognise when you are doing as well as can be done and the honesty to offer you guidance to course-correct when they see technical deficiencies that actually matter. Your supervisor should be willing to actually watch you work and give you specific feedback, not just theorise. You need to believe them, both when they ask you to change, and when they tell you that the approach you are taking is likely to be as effective as any other we know about.

    • If you are working with one of the problems and approaches for which a competency rating scale has evidence of predictive validity, record your sessions and mark yourself against the rating scale.

    • Resist the urge to change horses mid-stream: it’s unlikely that learning a new therapy will unleash large improvements in most of your clients unless you are entering a new specialist service where you are likely to see large numbers of clients with the same kind of problem for which there is a well-supported specific manualised approach. Constantly being on a learning curve of a new therapy means operating for long periods of time below your optimum level of competence. (An exception to this suggestion is if you work primarily with a population for which there are no really powerful evidence-based therapies, in which case, you may always be operating on the learning curve of scientific knowledge!)

    • Concentrate your efforts to be competent on the aspects of working alliance that are well-established and common across presentations: seeking agreement on goals for therapy, tasks for therapy and minimising discord and ruptures in the therapeutic bond.

    • Know that there is an independence between competence and outcome once therapists are ‘competent enough’; client outcomes are subject to a myriad of forces outside your control and do not always reflect how well you are doing your job.

  • I know none of my readers are low on conscientiousness. But say you were, do not interpret our current meta-analyses as evidence that competence isn’t important. Just because we haven’t figured out a good way to measure psychotherapy competence or done many studies examining its importance (absence of evidence) isn’t evidence of absence (of relationship).


One final caveat: this review was of competence in outpatient psychotherapy. There are many other competencies involved in being a psychologist or other mental health professional. Although our technology for objective indicators of quality psychotherapy performance are poor, other competencies like adequate knowledge of psychological disorders and laws governing mental health practice can be measured with far greater reliability and objectivity. Therefore, clinicians should be clear as to how much they know about the difficulties their client presents with, decide whether working with the client’s problems is within their field of expertise and if not, either gain additional training and work the case under supervision or refer to someone with the expertise.

For the original article, go to:
https://www.sciencedirect.com/science/article/pii/S0005796719302177

REFERENCES
Collyer, H., Eisler, I., & Woolgar, M. (2020). Systematic literature review and meta-analysis of the relationship between adherence, competence and outcome in psychotherapy for children and adolescents. European Child and Adolescent Psychiatry, 29: 417 - 431.

Ottman, K.E., Kohrt, B.A., Pedersen, G.A., & Schafer, A. (2020, In press). Use of role plays to assess therapist competence and its association with client outcomes in psychological interventions: A scoping review and competency research agenda. Behaviour Research and Therapy, 0: 1-12.

Webb, C.A., DeRubeis, R.J., & Barber, J.P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78: 200-211.

Matthew Smout