A recent cohort study carried out among outpatients in a mental health care setting showed that outpatients suffering from an anxiety disorder whose treatment adhered to the available clinical-practice guidelines had greater symptom reduction after one year, compared to patients whose treatment did not adhere to these guidelines [1]. However, some patients remain non-responsive to treatment, even when their treatment was found to adhere to the guidelines and despite the fact that they were able to receive multiple recommended evidencebased treatments. The ability to identify such patients before treatment begins is an important challenge in clinical practice. The ability to do so could lead to improvement of the guidelines at a more individualized level, which would directly benefit patients.

Recently [2] provided an overview of what is known about factors that influence non-adherence and non-response in anxiety disorder patients receiving either antidepressant medication or cognitivebehavioural interventions. Based on the existing literature, they suggest that factors relevant to predicting treatment outcome include: low treatment motivation, hidden secondary motives for seeking treatment, encountering barriers that hamper treatment attendance (e.g. transportation problems or difficulties arranging for childcare), pre-treatment symptom severity and the presence and severity of possible comorbid psychopathology [2]. Many of these factors have been investigated as part of intervention studies examining the effects of monotherapies. In these studies patients were randomly allocated to an experimental condition or a control condition. Therefore patients lacked the opportunity to select a treatment of their own choice. This may adversely affect the generalizability of the findings to daily practice. None of the studies mentioned by Taylor et al. [2] investigated the influence of all of the above-mentioned factors, in concert, on treatment outcome. Thus, the question of what the predictive power of these prognostic factors, alone or in combination, would be on the treatment outcome for the patient is still operative.

A systematic review of the prognostic factors of long term disability in mental disorders performed by Cornelius et al. [3] sheds light on some additional factors. In this review strong evidence was found for age as a relevant factor for continuous disability. Limited evidence was found for gender, education, unemployment, and socioeconomic status in general. Also, a patient’s cultural background should be considered an additional putative factor predicting non-response and continuous disability, especially because of established higher drop-out rates from general mental health treatment for ethnic minorities in the Netherlands [4].

The aim of the present study is to investigate the predictive value of the factors described above in predicting non-response and longterm disability. In the present study these factors will be explicitly studied in conjunction with one another. The context of this study is an outpatient clinical setting where treatment is optimized according to the available evidence-based treatment guidelines, and where patients were encouraged to choose their own preferred methods of treatment.

Study participants and procedure

The present study used data collected as part of a study that investigated the feasibility and effectiveness of adhering to clinicalpractice guidelines for anxiety disorders in secondary mental health care [1,5]. This study was approved by the medical ethics committee of the VU University Medical Centre Amsterdam. Detailed information about study design and measurement procedures can be found in the two aforementioned references. A general description of the relevant research procedures is given here.

A cohort for the study was formed of patients who were registered at the community mental health care centre in Almelo, the Netherlands, after the implementation of the Dutch multidisciplinary practice guidelines for anxiety disorders was begun [1,5]. For the present study we included patients aged 18 years or older who i) were diagnosed with a primary DSM-IV diagnosis of panic disorder with/without agoraphobia, social phobia, obsessive compulsive disorder (OCD), generalized anxiety disorder (GAD), posttraumatic stress disorder (PTSD), specific phobia or hypochondriasis; ii) gave written informed consent for participation, and iii) received treatment according to the Dutch anxiety disorder guidelines recommendations (see: www. ggzrichtlijnen.nl). Co-morbidity with other mental disorders was allowed.

In order to maintain representativeness, only two exclusion criteria were used for the current study: (1) a primary clinical diagnosis of a psychiatric disorder other than one of the anxiety disorders described above; and (2) not being fluent in Dutch since language difficulties would harm the validity and reliability of the data collected.

Investigated practice guidelines and measure of adequate guideline adherence

The Dutch multidisciplinary guidelines for anxiety disorders contain recommendations for both psychotherapy (mainly cognitive behavioural therapy, but also EMDR for PTSD) and pharmacotherapy (mainly selective serotonin reuptake inhibitors or serotonergic tricyclic antidepressants). These options are counted as equally valid.

A review of each participating patient’s medical file was used to establish whether treatment had actually been delivered according to the guideline algorithms, originally yielding a proportion of patients who had been receiving recommended care that was classified according to the following labels: “adherent”, “non-adherent” and “inapplicable” [1]. The medical files were reviewed by specially developed process indicators. A checklist was used to score the different indicators [5]. Ultimately, if an algorithm was followed correctly and all the necessary steps in the treatment had been taken, the case would receive the label “adherent.” If a single necessary treatment step in the algorithm had not been properly applied because of a failure on the part of the responsible health care provider, the case would receive the label “non-adherent.” If none of the treatment steps appeared to be applicable, the case would receive the label ‘inapplicable’ overall. For the present study only the ‘adherent’ cases were included for further analyses [1]. Measurements relevant to treatment outcome were performed at baseline and at a 1-year follow-up.

Primary outcome measures

We sought to construct two models to predict non-response with respect to clinical symptoms and to functional limitations. (i) Clinical symptoms were measured with the Symptom Checklist (SCL90-R) [6]. A patient’s overall score on the SCL90-R reflects his or her general level of psychopathology (range 0–360). Non-response was operationalized as not showing reliable change on the SCL90-R total score from baseline to 1-year follow-up, according to the Reliable Change Index (RCI) criteria defined by Jacobson and Truax [7]. Applied to the SCL90-R total score, this means that reliable change on the SCL90-R is indicated by a score of at least 30 points. Thus, all cases with a change score of less than 30 points were defined as nonresponders. (ii) To assess functional impairments at baseline and at the 1-year follow-up the Sheehan Disability Scale (SDS) [8] was used. This patient-rated measure asks the subject to rate on a scale ranging from 1 (“not at all”) to 10 (“extremely”): 1) the extent to which symptoms have disrupted work/school work; 2) the extent to which symptoms have disrupted social life/leisure activities, and 3) the extent to which symptoms have disrupted family life/home responsibilities. The sum of the scores on these three subscales yields the SDS total score (range 3 to 30), and provides a general impression of the level of functional impairment experienced by the patient. The persistence of functional impairment was also defined dichotomously. A patient with an SDS total score of 6 or higher at the 1-year follow-up was considered to suffer from persistent functional impairments. In the literature on anxiety disorders an SDS total score of 5 or less has been used to signal functional recovery [9,10].

Predictors

Demographic variables

The patients’ gender and age were derived from the medical files. Patients were asked to report their country of birth and also the birth countries of both of their parents, their educational level, employment status and monthly net income. In the present study a patient is considered to have a foreign background if at least one of his or her parents was born outside the Netherlands or its former colonies. Ultimately, the patients’ educational level was operationalized dichotomously, as having completed only primary education (yes/no). This where the level of education in the cohort of patients studied was already relatively low on average. It was hypothesized that reaching adequate treatment effect when adhering to the guidelines, would be especially challenging in patients that only finished elementary school at best.

Assessment of DSM-IV axis I disorders

The presence of a DSM-IV axis I disorder was assessed by the Mini International Neuropsychiatric Interview (M.I.N.I.) [11,12], administered by a trained clinician as part of the standardized intake procedure at the community mental health care center. In patients with co-morbid mental disorders the primary diagnosis was defined as the psychiatric disorder associated by the patient with the greatest degree of suffering. In determining the influence on treatment outcome and persistent functional impairments, the presence of a comorbid secondary anxiety disorder, and the influence of a comorbid depressive disorder were separately investigated.

Psychiatric status variables

The Personality Diagnostic Questionnaire-4+ was used as a screener for the presence of comorbid DSM-IV axis II personality disorders, at baseline [13,14]. The PDQ4+ is a self-administered, true/false screening questionnaire. The PDQ4+ total score, the sum of the scores of the individual items, can be used as an index of overall personality disturbance (range 0-99), with a total score of 30 or higher indicating a substantial likelihood that the subject has a significant personality disturbance [15]. This 30 point cut-off value was used as a dichotomous measure for the absence or presence of personality disturbance.

To assess patient motivation the Nijmegen Motivation List 2 (NML2) was administered at baseline [16]. Only the preparedness subscale (range: 10 to 60) of the NML2, which consists of 10 items that express the patient’s preparedness to actively invest in treatment and to make sacrifices for the sake of treatment, was investigated for its predictive value on treatment outcome. Research has shown this NML2 subscale to be the most significantly related to treatment outcome in outpatient mental health care of the subscales in the NML2 [16].

Secondary gain was operationalized in accordance with the DSMIV definition of “obtaining external benefit” (APA, 1994; p. 453). Patients were explicitly asked to indicate whether special support and mediation by therapists was expected (yes/no) and the aspects for which they expected support and mediation. Patients could tick the following items, where appropriate: job, social security claims, budget for getting help from relatives, financial problems, compensation for unusual healthcare costs, legal/police matters, accommodation, insurance, other.

To assess practical barriers that could hamper treatment attendance the item from the Dutch version of the World Health Organization Quality Of Life questionnaire (WHOQOL-BREF) [17] was used and administered at baseline. This item asks the patient to rate on a scale ranging from 1 (“very dissatisfied”) to 7 (“very satisfied”), their degree of satisfaction with the accessibility of the health services. In order to facilitate interpretation, ultimately this variable was also dichotomized. Scores ranging from 1 to 4 were recoded to 1, indicating that the patient was less than satisfied with the accessibility of the healthcare services. Scores 5 to 7 were recoded to 0, indicating that the patient was satisfied or very satisfied with the accessibility of the health services used.

Standard descriptive statistics were used to describe the study population. To check whether attrition bias posed a threat to the validity of the study results differences between study completers and study drop-outs, t-tests for continuous variables and χ2 tests for proportions were used on the original (i.e. non-imputed) data. Ultimately, missing data resulting from patients dropping out of the study were handled by using multiple imputation by chained equations (MICE), which operates under the assumption that given the variables used in the imputation procedure, the missing data are Missing At Random (MAR). There were three sets of variables selected for the imputation model [18]: all variables that appeared in the complete data model, all variables that related to dropping out of the study, and all variables that related to the two main outcome variables (the severity of psychiatric symptoms (SCL90-R) at follow-up measurement and the score for functional impairment (the Sheehan Disability Scale). Data imputations were performed with chained equations method in Stata 12.1 using predictive mean matching (PMM) as the imputation method with 20 imputations.

All final analyses were carried out on the multiple imputed data set, using all cases originally included in the study that were judged to have been adequately treated according to the guideline algorithms (guideline adherent; n=81). In constructing a prediction model for “non-response” and “functional impairment” we used a modelbuilding strategy called “purposeful selection of covariates” [19]. After a careful bivariate analysis of each independent variable, retaining any variable whose bivariate test has a significance level below 0.25 (step 1), a multivariate model is fitted (step 2) retaining all variables with significance levels below 0.10, checking that none of the coefficients changed markedly in magnitude (i.e. Δβ>20%) and re-entering a predictor if necessary in order to prevent such a change (step 3), and finally evaluating the possible addition of each independent variable individually (step 4). Non-linearity (step 5) and interaction effects (step 6) are evaluated, and ultimately the “final model” obtained is evaluated in terms of model fit by pseudo R-square and “area under the receiver operating characteristic (ROC) curve” (step 7) The basic statistical calculations in this study were performed with SPSS, version 17.0 (SPSS Inc., Chicago IL, US). The multiple imputations and analyses using logistic regression on the imputed data were performed using Stata 12.1 (Stata Corp., College Station TX, US).

As can be seen from the flowchart (Figure 1), 81 patients from the group initially included in the study were recommended for treatment according to the treatment guidelines and were judged to have received such treatment (guideline adherent cases). The attrition rate of patients in this group at the 1-year follow-up was 16%. Tables 1 and 2 present demographic and clinical characteristics, respectively, of the 81 adherent cases and characteristics of the study completers vs. patients lost to follow-up.

Figure 1: Flow chart for patient inclusion.

Table 1: General characteristics of the study sample of completers and patients lost to follow-up (original data).

Table 2: Clinical characteristics of the study sample of completers and patients lost to follow-up (original data)

As shown in Tables 1 and 2, the proportion of males was higher in the group of patients who were lost to follow-up compared with the study completers. Furthermore, patients lost to follow-up were less educated than study completers and were less motivated to complete treatment. Moreover, somewhat more patients lost to follow-up were diagnosed with a social phobia.

At the 1-year follow-up the estimated overall percentage of nonresponders on the SCL90-R among patients whose treatment adhered to the guidelines was 40%, while 63% experienced persistent functional impairments.

Table 3 depicts the results of the bivariate analyses as a sub-step of the study analyses and the final results of the procedure using the multivariate logistic regression analyses, performed with non-response as the dependent variable.

Table 3: Predictors of non-response on the SCL90R (change score less than 30 points), with estimated values based on the imputed dataset (n=81).

As shown in Table 3, only the “gender” and “secondary gain” variables were associated with non-response measured with the SCL90-R. Patients that at baseline

reported hopes to obtain external benefits by seeking treatment, appear to have a lower chance of showing non-response to treatment as measured at 1-year follow-up when looking at the final results of the multivariate regression analyses (p=.054). This while males tend to have a higher chance of showing non-response (p=.074). No interaction effects were found.

However, the fit statistics for this model are low: Mc Fadden’s pseudo R-square equals 0.07 and ROC=0.66, which indicates that the discrimination between responders and non-responders based on this model is rather poor. Table 4 presents the results of the bivariate and multivariate analyses with persistent functional impairments measured with the SDS as the dependent variable.

Table 4: Predictors of persistent functional impairments (a total score of 6 or higher on the Sheehan Disability Scale at 1-year follow-up), with estimated values based on the imputed dataset (n=81).

As shown in Table 4, the variables of “satisfaction with accessibility of health services”, “gender” and “presence of comorbid anxiety disorder” were associated with persistent functional impairments. The results show that when comparing patients that express being less than content with the accessibility of care, with patients that express being content or even more satisfied with the accessibility, the first group has a higher chance of showing persisting functional impairments at 1-year follow-up. A result that is highly significant (p=.008). Also, males tend to have a higher chance of persisting functional impairments as measured at 1-year follow-up (p=.087). Surprisingly, compared to patients without such comorbid condition, patients who were at baseline found to suffer from another comorbid anxiety disorder appear to have a lower chance of persisting functional impairments (p=.079). Again, no interaction effects were found. The fit statistics for this model, predicting functional impairment based on these three predictor variables, are quite good: Mc Fadden’s pseudo R-square equals 0.30 and ROC=0.82, indicating an excellent degree of discrimination between patients with and without persistent functional impairments as measured at the 1-year follow-up.

To further study the results with respect to the predictive value of “gender”; “secondary gain” and “presence of a comorbid anxiety disorder”, we compared baseline scores on the SCL90-R of the predictor variables. It was found that males scored significantly lower on the SCL90-R at baseline than females (mean total score= 173.56 versus mean total score= 203.60; t=-2.52, df=78.88, p=.014). Also, patients who at baseline reported their hope of obtaining external benefits by seeking treatment scored significantly higher on the SCL90-R at baseline, compared to the patients who did not (mean= 211.62 vs. mean= 179.38; z=-2.39, p=.017; calculations based on the imputed data). Baseline severity scores were found to be significantly related to symptom severity at the 1-year follow-up (r= .479, p<.001). Differences in baseline symptom severity thus might be a relevant confounding factor when studying the relationship between gender, secondary gain and treatment (non-)response.

Additional analyses of the results on functional impairment did not reveal differences on the SDS on the predictor variables of “gender”, “comorbid anxiety disorder” and “accessibility of health care services”, suggesting no confounding due to baseline differences.

This practice-based study examined the predictive value of variables that are known to be able to influence treatment prognosis in anxiety disorder patients. These variables were examined in concert in a sample of 81 patients treated in a specialized mental health care setting who were judged to have been adequately treated according to evidence-based guidelines. We tried to develop a prediction model that allowed the identification of patients who run the risk of showing treatment non-response or who will continue to experience functional impairments when being provided guideline concordant care.

Of the demographic variables considered (age, gender, foreign background, educational level, employment status and income) only gender was identified as a potentially relevant predictor variable for both treatment non-response and persistent functional impairments. Males had a higher chance of showing treatment non-response and also a higher chance of having persistent functional impairments at the 1-year follow-up. However, males were found to have lower baseline symptom severity scores than females which might explain the results pertaining to treatment non-response. The data suggest that gender as a predictor of persistent functional impairments may have something to do with a subgroup of males quitting treatment prematurely. Of the patient characteristics, hope of secondary gain and the presence of a comorbid anxiety disorder were identified as being potentially relevant to treatment prognosis.

Contrary to what was expected from prior research [20-22], patients who reported a hope of gaining external benefits actually had a lower chance of showing treatment non-response. These patients were found to suffer from more severe general psychopathology at the baseline. It may be that the reported motives for secondary gain might reflect a genuine need for help in managing life. This suggests that as long as additional support is available in addition to the regular evidence-based care for anxiety disorders, for example the availability of a community psychiatric nurse, greater treatment effects can be achieved in these patients. Patients who suffered from a comorbid anxiety disorder at baseline were also found to have a lower likelihood of persistent functional impairments. On the other hand, patients who were less than satisfied with the accessibility of health care services had a substantially greater risk of persistent functional impairments. Actually, this was the only study result that was significant when maintaining the orthodox significance level of 5 percent and therefore probably the most relevant result.

There are some limitations to the results of the current study. One of the most important issues is the relatively small sample size used in this study. In addition, because of the small sample size and the explorative nature of our study we maintained a liberal significance level of .10 for the selection of predictors. This makes replication of our study necessary. The small sample size may explain why the results for some of the predictors of treatment prognosis identified as relevant in other studies failed to reach significance in the current study. While this may be true, the size of the estimated Odds Ratio’s in our study does suggest however that the influence of these variables is probably not very strong. This is likely to be especially true in a situation where patients get to choose among recommended evidence-based treatment options and can also receive multiple evidence-based treatments at the same time, as is the case in everyday clinical practice, as opposed to the more controlled circumstances of most predictor studies. Another limitation of the current study might be the fact that the treatment results were measured only at the 1-year follow-up. Some of the variables studied may have predictive value when studied over a shorter time period. The one-year time period for measuring treatment results was chosen in this prediction study on guideline concordant care because one year is the minimum length of time for the main recommended treatment steps to be put in effect for most anxiety disorders, in order to determine if there has been a sufficient treatment response.

There are several strengths to the present study. The patients included in the study sample are representative of “real world” mental health care. There were almost no exclusion criteria. To our knowledge the study presented here is also the first to look at the conglomerate of possible predictors of treatment response that have been identified as relevant to predicting treatment prognosis (Taylor et al., 2012; Cornelius et al., 2011) at the same time. This was done in a setting where patients were able to choose among several available evidencebased treatment options and were also able to receive a combination of treatments when the clinician adhered to treatment guidelines for anxiety disorders.

From the results of the present study it can be concluded that it is hard to predict which anxiety disorder patients will or will not profit from guideline-adherent treatment in terms of demonstrating adequate treatment response. Unfortunately, this means that for the time being selecting an effective cure for the individual patient will for the most part remain a process of trial and error.

However, we may also conclude that with the knowledge currently available no one suffering from an anxiety disorder as a primary diagnosis should be prevented from being offered some form of evidence-based care according to the available evidence-based treatment guidelines, if one adapts to gender and removes barriers in attending treatment.

This study was carried out by the Dimence Institute of Mental Health, Deventer, the Netherlands. It was supported by the Netherlands Organisation for Health Research and Development (ZonMw) (OOG; 100-002-022). The authors thank all of the patients and staff of Dimence who were involved in the study.