Depression is prevalent among individuals living with HIV, with evidence suggesting that more than one third of people with HIV/AIDS may have mood disorders or clinically significant depressive symptoms [1-6]. A meta-analysis comparing rates of depression disorder in HIVpositive and at risk HIV- negative patients demonstrated a twofold increase in the prevalence of major depression in patients infected with HIV [3].

Sub-Saharan Africa (SSA) is home to over two thirds of all patients living with HIV, with AIDS being the leading cause of death [7]. It is estimated that about 1.6 million people in Uganda are living with HIV/ AIDS with an HIV prevalence of 7.3%. There is a dearth of literature on depression among individuals living with HIV/AIDS in sub-Saharan Africa. The sparse literature shows elevated rates of depression among HIV-infected individuals relative to community samples [5,8-16] that is consistent with the developed world [14-17].

The social, economic and health impact of depression in sub- Saharan Africa is great. Depression is associated with mortality [18-21], work disability [20,22-24], lower quality of life [19,25-29], risk of heart disease [30] and high-risk behaviors for contracting HIV infection [31]. Depression is associated with lowered adherence to antiretroviral medication [32,33].

Depression in sub-Saharan Africa presents in forms (culture specific idioms, somatic, based on interpersonal relationships or spiritual in nature) that may obscure detection [34-36]. However, depression exists at possibly higher prevalence rates than in western countries [35] according to several studies [19-22,26-29,36-44] of community and non-HIV specific clinic populations, with generally higher rates for women than men [23,40]. Finally, though depression may present differently in sub-Saharan Africa compared to western countries, it [depression] is reasonably easy to elicit when present and sought [23].

Depression in SSA has been measured using standard tools developed in western countries like the Beck Depression Inventory (BDI) [21,32,40,45,46]; Hopkins Symptom Checklist (HSCL) [18,22,38,40,44,47]; the Center for Epidemiologic Studies Depression Scale (CES-D) [31,39]; the Edinburgh Postanatal Depression Scale (EPDS) [36,42]; the Patient Health Questionnaire (PHQ) [5,38]; the World Mental Health Survey of WHO Composite International Diagnostic Interview (WMH-CIDI) [19] and the Composite International Diagnostic Interview (CIDI) [20]. There have been few attempts to validate these instruments in this culturally different setting [5,27,38-40,46] and to adapt the instrument to site specific culture and terminology [44]. While some may argue that instruments developed in one culture can be used in another provided careful attention is paid to conceptual translation [47], this has not always been done when using these standard instruments. The most common practice is to do a direct translation of the instrument from its original language to the local language [20,31]. This kind of translation does not always ensure conceptual translation and may even sometimes change the meaning of the concept under translation.

An important limitation of most standard instruments or diagnostic measures is that they may be culturally inappropriate unless proved otherwise [34]. Descriptions of certain health conditions like mental illness developed in one set of cultures may not be equally applicable to other cultures. Given this assumption, if the nature of emotions, thoughts, and behaviors can be expected to vary by culture [48], uncritical application of standard instruments cross-culturally might yield misleading or erroneous results where such instruments are locally inappropriate [49]. Inconsistent findings in any research effort may result from random processes and non-equivalent measures, procedures, or samples, but may also be explained by problems of low validity. Problems of validity are not new to epidemiology [50], but are more likely to occur in transcultural epidemiology which is defined here as research in which the views, concepts or measures of the investigator extend beyond the scope of one cultural unit to another [51].

Context affects research validity. Evidence of measurement validity and reliability cannot be assumed to generalise across populations. This lack of generalisability may be especially problematic when the original measure is translated into another language, as is common in transcultural studies. Creating a culturally acceptable, comprehensible, relevant and semantically equivalent translation is difficult [52] making it essential to study the psychometric properties translated measures that might have changed during imperfect translations. While we acknowledge the limitations of these ‘foreign’ generated instruments, we do not say that these instruments are irrelevant for studying health conditions in other contexts but rather that their appropriateness will vary across cultures and by illness, and therefore needs to be explored. As such, instruments may need to be adapted to the new situations or in other cases completely new instruments developed.

The BDI is one of the most widely used instruments for assessing depressive symptom severity. Not only is it used for assessing the intensity of depression in psychiatrically diagnosed patients [53], but also for detecting depression in nonpsychiatric patients with conditions like HIV/AIDS, diabetes and other chronic conditions [32,40,45,54,55]. It has also been used to detect depressive symptoms in normal populations.

The BDI has been used to assess depressive symptoms in sub- Saharan Africa [39,40,45,56]. However, the psychometric properties of the BDI have not been well studied in SSA especially among HIV infected patients.

Classical Test Theory (CTT) has been the underpinning for most test construction and theory. Item response theory (IRT) is a new approach to test development [57,58]. It is an improvement of the CTT approach. The two main achievements of IRT is the ability to separately estimate parameters about the test items and the people taking the test at the same time under the same scale. CTT focuses primarily on testlevel information, whereas IRT focuses on item-level information.

Using IRT, we set out to characterize the BDI in a low income HIV-infected population taking antiretroviral medications. It is particularly important to characterize depression in this population because depression is associated with decreased adherence and disease progression.

Study design

This analysis is based on data collected from a cross-sectional survey of 123 adults attending the Mother To-Child-Transmission Plus (MTCT Plus) Program at Mulago Hospital in Kampala, Uganda.

Study setting

The study setting has been described in detail elsewhere [32] Briefly, Mulago Hospital is Uganda’s largest teaching, referral, and research hospital. The MTCT Plus program is located at Mulago Hospital under the Makerere University-Johns Hopkins University Research Collaboration.

Study procedures

Details of recruitment and data collection have been reported elsewhere [32]. Briefly, we recruited individuals who were newly initiating ART and individuals who were on chronic ART. In addition to the other data collected, the research assistants administered the BDI-II at the beginning of the study to detect depressive symptoms. The instrument took about 10 minutes to administer.

The instrument

The BDI-II is a 21-Guttman item test presented in multiple choice format which purports to measure presence and degree of depression in adolescents and adults. The BDI was derived from clinical observations about the attitudes and symptoms displayed frequently by depressed psychiatric patients and infrequently by nondepressed psychiatric patients [59]. The clinical observations were consolidated systematically into 21 symptoms and attitudes which could be rated from 0 to 3 in terms of intensity. The items were chosen to assess the intensity of depression and were not selected to reflect a particular theory of depression. The 21 symptoms and attitudes were: (1) Mood, (2) Pessimism, (3) Sense of Failure, (4) Lack of Satisfaction, (5) Guilt Feelings, (6) Sense of Punishment, (7) Self-dislike, (8) Self-accusation, (9) Suicidal Wishes, (10) Crying, (11) Irritability, (12) Social Withdrawal, (13) Indecisiveness, (14) Distortion of Body Image, (15) Work Inhibition, (16) Sleep Disturbance, (17) Fatigability, (18) Loss of Appetite, (19) Weight Loss, (20) Somatic Preoccupation, and (21) Loss of Libido.

Analyses

We conducted univariate analyses of the baseline characteristics of our study population, including socio-demographic and clinical variables. Categorical variables are presented as frequencies and percents while continuous variables are presented as medians and interquartile ranges [IQRs]. Depression responses for each subject were scored by summing the ratings given to each of the 21 items. The depression scores were then categorized into four categories according to a recommended guideline: none or minimal depression, <10; mild to moderate depression, 10-18; moderate to severe depression, 19-29; and severe depression, 30-63.

Classical test theory (CTT): Item and test characteristics were first evaluated using CTT analysis. Item parameters included the item mean (item difficulty) and point biserial correlations. According to Varma (Varma, 2009), a point-biserial value of at least 0.15 is recommended. The internal consistency reliability of the test was calculated using Cronbach’s alpha and adequate reliability was demonstrated with a reliability index of at least 0.70 [60]. The CTT analyses were performed using Conquest software [61].

Item response modeling: We used an item response modeling approach to calibrate the data [58]. Preference of IRT over CTT has been a result of several limitations of CTT [62]. The primary limitation of CTT is that the item and scale statistics apply only to the specific group of subjects who took the test. That means that if the scale is to be administered to people who are different in some way, such as being a member of a minority group or patients rather than students, then it is necessary to re-establish its psychometric properties and perhaps develop new norms. Similarly we would have to go through the same renorming process if any of the items were altered, or if items were deleted in order to develop a shorted version of the scale. The second problem is that it is impossible to separate out the properties of a test from the attributes of the people taking it. Thus the instrument’s characteristics change as we test different groups. A third problem is the assumption that each item contributes equally to the final score. While this is also true for IRT, the important difference is that we test to see if this is acceptable, that is we test for fit of the items. In CTT, the items are simply summed up irrespective of how much each item correlates with the underlying construct. Fourth, it is also assumed that each item is measured on the same interval scale. This assumption often fails on two grounds: items are most often ordinal rather than interval, and the ‘psychological distance’ between response options differs from one item to the next [63]. IRT has been designed to overcome these limitations [64]. One advantage of IRT is that the scale that emerges from an IRT analysis truly has interval-level properties. Secondly, it provides a more precise estimate of measurement error. Third, when the model fits, it is invariant theoretically: that is, that item characteristics are independent of the sample from which they are derived.

Analysis was done using Conquest [61] and ConstructMap [65] software. These two soft wares are based on Rasch’s family of logistic models. Two models for ordinal data were considered: the rating scale model (RCM) [66] and the partial credit model (PCM) [67]. The PCM does not assume that the distances between ordinal responses are the same for all items (i.e the distance between response options a and b, b and c, etc. is not the same across all items). In contrast, the RCM assumes that the distances between ordinal responses are the same for all items (i.e. the distance between response options a and b, and b and c, etc is the same for all items). The best fitting model was assessed by comparing the deviance parameters as well as the weighted fit indices for the items and the respondents (i.e. infit statistics). Item misfit was established by infit mean square (MSQ) values outside the range of 0.75 to 1.33, with significant t values [58]. For the weighted t statistic, any value <-2.00 or > 2.00 was indicative of misfit. The likelihood ratio test (LRT) and MSQ were used to determine the best fitting model.

Using the estimated parameters, a Wright map was constructed. The Wright map is an empirical map based on respondents’ self-reports [58]. The Wright map of person-item estimates displays the item locations and the extent to which the item locations reflected the full range of depressive symptoms. It is useful for identifying a set of items that span the range of the construct. This map served to qualitatively assess content representation by determining if the items assessed minimal, moderate or severe depression.

To investigate the reliability of the instrument, first the standard error of measurement for each person location was estimated. Using the standard error of measurement, the information for the whole instrument was calculated. The information for the whole instrument is the sum of the information for each item, where information for each item is the reciprocal of the square of the standard error of measurement. A quality-control index of consistency was developed. Internal consistency was assessed by calculating the reliability coefficient using the expected a posteriori (EAP) procedure.

Validity of the instrument was investigated using evidence based on instrument content, internal structure and relations to other variables [68]. These are the most recent standards for educational and psychological testing. To assess presence of internal structure, we investigated consistency of the items with the instrument as a whole using mean locations of each group. In addition, differential item functioning (DIF) was investigated to determine if items functioned in a similar way for respondents across important subgroups. Finally, we explored evidence based on relations to other external variables that the construct [depression] should predict. In particular, we explored the relationship between depression and adherence to antiretroviral medications.

Ethical considerations: All participants provided informed consent and all procedures were approved by the Faculty of Medicine Research and Ethics Committee and the Institutional Ethics Review Board of Makerere University, the Uganda National Council of Science and Technology, and the University of California, San Francisco Committee on Human Subjects.

One hundred twenty three patients were included in the analysis. The study population was predominantly female with low income. Patient characteristics are summarized in Table 1. Fifty six percent (n=63) scored 10 or less on the whole instrument Table 2.

Table 1: Patient characteristics.

Table 2: Distribution of depression scores.

Item response modeling

The Partial Credit Model was used for analysis. Compared to the rating scale model, the partial credit model fitted the data better as one would expect, given the nature of the alternative responses to the items. (X2=172.0582; df=24, p value<0.0001)

Item fit

Weighted fit statistics showed a good fit for most of the items. The weighted mean square range was 0.83 to 1.43. As an effect size, there is no absolute limit to what is a good means weighted square value, but previous researchers have indicated that 0.75 is a reasonable lower bound and 1.33 is a reasonable upper bound [58]. Item 11 (agitation) showed the worst fit. Infit mean squares are given in Figure 1.

Figure 1: Infit mean squares.

Wright map

The right-hand side of the map (Figure 2) shows the calibrated item threshold locations. The kth Thurstone threshold is the point at which the probability of the scores below k is equal to the probability of the scores k and above. Because there are four response categories, there can only be three possible threshold values: 1, 2 and 3. A threshold of 1 refers to the point at which response levels 1, 2, and 3 together become more likely than level 0; a threshold of 2 is the point at which levels 2, 3 and 4 together become more likely than levels 0 and 1 and so on. From the Wright map we can see that there is generally a good spread of the items over the construct “feelings of depression”. The items cover the lower, middle and higher end of the construct. However, from the map there are some gaps in the construct that are not well covered by the items. Specifically, we notice a lack of enough items that cover severe depression. There is a wide gap between item thresholds 14.1 and 11.2. Likewise there is need for more items between thresholds 14.1 and 4.3. On the lower end of the scale there is a gap between 3.1 and 11.1. The consequence of these gaps is that the instrument may not adequately discriminate among respondents up at the high end. However, given that the concern is not the exact level of depression but whether one is depressed or not given a cut off point, this may not be so much of a problem and the current items may suffice.

Figure 2: Wright map.

The locations of the bars of the histogram (i.e. the Xs) are the estimated locations of the respondents on the variable. The respondents tend to cluster in the lower half of the scale (below logit score 0) reflecting the fact that most patients had minimal to moderate depression (Table 3). There were few respondents at the extremes. The mean depression score was 8.86 with a standard deviation of 5.44.

Table 3: Item point-biserial correlations for the BDI-II.

Evidence for Reliability

Standard Error of Measurement (SEM): When using an instrument on an individual respondent, the sem is the most important tool for assessing the usefulness of that estimate of location (Figure 3). If the sem is too large, the measurer will not be able to make intelligible interpretation of the results. For this sample, the mean of the sem was 0.569 with a standard deviation of 0.407. This gives a 95% confidence interval width of 1.596 logits. This is about 23% of the width of the entire Wright map from maximum to minimum locations. This implies that the instrument is not accurate for individual usage. To see this, let us consider the second threshold. The confidence interval spans (for the second threshold) the range from items 13 and 16 to items 4 and 20 covering a wide range of depression states indeed. Thus, this instrument is probably not very useful for accurate clinical diagnosis, but it may well be useful for initial screening or as a basis for group measures.

Test information: The uncertainty in an estimate of person location is usually characterized for individuals using the standard error of measurement. Derived from the standard error of measurement, the information curve for the BDI-II instrument is given in Figure 4. Together with Figure 3, these graphs show that the most sensitive part of the instrument is from approximately -1.5 to +1.5 logits. Comparing with the Wright map, this range contains thirty six out of the forty seven items thresholds (77%) observed in these data. Thus, the instrument’s range of maximum sensitivity makes general sense with respect to the items.

Figure 3: The standard error of measurement for the BDI-II instrument.

Figure 4: Test information curve.

Internal consistency coefficients: The Cronbach’s alpha was 0.77 but when only complete cases were used the Cronbach’s alpha coefficient was 0.79. The expected a posteriori (EAP) reliability was 0.86.

Evidence for validity: The structure of the discussion about validity below is based on the 1999 Standards for Educational and Psychological Testing [68].

Evidence based on instrument content: While the author of the original BDI [59] did not explicitly apply IRM procedures in developing this instrument, these procedures [IRM] were employed implicitly. IRM is a new approach to instrument development that was not in existence then. The first step in developing an instrument using IRM is to develop a construct map. A construct map is a content structure in which the relative behaviors of a generic respondent having various amounts of the construct are arranged in order on one side, and potential items are arranged in order of endorsability on the other. Beck et al designed the instrument with the intent of building in strong content. The content was based on systematic observations and records of the characteristic attitudes and symptoms of depressed patients [59]. He selected a group of these attitudes and symptoms that appeared to be specific for these depressed patients and which were consistent with the descriptions of depression contained in the psychiatric literature. On the basis of this procedure, he constructed an inventory composed of 21 categories of symptoms and attitudes. He arranged responses in each category using a Guttman-like scale that endeavors to cover the entire spectrum of depressive symptoms. This intention of structure is illustrated in the Wright map (Figure 2) that fairly covers the different levels of depressive symptoms. Based on these observations, we say that there is evidence of validity based on instrument content.

Evidence for validity based on response processes: Evidence based on the response process consists of studies of how respondents react to the items and the instrument as a whole. These might consist of ‘think alouds’, ‘exit interviews’ or ‘cognitive interviews’ with samples of respondents. Such evidence was not available with our secondary analysis of the data, so this aspect of the framework is not relevant at this time.

Evidence for validity based on internal structure: One major criterion for internal or construct validity is an a priori theoretically based hypothesis about the order of item endorsability, the ease with which respondents rate items strongly. The Wright map is a very good tool for investigating this [58]. No such expectations were made for the BDI. The authors clearly stated that “the items were chosen on the basis of their relationship to the overt behavioral manifestations of depression and do not reflect any theory regarding the etiology or the underlying psychological processes in depression’ [59]. This is also illustrated in the Wright map (Figure 2) where there is no discernible empirical order to the items of the BDI. Instead, the feature of the BDI-II items that maps out the BDI variable is the transitions between the response categories. One thing we would expect is a similar pattern of item locations at each threshold level but this is not the case. Hence this source of validity evidence is not available.

An important piece of evidence that an item is functioning as expected is that the increasing response levels of the item are operating consistently with the instrument as a whole. With the item response modeling approach, one way that this consistency can be made manifest is that respondents higher on the construct would, in general, also score higher on each item. In terms of the Wright map, one would consider the locations of the respondents within each score group for an item: if the mean location for each group tends to increase as the scores increase, it seems reasonable to say that this particular expectation that comes from the items design has been fulfilled. This analysis showed that for most items, the mean increases as the score rises (Table 4).

Table 4: Mean locations of respondents within each score group.

A standard requirement of the items design is that, across important subgroups, items function in a similar way for respondents who are at the same location--- that is they should exhibit no evidence of differential item functioning (DIF). DIF is, therefore, said to be present when respondents at the same location on the variable give different responses across different subgroups. For this analysis, we investigated the effect of CD4 cell count on item functioning. Respondents were divided into two subgroups: those with CD4 cell count below 250 cells/uL and those with CD4 cell count above 250 cells/uL. A cut off of 250 was used because this is the level around which an HIV-infected patient in this population is deemed to have AIDS and thus started on antiretroviral therapy. Overall, respondents with CD4 cell count levels below 250 were 0.5 logits higher on the depression trait. This does not show DIF but evidence of differential impact. This result is not surprising, as being diagnosed with HIV would usually be associated with negative feelings. Test results for items with DIF are shown in Table 5. In terms of DIF, a recommended standard of effect size is as follows: a logit difference value less than 0.426 is “negligible”, a value between 0.426 and 0.638 is “intermediate” and a value over 0.638 is “large” [58]. Overall, there was statistically significant DIF for the item set (Chi sq=52.02, df=20, p-value <0.0001). Applying these criteria to the statistically significant logit differences in Table 5 shows that items 5, 11, 13, 14, 15, and 17 had large DIF. The rest of the items had no DIF.

Table 5: Testing for DIF.

Evidence based on relations to other variables: Where there are other external variables that the construct should (according to theory) predict, a strong relationship between the instrument under scrutiny and these external variables can be used as validity evidence. In another paper from the same study (Byakika-Tusiime et al., 2009), we demonstrated that depression was significantly associated with adherence to antiretroviral therapy [OR=0.32, 95% CI 0.11-0.93].

Evidence based on consequences of using an instrument: The final form of validity evidence relates to consequences. We did not have any data on consequences of using this instrument hence our analysis did not evaluate this form of evidence.

There have been little data regarding the validity of the Beck Depression Inventory in an HIV-infected population despite the fact that the BDI is commonly used to assess symptoms of depression in this population.

The findings of this analysis showed that the BDI-II has good psychometric properties when used in a low income HIV-infected population in Uganda. These findings corroborate those found in other populations [69,70]. However to our knowledge, this is the first attempt to validate the BDI-II in an HIV-infected population in sub-Saharan Africa. It is also the first attempt to use item response modeling to calibrate this instrument. Previous calibrations were based on classical test theory and now IRM further confirms the robustness of the BDI-II instrument.

The weighted mean square indicates that most of the items are fitting within reasonable bounds. Thus, the overall finding is that the BDI-II data fit the Partial Credit Model reasonably well. The Wright map generally revealed a relatively good spread of the items over the construct “feelings of depression” although there were a few gaps identified in the instrument that may need bridging with more items or item responses. The biggest gap was observed at the top end of the instrument—that measures severe depression. The need to add more items or item responses for adequate discrimination of respondents at these levels would largely depend on the ultimate purpose of this instrument. If the purpose is to detect respondents with severe depression then it should probably be augmented with more items/ item responses at the upper end. On the other hand if the purpose is to screen for minor to moderate or moderate to severe depression, then the current items seem to suffice.

The majority of respondents in this sample had mild to moderate depression and the distribution of the respondents shows that many respondents in this sample were below -1.5 logits. However, the test information curve tells us that the most sensitive part of this instrument is the region of moderate to severe depression (-1.5 to +1.5 logits). This implies that the instrument is not functioning optimally for quite a large proportion of this sample. However, since the ultimate purpose of this instrument is to detect levels of depression that would require intervention (moderate to severe depression) then this is not a big problem. That is, the instrument captures respondents with moderate to severe depression, which is the main concern.

The findings of this analysis provide evidence for the validity of the instrument’s usage. Build of this evidence has been based on test content, internal structure, and relations to other variables. This is an alternative to the traditional way of assessing validity; that is, criterion, content and construct validity [68]. We have shown evidence based on instrument content, evidence based on internal structure and evidence based on relations to other variables.

We found significant differential item functioning with CD4 cell count subgroups. DIF is an undesirable characteristic of any test. Items 5, 11, 13, 14, 15, and 17 had large DIF. This finding should be taken with caution. There is need to establish that the DIF is indeed not a result of random fluctuations and would thus require doing repeated samplings to confirm this. If indeed DIF is established, the best strategy is to develop alternative items that do not exhibit DIF. Another less popular alternative is to use two different calibrations for the two groups. However together looking at all the items exhibiting DIF, those exhibiting positive DIF cancel out those with negative DIF. Hence overall we can say that there is no significant DIF effect.

Internal consistency coefficients in this sample were good. We found a Cronbach’s alpha of 0.79 and an EAP reliability coefficient of 0.86. This demonstrates low measurement error. The reliability coefficients obtained from this analysis are similar to what has been found elsewhere [69]. Within nonpsychiatric populations coefficient alphas of the range from 0.73 to 0.92 have been observed.

There were a few limitations to this analysis. First, the sample size was rather small. We would require a bigger sample size to confirm these findings. Secondly, the study sample was mainly female (75%). This may limit generalizability of our findings.

In conclusion, this analysis demonstrated good psychometric properties of the BDI-II when used to screen for depression in a low income predominantly female HIV-infected population. The data fitted the partial credit model reasonably well suggesting that the currently used scoring method of the items is reasonable. These findings therefore, support the use of the BDI-II in assessing depressive symptoms for HIVinfected patients in sub-Saharan Africa especially women. The quality of item calibration must be supervised continuously. We cannot expect the items in an instrument to retain their calibrations indefinitely or to work equally well for every person with whom they may be used.

We acknowledge all members of the research team of the parent study who recruited the study participants, collected and managed the data: Mary Kasango, Annet Kawuma, Muzamir Baryamushanga, Grace Manyangwa and Irene Zawedde. We also wish to thank the project administrator Sarah Nakandi and the driver Ibrahim Kiviri. Last but not least we thank all the patients in the MTCT+ program who participated in the parent study.

Funding was provided by the Bill and Melinda Gates Grant; the R0-1 MH54907, R21 AA015897, K24 AA015287 grants; and by the Fogarty AIDS International Training and Research Program/University California Berkeley (1D43 TW00003).