Journal of Psychology & Psychotherapy
Open Access

The Child and Adolescent Family Functioning Inventory (CAFFI):Development and Psychometric Properties

Paul A. Sunseri^{* *}Correspondence: Paul A. Sunseri, New Horizons Child & Family Institute, 5000 Windplay Dr., Suite 2, El Dorado Hills, California 95762, United States, Tel: 5306476458, Email:

Author info »

Abbreviations

CAFFI: Child and Adolescent Family Functioning Inventory; IIFT: Intensive In-Home Family Treatment; IOP: Individualized Outpatient Program; CFA: Confirmatory Factor Analysis; EFA: Exploratory Factor Analysis; RMSEA: Root Mean Square Error of Approximation; TLI: Tucker-Lewis Index; CI: Comparative Index.

Introduction

The quality of interpersonal relationships within families can precipitate, exacerbate, or buffer against childhood mental illness [1]. Family functioning can be defined as the ability to solve problems collaboratively, manage stress and conflict, achieve closeness and connection, and communicate effectively [2]. The ability to efficiently and accurately assess key family dynamics is an essential clinical tool in order to demonstrate if treatment is effective [3].

The link between how well a family functions and the severity of a child’s mental health condition has been established and improving family functioning through targeted clinical interventions results in an improvement in a child’s mental health condition. Studies have examined the relationship between how well a family functions and the severity of a child’s mental symptoms. Two large-scale studies, [4,5] found that the severity of a child’s symptoms, response to treatment, and other important treatment outcomes are highly associated with how well the child’s family functioned. In a review of 98 studies of predictors and moderators of childhood anxiety, baseline family functioning consistently predicted youth outcomes [6]. Post-treatment discharge environment (to a lower level of care or a higher one) is also related to baseline family functioning [7].

Seriously mentally ill children and adolescents with higher baseline family functioning experience more profound behavioral improvements over the course of treatment and require less intensive aftercare services [4]. Additionally, improvements in family functioning that occur over the course of treatment have direct clinical relevance. Mills, et al. [8] report that the strength of the relationship between improved family functioning and positive changes in the child’s mental health condition are profound: More than a 20-point improvement on a standardized measure of behavior was found for every one unit of improvement in family functioning. Not surprisingly, family therapy and family- based treatments are emerging as the treatment of choice for severe child and adolescent mental illness.

Improving family functioning, therefore, plays a vital role in effectively treating children and adolescents with mental health conditions. Substantial empirical support exists which demonstrates the efficacy of several evidence-based models of family therapy including Multisystemic Therapy [9], Functional Family Therapy [10], Multidimensional Family Therapy [11] and Attachment-Based Family Therapy [12]. The ability to accurately measure family functioning at the onset of treatment, knowing which problematic aspects of functioning require immediate clinical attention, and demonstrating improvements in family functioning over the course of treatment are essential for providing effective, evidence-based clinical care.

Theoretical model of the CAFFI and rationale for its development

The CAFFI is based on the theoretical model that informs Intensive In-Home Family Treatment (IIFT) [2], IIFT is a trans- diagnostic, family-based model developed for treatment-resistant, severely ill children and adolescents who are at risk for placement into high levels of care such as a psychiatric hospital or residential treatment program, or for youth who have not responded to other types of mental health care. IIFT is based on the theory that children and their families are in a continuous state of bidirectional mutual influence, both positive and negative, which contributes to the child’s mental health condition and that these dynamics can be improved through direct, targeted clinical interventions.

IIFT holds that children and adolescents with serious mental health conditions often have difficult-to-parent temperaments (strong-willed, oppositional, prone to emotion dysregulation, etc.) which influence their parents’ behavior in undesirable ways. Despite often good intentions, parents often fall into dysfunctional behavior patterns in response to challenging behaviors. This, in turn, worsens the child’s behavior, thereby resulting in less effective and more dysfunctional parenting responses. This bidirectional mutual influence—a moment-by- moment resonance (feedback loop) between family members— can result in chaos in the family that worsens over time resulting in the family presenting for treatment.

Negative resonant exchanges are those that mutually and progressively worsen both the parent and the child’s behavior, often resulting in high affective arousal that increases conflict, poor communication, and ineffective problem-solving strategies that become repetitive and entrenched over time. In contrast, positive resonance exchanges that promote warmth, empathy, collaborative problem-solving, and other attachment-enhancing behaviors become increasingly rare in the family. Treatment with IIFT consists of identifying and disrupting negative resonant exchanges and increasing positive resonant exchanges to the point where they become habitual. Attachment is then enhanced which results in more closeness and connections within the family and a reduction in the child’s mental health symptoms.

In addition to the bidirectional influences between the child and parent, sources of positive and negative resonance can also exist between the child and three other important relational subsystems: siblings, peers, and school. These other relationships and their effects on the child’s mental health condition can be similarly mitigated through treatment. All components of each of the five relational subsystems (the child’s temperament, parents, siblings, peers, and school) interact in complex ways that require careful assessment and clinical intervention.

The ability of the treating clinician to quickly and accurately assess these key family dynamics is essential. The CAFFI was developed to provide clinicians with a brief, efficient self-assessment measure administered at the start of treatment. It identifies where the family is struggling the most among the five relational subsystems, thereby providing the treating clinician with clear guidance as to where to direct his or her clinical attention. Additionally, The CAFFI can also be used as outcome measure to periodically track treatment progress over time with the goal of reducing a family’s scores as close to the normal (nonclinical) range as possible.

The Child and Adolescent Family Functioning Inventory

The CAFFI is a brief (25-item), no cost, publicly available self- report inventory completed by parents that assesses key dynamics between the parent(s) and the child or adolescent referred for mental health care. It can generally be completed in about 5-10 minutes and requires a reading level of only 5.7 using the Flesche-Kindcaid Readability Method [13]. The inventory consists of five subscales: Ability to Solve Problems, Managing Stress and Conflict, Closeness and Connection, Effective Communication, and Other Relational Subsystems, plus a Total Score. Each of these subscales and the items contained within were derived from prior research on the relationship between key aspects of family functioning and the severity of a child’s mental health condition and consistent with bidirectional resonance theory [2,4,5]. Each subscale has five items (Table 1) which are scored on a 5-point Likert scale from (1) strongly agree to (5) strongly disagree, so possible scores range from 5-25, with a Total Score ranging from 25-125. Higher scores indicate greater family dysfunction. The CAFFI can be used for a wide age group with norms established for ages 5-18.

Table 1: CAFFI subscales and individual items.

Methodology

In order to establish test-retest reliability of the CAFFI, the instrument was first distributed to a small sample of parents with children from the ages of 5-18 on various forms of social media who agreed to complete it twice separated by a two-week period (N=29). This initial sample consisted of families with children both with and without mental health conditions.

Data used to establish the remaining psychometric properties, as well as determine both clinical and nonclinical norms, were collected by distributing the CAFFI using SurveyMonkey to parents of children ages 5-18. Surveys were completed anonymously by parents and asked for no identifying information beyond basic demographics (the child’s age, gender, etc.). To obtain exclusively a nonclinical sample, the survey containing the CAFFI had two initial screening questions to exclude parents of a child or adolescent with any sort of history of receiving mental health care (“Has or is your child currently taking psychotropic medication (antidepressants, stimulants, mood stabilizers, etc.?” and “Has or is your child receiving any form of mental health care (individual therapy, group therapy, an Individualized Outpatient Program (IOP), drug and alcohol counseling, etc.?”). If the parent answered “no” to either question, they were directed to complete the CAFFI. To obtain a clinical sample, the survey containing the CAFFI included one screening question to only include parents of a child or adolescent with a history of receiving mental health care (“Is your child currently (or recently) receiving any form of mental health care such as individual therapy, group therapy, family therapy, psychiatric care/medication, an Individualized Outpatient Program (IOP), drug and alcohol counseling, etc.?”). If the parent answered yes, they were directed to complete the CAFFI. The final samples included a total of 791 nonclinical (normative) cases and 257 clinical cases for a total of 1048 families included in these analyses.

The analyses conducted for this study consisted of descriptive statistics, reliability analyses, a Confirmatory Factor Analysis (CFA), as well as independent-samples t-tests. In addition to these tests, further analyses were conducted during the course of data collection in order to determine the total sample size for this study to target. First, regarding this sample size determination, one goal of this study was to determine what scale scores could be relied upon for the purposes of determining, and separating between, normative and clinical samples. In order to achieve this goal, a series of analyses were completed in which descriptive statistics were conducted on various percentages of cases randomly selected from the full data set, which consisted of random samples of 10%, 25%, 50%, and 75% used in these analyses. These results were then reviewed and compared in order to help determine at what point stability in the scores would be achieved when comparing various subsamples within each specific sampling percentage. These results were then used in order to help determine whether the total sample size obtained in this study was adequate. Results indicated stability at a sample size of below 100; however, this study used a total sample size of 1,048 in the interest of increasing statistical power and providing even more accuracy in the data obtained.

Descriptive statistics conducted consisted of sample sizes and percentages illustrating the distribution of respondents on the basis of race or ethnicity and gender, along with measures of central tendency and variability calculated and reported in relation to respondent age. The latter also included measures of skewness and kurtosis.

Confirmatory Factor Analysis (CFA) was selected for use in this study instead of Exploratory Factor Analysis (EFA) as an aim of this study was to confirm the five-factor solution developed and proposed by the researcher, as opposed to determining the number of factors and the nature of those factors as would be the result of EFA. The specific factor structure proposed in this study consisted of the 25-item scale, with each subscale being composed of five of these 25 items. It was in the interest of the researcher to then confirm this specific factor structure, following confirmation of these scales’ high internal consistency reliability.

Reliability analyses as well as a CFA were conducted in order to determine whether this scale had an acceptable level of internal consistency reliability, the scale structure was appropriate in relation to the current study, and fit the data well. Internal consistency reliability was determined using Cronbach’s alpha, with a CFA conducted using IBM Amos 25.0 in order to determine the appropriateness of this factor structure as well as how well this factor structure fit the data. Finally, a series of independent- samples t-tests were conducted in order to determine whether there were significant mean differences in scale or subscale scores comparing the normative with the clinical participants. These independent-samples t-tests included Levene’s tests for the equality of variances, which served to determine whether there were significant differences in the variances of the two groups formed by the independent variable, group membership. When significant, indicating a violation of this assumption, the independent-samples t-test was then conducted in which the assumption of the equality of variances was not incorporated; in all other cases, the independent-samples t-test was conducted in which the assumption of the equality of variances was incorporated into the t-test equation. An alpha of 0.05 was used in all analyses as the standard for statistical significance, and all analyses with the exception of the CFA were conducted using IBM SPSS 25.0.

Results

Descriptive statistics

A series of descriptive statistics were conducted on these data in order to help describe this sample of respondents. This consisted of sample sizes and percentages of responses with respect to the child’s race or ethnicity and identified gender, along with measures of central tendency and variability conducted on the child’s age. First, with regard to ethnicity, a total of 499 participants (62.6%) were white or Caucasian, with 104 (13.0%) Hispanic or Latino. A total of 79 participants (9.9%) were Asian or Asian-American, with 74 (9.3%) black or African-American. Next, 26 participants (3.3%) stated that they were of another race, with 13 (1.6%) indicating that they were American Indian or Alaska native, and with two participants (0.3%) native Hawaiian or other Pacific Islander.

Next, with regard to identified gender, 433 participants (54.3%) were male, with 329 (41.3%) female. A total of 15 respondents (1.9%) indicated that they were non-binary, with nine (1.1%) indicating that they preferred not to say, six (0.8%) providing a response of “other,” and with five participants (0.6%) stating that they were transgender. Finally, with regard to participant age, mean age was found to be 10.8 years, with a median age of 11.00 years. A standard deviation of 4.3 years was found, along with a skewness of -0.21 and a kurtosis of -0.94, indicating no substantial skewness or kurtosis.

Test-retest reliability

A series of Pearson’s correlations were conducted in order to examine whether this initial test-retest sample (N= 29) evidenced an acceptable level of reliability. This was found with regard to all five sub-scales; The Ability to Solve Problems; r (59)=.908, p<0.001; stress and conflict; r (59)=0.881, p<0.001; Closeness and Connection; r (59)=0.905, p<0.001; communication; r (59)=0.880, p<.001; and other relational problems; r (59)=0.831, p<0.001. These positive, very strong, and statistically significant correlations suggest a very high degree of test-retest reliability with regard to these five sub-scales.

Reliability analyses

A series of internal consistency reliability analyses were conducted on the scale items in order to ensure that an acceptable level of reliability was present within the subscales as well as the overall scale. The valid sample size included in these analyses ranged from a minimum of 757 with respect to the reliability analysis conducted on the entire scale incorporating all 25 items, to a maximum of 795 valid cases, which related to the Closeness and Connection subscale.

All five subscales incorporated a total of five items each, with the overall scale incorporating all 25 items. The Ability to Solve Problems was found to have a Cronbach’s alpha of 0.892, with Managing Stress and Conflict found to have a Cronbach’s alpha of 0.850. Closeness and Connection was found to have a Cronbach’s alpha of 0.901, with Effective Communication found to have a Cronbach’s alpha of 0.861. Finally, Other Relational Subsystems were found to have a Cronbach’s alpha of 0.737, with the overall scale found to have a Cronbach’s alpha of 0.949. With the standard of 0.7 used to indicate an acceptable level of internal consistency reliability, the results of these analyses indicated acceptable or better reliability in all six cases [14,15].

Confirmatory factor analyses

In addition to the reliability analyses conducted, a Confirmatory Factor Analysis (CFA) was also conducted in order to confirm that the factor structure used for this 25-item Likert scale was appropriate in relation to these data. The results of the analysis focused on the magnitude of the standardized path estimates, the significance of these paths, as well as measures of model fit. The standards used to indicate an acceptable degree of correspondence between this factor structure and the data consisted of standardized path estimates of 0.30 or above, significant path estimates, as well as measures of model fit indicating fit that was acceptable or better. Regarding this final item, this consisted of a normed chi-square below five [16], a Tucker-Lewis Index (TLI) of 0.90 or above, and a Root Mean Square Error of Approximation (RMSEA) below 0.08 [16]. First, all path estimates were found to achieve statistical significance at the 0.001 alpha level, while all standardized regression weights were found to be above 0.45. With regard to measures of model fit, a normed chi-square of 3.539 was found, along with a TLI of 0.929, and an RMSEA of 0.049, 90% Comparative Index (CI) (0.046, 0.053), p=0.637. These results indicate significance with respect to all path estimates, acceptably high standardized path estimates, along with measures of model fit indicating acceptable or better fit. Overall, these results indicate an appropriate factor structure in this case, and a factor structure that fit the data well, with these results presented in Table 2.

Table 2: Confirmatory Factor Analysis (CFA).

Independent-samples t-tests

The independent-samples t-tests conducted for this study served to examine whether there were significant differences between the normative and clinical groups with respect to these scale and subscale measures. These two groups would be expected to differ given the research demonstrating the relationship between how well a family functions and the severity of a child’s mental health condition [4,5]. First, with regard to Levene’s test of the equality of variances, the results of this test were not found to achieve statistical significance with respect to the first subscale, the Ability to Solve Problems, with significance being found with respect to the remaining four subscales as well as the overall scale. This result indicates variances that were not significantly different between the normative and clinical samples with respect to the Ability to Solve Problems, and with significant differences being indicated in the remaining five cases. The independent-samples t-tests were then conducted assuming the equality of variances with respect to Ability to Solve Problems, and not assuming the equality of variances in all other cases. With respect to the results of these t-tests, statistical significance was indicated in all six cases, with significantly higher means (more dysfunction) found in the clinical sample with respect to all five subscales as well as the overall scale as compared with the normative sample. These relevant descriptive statistics are reported in Table 3, with the results of the independent-samples t-tests conducted presented in Table 4.

Table 3: Independent-samples t-tests descriptives.

Table 4: Independent-samples t-tests.

Discussion

The results of these analyses indicate that the CAFFI demonstrates high reliability based on the results of the alpha coefficients found. In addition, the CFA indicated an appropriate factor structure and a good fitting model based on the magnitude of the standardized path estimates, the significance of the estimates, and the measures of model fit. In addition, the independent-samples t-tests indicated significant mean differences in all five subscales, as well as the overall total score when comparing normative and clinical samples with the clinical sample predicted as having significantly higher means.

Consideration was given to obtaining clinical cutoffs to distinguish those families that more clearly fall into the clinical range. However, Streiner [17] argues that the rationales for taking variables that are measured on a continuum and putting them into categories (for example, above or below a cut-point) are weak and that categorization results in lost information, reduced power of statistical tests, and increased probability of a type II error.

The recommended method for using the CAFFI is to compare a given family’s scores on the measure’s five subscales and the Total Score to the mean scores of the nonclinical, normative sample. If the obtained scores are higher on any of the subscales (i.e., there is more dysfunction than the nonclinical mean), the treating clinician would then work to reduce those scores as far down into the normal range as possible which in turn would be predictive of a reduction in the child’s mental health symptoms [8,5].

Conclusion

This paper describes the development and establishes psychometric properties of the Child and Adolescent Family Functioning Inventory (CAFFI), a new, brief, no cost, and publicly available clinical assessment instrument. Family therapy and family-based treatments are emerging as the treatment of choice for severe child and adolescent mental illness due to substantial research demonstrating their efficacy. Family dynamics play an important role in a child’s mental health condition and research has shown that as family functioning improves with treatment, the severity of a child’s symptoms decreases. Therefore, the ability to efficiently assess these key family dynamics is essential in clinical practice, thereby allowing the clinician to begin helping families as early as possible in the treatment process.

Limitations

This is, of course, the first attempt to establish the psychometric properties of the CAFFI. Additional research would be required further test both reliability and validity. Although not a part of this study, it would be useful to compare derived scores on the CAFFI to other measures of family dynamics although all of the items on the CAFFI have very high face validity (the items appear at face value to measure what they claim to measure). No attempt was made in this study to distinguish scores based on gender or ethnicity, so this would be important to explore in future research, as well as perhaps obtaining separate norms for younger children and older adolescents. Finally, SurveyMonkey limited data collection to North America, so more data would need to be collected to better establish both clinical and nonclinical norms in other geographic locations and for other cultures and ethnicities.

Declaration

For a copy of the CAFFI and its accompanying scoring sheet, please go to: http://myiift.org/CAFFI.html

References

1. Diamond GS, Diamond GM, Levy SA. Attachment-based family therapy for depressed adolescents. Am Psychol Assoc. 2014.

   [Crossref] [Google Scholar]

2. Sunseri PA. Intensive in-home family treatment for severe adolescent mental illness: A theoretical model and case study. Am J Fam Ther. 2021;3:1-16.

   [Croosref] [Google Scholar]

3. Sperry L. Couple and family assessment: Contemporary and cutting‐edge strategies. (3rd Edition). Routledge. 2019.

   [Croosref] [Google Scholar]

4. Sunseri PA. Family functioning and residential treatment outcomes. Resid Treat Children Youth. 2004;22(1):33-53.

   [Croosref] [Google Scholar]

5. Sunseri PA. Hidden Figures: Is improving family functioning the key to better treatment outcomes for seriously mentally ill children?. Resid Treat Children Youth. 2019;37(1):46-64.

   [Croosref] [Google Scholar]

6. Compton SN, Almirall D, Sherrill J, March JS, Ginsburg GS, Piacentini JC, et al. Predictors and moderators of treatment response in childhood anxiety disorders: Results from the CAMS trial. J Consul Clin Psychol. 2014;82:212-224.

   [Croosref] [Google Scholar] [PubMed]

7. Sunseri PA. The prediction of unplanned discharge from residential treatment. Child Youth Care Forum. 2001;30(5):283-303.

   [Croosref] [Google Scholar]

8. Mills L, Hall J, Ridenour A, Murphy M. (in press). The importance of parent engagement in outdoor behavioural health programs for youth. J Ther Sch Prog.
9. Henggeler SW, Sheidow AJ. Empirically supported family-based treatments for conduct disorder and delinquency in adolescents. J Marital Fam Ther. 2012;38(1):30-58.

   [Croosref] [Google Scholar] [PubMed]

10. Sexton T, Turner CW. The effectiveness of functional family therapy for youth with behavioral problems in a community practice setting. J Fam Psychol. 2010;24:339-348.

    [Croosref] [Google Scholar] [PubMed]

11. Liddle H, Dakof G, Turner R, Henderson C, Greenbaum P. Treating adolescent drug abuse: A randomized trial comparing multidimensional family therapy and cognitive behavioral therapy. Addiction. 2008;103(10):1660-1670.

    [Croosref] [Google Scholar] [PubMed]

12. Diamond G, Russon J, Levy SJ. Attachment-based family therapy: A review of the empirical support. Fam Process. 55(3):595-610.

    [Croosref] [Google Scholar] [PubMed]

13. Kincaid JP, Fishburne RP, Rogers RL, Richard L, Chissom BS. Derivation of new readability formulas (Automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Inst Simul Train. 1975.
14. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ. 1997(7080);314:572.

    [Croosref] [Google Scholar] [PubMed]

15. Cohen J. Statistical power analysis for the behavioral sciences. Routledge. 1988.

    [Croosref] [Google Scholar]

16. Schumacker RE, Lomax RG. A beginner's guide to structural equation modeling. (4th edition). Routledge. 2015.

    [Croosref] [Google Scholar]

17. Streiner D. Breaking up is hard to do: The heartbreak of dichotomizing continuous data. Can J Psychiatry. 2002;47(3):262-266.

    [Croosref] [Google Scholar] [PubMed]