The Consolidated Standards of Reporting Trials (CONSORT) Guidelines were created in 1996 and most recently updated in 2010 [1,2]. The guidelines were specifically developed to alleviate the problem of inadequate reporting of randomized controlled trials (RCTs), which is associated with bias in estimating the effectiveness of interventions [2,3]. CONSORT strongly recommends and encourages transparency with reporting methods and results so that readers can accurately interpret and assess strengths and limitations of the studies [4-8]. Several follow-up studies have noted a positive effect of the guideline recommendations on the overall reporting of RCTs [5,7]. One of the recommendations of CONSORT is the inclusion of a flow diagram that maps the path of each study subject through the entire trial, from randomization to analysis and follow-up [2]. The CONSORT flow diagram not only enables readers to more easily track the number of participants, but it also assists in determining if intention-to-treat (ITT) analysis was carried out by evaluating a structure and process of the study population in the RCT and its attrition. Although many journals have adopted CONSORT as part of their submission criteria, published articles from literature indicate that many RCTs do not include the recommended CONSORT flow diagram in the publication [9,10] and even fewer studies reported associations between the COSORT compliance and the CONSORT endorsement by journals.

As disproportionate attrition of the patients with various reasons such as missing data, loss to follow up, adverse drug effects, and others, can distort the initial randomization scheme and potentially threaten the internal validity of the study, which can subsequently lead to an inaccurate conclusion and inappropriate application to clinical practice [11,12], this study was prepared to assess the level of CONSORT compliance by including a flow diagram, to describe the extent of attrition during the stages of study subject enrollment, allocation, loss to follow-up, and analysis, and to evaluate an association between the CONSORT compliance and the CONSORT endorsement by journals. Our study focused on infectious diseases because RCTs in the diseases often include more than one analytic group, such as clinical or microbiologic outcomes, which requires additional descriptions about attrition of their study population.

Data sources and search

A cross sectional evaluation using a systematic literature search was conducted among all English publications of RCTs of anti-infective agents in the top 10 general medicine journals and the top 5 infectious disease journals with highest impact factors, which yielded a total of 14 journals as one journal belonged to both groups. The journals were the New England Journal of Medicine, Journal of the American Medical Association, British Medical Journal (Clinical Research Ed), Archives of Internal Medicine, PloS Medicine, Annals of Internal Medicine, Clinical Infectious Diseases, the Journal of Infectious Diseases, the Lancet Infectious Diseases, AIDS, Emerging Infectious Diseases Journal, Annual Review of Medicine, Canadian Medical Association Journal, and Annals of Medicine Journal. The review was made on the published articles from January 1, 2010 to December 31, 2010 and search terms included randomized controlled trials in combination with the Medical Subject Headings for “anti-infective agents.” Endorsement status of the CONSORT by the journals in year 2010 was determined by the endorsement status indicated from the CONSORT website on the access date [13], communication with the editor’s office of each journal, and instruction to the authors to adhere to CONSORT statement from the journal. Priority was given to responses from the editor’s office, and “endorsed” status was considered if two or more sources of the information were consistent.

Study selection and data analysis

Medline search identified 129 articles from 14 different journals. Following exclusion criteria, 67 articles were selected for further review to determine their eligibility based on the criteria of being 1) randomized controlled study design and 2) original research on infectious disease. Compliance to the CONSORT statement on participant flow in the results section [1,2] was determined by inclusion of one or more flow diagrams describing the study population. The CONSORT flow diagram accounted for the stages of enrollment, allocation, follow-up, and analysis. To further describe the level of CONSORT compliance by each stage, compliance was determined in three levels (i.e., complete, partial, and missing). If the study fully accounted for the number of the participants and gave specific reasons in each stage, the study was considered “complete” compliance in the specific stage. “Partial” status was defined if the stage description did not completely provide information on the number of excluded participants or reasons but did include some information. “Missing” status was defined if the stage description did not include any information on the number of participants or reasons for exclusion. Information on the extent of attrition, and employment of ITT analysis was collected and summarized from CONSORT flow diagrams as well as from body text of the articles.

Analytic goals of our study were primarily reporting results from the descriptive analyses on frequency or proportion of CONSORT compliance by stage and journal type. Multivariate logistic regression was conducted to determine potential predictive factors for CONSORT compliance of including a flow diagram and adjusted ORs were calculated. The significance level of the multivariate analysis was set at 0.05 and the analysis was performed with SPSS statistical software, version 20 (SPSS Inc, Chicago, IL).

The study identified 129 RCT articles published from January 1st to December 31st in 2010 from the selected 14 journals and only 67 RCTs from 9 journals met the inclusion criteria (Figure 1). Of the 67 infectious disease RCTs, 52 articles (77.61%) included the CONSORT flow diagram. Forty-four (65.67%) articles described an ITT approach in their methods (Figure 1). About a half (50.75%) of the selected articles were HIV/AIDS, followed by malarial infection (8.96%) or parasitic infection (8.96%) Table 1.

Table 1: Distribution of the study articles by infectious disease category ^*Other diseases include sexually transmitted diseases, hepatitis, influenza,
Mycobacterium infection, or other viral infections.

Figure 1: Flow diagram of RCTs included in the study

The selected articles were evaluated for CONSORT compliance by evaluating the inclusion of participant flow diagrams and the level of the compliance in each stage of the CONSORT descriptions. Of the 52 RCTs that included the CONSORT participant flow diagram, higher proportions of the articles included ‘complete’ descriptions on stages of enrollment (40 studies or 76.92%) and allocation (44 studies or 84.61%). However, smaller number of studies included ‘complete’ descriptions on the follow-up stage (16 studies or 30.76%) Figure 2.

Figure 2: The level of CONSORT compliance^*
*Complete: The study fully accounted for the number of the participants and gave reasons. Partial: The study did not completely provided information on the number of excluded participants or reasons. Missing: The section was not provided.

Journals that did not endorse CONSRT guidelines include the Clinical Infectious Diseases journal, the Lancet Infectious Diseases journal, and the New England Journal of Medicine journal during the study period and they accounted 53.7% (36 out of 67 articles) of the total number of articles (Table 2). Of the 52 articles that included the CONSORT flow diagram, 33 (64.5%) were published in journals not endorsing CONSORT Statement in their instruction to the authors (Table 2). Findings from our study also showed that the CONSORT endorsement by the journal was a negative predictor for including CONSORT participant flow diagram. (ORadj=0.144; 95% CI 0.036- 0.575, p<0.05).

Table 2: Characteristics of selected articles by journal, endorsement of the CONSORT Statement, and inclusion of the CONSORT participant flow diagrams

Our study provides a snapshot assessment of the level of compliance to the CONSORT statement on participant flow by including a flow diagram from published RCTs of infectious diseases in 2010. A comprehensive CONSORT flow diagram reduces the time for readers like clinical pharmacists to follow the flow of the study participants so that key information related to study subjects in each phase of the clinical trial can be captured so that clinicians could make an accurate interpretation and assessment of the strengths and limitations of the findings from RCTs [4-8] to be applied to pharmacy practice.

From the results of our study, over three out of four RCTs included a CONSORT flow diagram. Our findings corroborate with other published studies from non-infectious diseases – such as obstetric anesthesia (89%)[14], respiratory disease (69%) [15], acute and chronic myeloid leukemia, and myelodysplastic syndromes (89%) [16], and restless legs syndrome (88.9%) [17]. While our study reported that majority of the articles with CONSORT participation flow diagrams were from non-CONSORT endorsing journals, our results are not consistent to those reported by Hopewell et al. [18] reporting that 90.11% of studies with the diagram were published in CONSORTendorsing journals.

Our review also showed that the reporting of the integral elements of the RCTs in the flow diagram was lacking. In the elements examined, i.e. overall number of people screened for study eligibility, the number of people that were randomized, the number of people allocated to either control group or intervention group, and the overall number of people analyzed, variabilities in the level of descriptions about the study participants were demonstrated, particularly for the numbers in the follow-up and the analysis phases. Details of the attrition rate of the participants or those who discontinued the intervention were poorly accounted for; only 30% of the trials reported the overall number of lost-to-follow-up, which can lead to bias in estimating the effectiveness of interventions, thereby undermining the aim of CONSORT [2,3]. The variability in the clear reporting of the elements of the flow diagram was not limited to our study. A study by Kehoe et al on nutrition and pregnancy found that only 31% of their studies reported the overall number of people screened for study eligibility [19]. Follow-up analyses as well as analyses of the groups to which subjects were originally allocated are usually included as part of good research method practices [20].

To achieve an ITT analysis in studies, all randomized subjects must be included in the final analysis in the arm to which they were allocated, irrespective of whether subjects received the allocated treatment, left the study, or failed to comply with treatment regimen [11,20,21]. A total of 65.67% (44/67) of the RCTs in our study reported an ITT approach in their methods. Gravel and et al reported a similar result that 62% of articles reported the use of ITT [22]. This is critical for interpretation and application of the findings in clinical practice [11]. As argued by many authors, the exclusion of study subjects from final analysis may favor one treatment group over another and can bias the results of the study, which can affect the validity of the inferences drawn from the study [23]. A disproportionate attrition rate threatens the internal validity of RCTs [4,11,12,24] and it has also been described that statistical analysis may make unreasonable assumptions when data are incomplete or missing [25].

Our findings indicate that articles with the CONSORT flow diagram did not include crucial elements of the RCTs such as numbers of study participant attrition or numbers in the ITT analyses with sufficient clarity. Unclear and incomplete descriptions about a RCT’s study population and its attrition process can be misleading for clinicians and be a potential source of inaccurate interpretations. The problem of missing data in RCTs cannot be completely avoided, but it can be minimized through careful study design [26], and transparency in attrition descriptions could allow clinicians including clinical pharmacists to accurately interpret and assess strengths and limitations of RCTs [7]. The uncertainty related to missing data in a study can be minimized by carrying out a sensitivity analysis of the missing population to investigate the effect of the missing data from the assumption made in the ITT analyses [27,28].

This study has a few limitations. Our evaluation was limited to RCTs involving anti-infective agents, publications in English language, and trials published in the highest impact factor general medicine and infectious diseases journals. Therefore, our findings might not be a true representative of all published studies of RCTs or clinical conditions. “CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomized trials” was published 24 March 2010 [3]. Although there was a time gap between the time the updated guidelines were published and the search of our study, inclusion of the flow diagram were consistently recommended and the progress of the trial was divided into four stages of enrollment, allocation, follow-up, and analysis in the 2001 and 2010 guidelines [1,2].

In conclusion, about one out of four published RCTs in the highest impact factor medical- and infectious disease journals did not include the CONSORT participant flow diagrams in 2010 and the CONSORT endorsement by the journal was not a positive predictor for the inclusion of the diagram. Clear and complete description of the attrition, especially on the follow-up process, can enhance valid interpretations of the findings by clinical pharmacists. Further studies are needed to explore if similar patterns are observed from other clinical conditions.

This study was supported in part by funding from the Health Resources and Services Administration grant D34HP6042 and Seoul National University College of Pharmacy Education and Research Foundation grant.