In the present investigation the Randomized Clinical Trial (RCT) is the “gold standard” for evaluating new therapies or strategies in medicine [1]. A low quality in RCTs design or publication, it could lead to an underestimation of the risk or even false, using treatments that may be less or even harmful, and could produce daily clinical practice consequences [2]. So a higher quality in published clinical trials to help choose the best therapies for patients.

There are different methods of assessing the ethical quality of Clinical Trials (CTs) publications, amongst which Dechartres et al. identified 26 scales or checklists based on CONSORT (Consolidated Standards of Reporting Trials) [3]. In this way they obtained increased data on methodological issues, but only 29% adequate masked allocation. They also found that quality evaluation still presents significant problems [4]. Katrak et al. also observed heterogeneity in the use of criteria and unclear definition in the scales, which might limit the relevance of the results [5].

No studies have been found assessing the methodological and ethical quality of the CTs published in Annals of Family Medicine journal.

The aim is to analyze quantitative and qualitative methodological and ethical quality of published clinical trials in Annals of Family Medicine journal, between 2010-2013. We chose this journal because it has the highest impact factor in Family Medicine (4.613), published CTs, CONSORT endorsement and has an important general quality. We want to know how clinical trials are performing in primary care, as this will help to improve the design and publication of these.

On March 25, 2014 we use a hand search and subsequent checklist to analyze CTs published in Annals of Family Medicine since January 1, 2010 to December 31, 2013.

It is a print edition with free online access (no “open access”), which follows the CONSORT standards [6,7] and the ICMJE (International Committee of Medical Journal Editors) recommendations [8].

We follow the CONSORT statement [3] and PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [9], which provide higher quality checklists [8] for CTs publishing [10,11], and have been implemented in over 400 journals and editorial groups as ICMJE [3]. We will also use the most recent edition of the Declaration of Helsinki (2013) [12,13].

Inclusion and exclusion criteria

We obtained published studies from CONSORT [14] to increase the quality [15]. We included all CTs carried out in humans. Exclusion criteria were: 1) CT assessing diagnostic test or costs; 2) “Letters to the editor” type studies and pilot tests; 3) Continuation CT (as full publications or “short reports”) for the study of secondary variables or secondary analysis of a previously published CT with the same number of subjects.

Checklist to assess the ethical and methodological quality

We reviewed a checklist [16] based on CONSORT [17] that had 80 items divided into 8 sections. We have updated to a 133-item checklist in eleven sections. It was applied to assess their ethical and methodological quality. Following the Declaration of Helsinki [18], items were modified or created to adapt them to Primary Care (PC) with the assistance of the PRECIS (Pragmatic-Explanatory Continuum Indicator Summary) tool [19]. More explicit variables were created for placebo use. We introduced AHRQ [20] (“Agency for Healthcare Research and Quality”) criteria and the Jadad quality scale. We use the assessment by a DSMB (Data and Safety Monitoring Board) and “stopping rules” (Haybittle-Peto, Pocock, O’Brien-Fleming) [21]. We also evaluated sequential trials, optimized evaluated by the CREC (Clinical Research Ethics Committee) or futility analysis. We stressed race and language, incentives, training in Bioethics, types of conflicts of interest and sponsor influence.

Analysis and data extraction

Two tables with frequency and rates for qualitative variables were made. We built a table with the mean, median, mode, minimum, maximum and statistical percentiles (P25, P50, P75) for quantitative variables. Data were calculated using SPSS 20.0.

Comparable variables on the κ, κ means and 95% CI value was obtained [22].

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent was no necessary to obtain because no individual participants were included in the study.

We found 35 CTs that met the inclusion criteria (Figure 1). Most were observational studies. Afterwards we eliminated three of these (not meet the inclusion criteria) and final analysis was performed.

Figure 1: Flow diagram with the choice of studies for analysis according to PRISMA [10].

Table 1 shows the frequencies and rates for the most representative items.

Table 1: Methodological variables: frequencies and rates.

Assessment of methodological quality

In the section on “justification and objectives” of our checklist, we observed that the following appear in all: 1) CT in heading or subheading, 2) related CTs, 3) systematic review cited and 4) a priori hypothesis.

Regarding the “clinical trial type”, 68.6% were performed in > 5 centers and 14.3% in ≤ 5. In 80% some authors are epidemiologists or statisticians.

In the 3rd section, the “clinical trial design” is evaluated. In 80%, there were controlled parallel-groups and controlled factorials in 4 CTs. There is a tendency to be more pragmatic than explanatory according to the criteria of AHRQ.

Baseline comparison was made in a table following CONSORT [14] for almost all CTs (34), and “p” in 68.6%. Only the CT by Licciardone et al. [23] is sequential; moreover it uses the O’Brien Fleming stopping rule. The effectiveness of masking and withdrawals due to adverse events were found in only 5 CTs and the latter were uncertain in 6 studies (Table 2).

Table 2: Ethical variables: frequencies and rates.

The “choice of subjects” is the next section (4^th). The subjects are White in 54.3%, 45.7% African Americans, 37.1% Hispanic and in 8 studies Asian. Race is not indicated in 42.9%, not specified in 13 CTs, and in 4, subjects are ethnic minorities such as American Indian and Vietnamese. The inclusion of immigrant or low-income groups has increased in the last three years: 2011 (1), 2012 and 2013 (3). We have not observed the inclusion of “vulnerable” people [24].

In the 5^th section the “sample size” is evaluated. Quantitative variables are reported in a table (Table 3). The “alpha” error is 74.3%, “beta” is 82.9%, the magnitude of the minimum difference to be found is 80%, and the “flowchart” is 85.7%.

Table 3: Most representative statistic for numeric variables.

Regarding the “Description of treatment” (section 6), with an effective standard treatment, placebo or non-intervention has not been used as a control in 62.9%. A strong and compelling case for the use of placebo or non-intervention for methodological reasons was given in 6 CTs, and the determination of efficacy and safety in two. In 37.1%, there were serious adverse events in general (including death), and it was unclear in 3 studies. The “intention to treat analysis” was conducted in 15 CTs, “not indicated” (to be performed but not mentioned) in 3, and was “unclear” (to be mentioned but not performed properly) in one. According to the Jadad score, 15 studies were of good quality (3), 16 low-grade (1 or 2) and 4 excellent (5).

The 7th section deals with the “development of the trial and assessment the answer”. The “primary outcome variable” or “endpoint” is not described in one CT and is “unclear” in 11.4%. “Multivariate analysis” is applied in 42.9%. There is a DSMB in 14.3%. Finally, CONSORT standards mentioned that it followed in 28.6%.

Assessment of other ethical quality parameters

In the 8th section, the most important variables are represented in Table 2. Two CTs included minors as research subjects. Parents or representative completed Informed Consent (IC) in one of them and the other this completion was “unclear”.

The confidentiality of the participants’ personal information was found in only 17.1%. In 8.6% of the CTs, people were injured by their participation, but none were compensated. In 22.9% a study protocol is not stated.

In all CTs possible conflicts of interest were stated. The influence by the sponsor was not described in 48.6%. The studies reviewed are registered in 57.1% of the cases.

Calculation of Kappa index

The authors (FGG and ACG) independently reviewed the 35 articles chosen following the checklist. For the completion of the variables they followed detailed instructions and discrepancies were resolved by consensus. In 82 variables we found a κ average of 0.93 (95% CI, 0.90-0.96), optimal) [25]. In Table 4 the variables assessed by κ appear.

Table 4: Variables with corresponding κ value.

In current Family Medicine RCT is one of the main tools to compare a new treatment with other standard or a placebo. Therefore the quality of the publication of new RCTs, is crucial for a reliable picture of the results they can obtain in order to implement them safely on clinical practice.

Clinical trials published in “Annals of Family Medicine” journal are most frequently named in Family Medicine. Therefore, our study evaluates the quality of these, for a fairly good idea of how they are developing RCTs in Family Medicine. We assess one journal only to have a minimal publication bias.

We observed a higher general quality if we compare our study with similar. In a related study [16] there is a progressive increase in masked randomization: 33.1% (1993-95), 56.7% (2004) and 68.6% in 2010-13. It could be explained because after CONSORT, it was stressed that it was essential to reduce selection bias. In contrast, double blind trial is less frequent (5.7%) versus earlier revisions (43.9% (1993-1995) and 43.3% (2004) and placebo use as a control element has also decreased: 37.0% (93-95), 16.7% (2004) and 11.4% (2010-13). Probably due to the treatments used less frequent damaging. The figures for “intentionto- treat” and “multivariate” analysis remain stable (47.7% and 44.2%, 42.9% (both), although both were higher in 2004 (70% each). This may be due to the increased statistical power searched in recent years.

Regarding the indicators of ethical quality [16], the number of CTs without a reference to CREC has dropped, particularly in the latest period (25.5% (93-95), 20% (2004) and 5.7% (2010-13). This is perhaps attributable to CONSORT standards. Coinciding with Ndebele et al. [12], CTs on children or disabled persons have been very infrequent in the three studies. A possible explanation could be that in Family Medicine children as research subjects are uncommon.

In Sinha et al. [26], a Jadad score ≥3 was found in 40%, which is slightly higher than the rate found in our study (54.3%). This is perhaps because there are less RCT.

We agree with Agha et al. [27] that the country where most CTs were carried out was the US (15). Possibly because it is the country where more research is conducted.

In contrast with our results, O’Meara et al. [28], in their review observed very limited information on the sample size and masking. We think it could be because most studies reviewed are low quality.

We agree with Laframboise et al. [29] when they describe all the justification and objectives. This description is possibly necessary to assess the CTs results.

Panic et al. [30] grant and sponsors were described in 70%, which is more frequent than in our findings. This could be because “Annals of Family Medicine” RCTs does explain those data less than conflict of interest.

Similar to our study, the most common source of funding in Lai et al. [31] was public (58.1%), probably because CTs are very expensive. In this study, the significant adverse events were common (64.9%), possibly by not studying safe medical interventions. Moreover the methodological quality is higher in our study, only appear in this review in 45.9% the primary endpoint, and sample size calculation in 39.2%, masked randomization in 29.7%, blind in 12.2%, and “intentionto- treat” in 27%. Ifeacho et al. [32] observed lower rates than ours in relation to the appearance of CT in the title (40%), description of the design (12%), sample size (20%) and random assignment (32%). Moreover their study obtained less frequent results regarding the type of masking (24%), funding-sponsors (28%) and statistical methods (22%). Poolman et al. [33] found fewer cases of an epidemiologist as author (9%), multicenter types and following of CONSORT (13%) standards than in our study. We believe it might be because the journals assessed in these studies did not follow CONSORT standards completely.

Anttila et al. [34] performed after CONSORT, obtained similar results to ours on justification, objectives and design.

In 2011 Kiehna et al. [35] found data with rates lower than our study in the justification and design, in contrast with the description of significant adverse events. The sequence of randomization (41%), the analyses of “intention-to-treat” (33%) and masking (19%) were also significantly lower than those included in our review. Augestad et al. [36] observed less frequency of CT registration (25%), funding (95%), performing in PC (70%) and masking type (15.6%). The mean Jadad score was 2, also below our findings. This is perhaps because they assessed more pharmacological studies with questionable quality.

Coinciding with Baskerville et al. [11] we only have access to the published CT and not the original data (that may differ). There is variability between the studies regarding the measurement of the “endpoints”, due to the many types of variables found by the study treatment.

Would be desirable a study based on an analysis of disciplines, specialties or a particular subject in the future.

The authors recommend that a greater emphasis on proper implementation of the CONSORT standards, could take an improvement in quality of Family Medicine clinical trials published.

Therefore, a deeper insight into obtain a better methodological and ethical quality, it will lead us to further analysis in the evaluation of a new medical treatment before use with patients. In this way, it could avoid many of the side effects or ineffectiveness of new therapies, so that when they were cast safe and effective into the public. This also would be a great saving for the health system costs and potential demands of patients in the future.

We observe that the most frequent CT published in Annals of Family Medicine (2010-2013) are: multicenter >5, with public funding, cluster controlled open parallel, with decentralized masked randomization using an active treatment as a control, in a 6-12 months period; made up of in a biracial patient group and applying a method of medical intervention with positive results in 50% (measured by questionnaire). In 50% the intention-to-treat analyses are performed, and they show serious adverse events and good quality (Jadad 3). Furthermore, IC (verbal or not mentioned), no incentive and no conflicts of interest occur more frequently than in other publications cited. Lastly, we find optimal κ.

Following the CONSORT standards publication, it has increased the overall quality of the CTs published. But there are some areas for improvement in the methodological and ethical quality of the CTs published from 2010 to 2013 in Annals of Family Medicine.

This study would not have been possible without the support offered by a future member of my family throughout the investigation process.

Author Alberto Castaño-García declares that he has no conflict of interest. Author Francisco Guillén-Grima declares that he has no conflict of interest. Author Pilar León-Sanz declares that she has no conflict of interest.