Journal of Clinical Trials
Open Access

The Ten Things we Always Wanted to Know about Randomised Controlled Trials but were Afraid to Ask

Jacques Lelorier^{* *}Correspondence: Jacques Lelorier, Department of Medicine, University of Montreal, Montreal, Canada, Email:

Author info »

Introduction

The randomised controlled trial is the cornerstone of evidencebased medicine. Its proper interpretation is thus of key importance in establishing practice guidelines. However, 'details' which might be important in the interpretation of a particular trial are often not reported. The aim of this paper is to mention ten such “details” which, in our opinion, might be of interest to the reader.

Literature Review

Question 1

If it is very likely that the patients in at least one arm of the trial were previously exposed to the trial drug or a member of its class. Please inform me of the percentage of these patients. If this percentage is substantial, we would also like to know, in a sensitivity analysis, the percentage of side effects in the previously exposed and previously unexposed patients.

In some trials of novel drugs, patients in one or both of the two arms are randomised to receive a drug which is already in the market or for which one or more drugs of the same class are already in the market. It is thus quite likely that some of the trial patients might have already been exposed to this drug or a member of its class prior to their recruitment. When this is the case it is extremely unlikely that they will be offered to participate in the trial if they had had side effects or lack of efficacy when previously exposed to the drug under consideration or a similar drug. For instance, it is highly unlikely that patients with a previous statin induced myopathy might have been randomised in a statin trial. As a consequence, if a substantial number of the randomised patients had previous exposure to a statin, they are much less likely to develop a myopathy, thus giving a false impression of the tolerability. In fact, the excess rate of myopathy observed in randomized controlled trials is about 1/10000 person-years, whereas in a cohort study, the rate of any diagnosed muscle problem was about 4.1/10000 in the first year of treatment [1]. Thus, these results suggest that there is a higher rate of side effects in clinical practice compared to randomized controlled trials, and one of the reasons could be the exclusion of patients that were previously exposed to the drug or a member of its class.

Question 2

Please provide us with the results of the randomised blinding test: Testing for randomised blinding is the only valid way to determine whether a trial is really blind. This is easy to do. At the end of the trial, ask the patients and the investigator to guess their treatment allocation and the reasons that motivate their guess. The percentage of correct guesses by patients and investigators provides a simple assessment of the success of the blinding procedures. In situations where blinding is successful, trialists and readers can be confident that guesses about treatment allocation have not biased the trial’s outcome. In situations where blinding fails, trialists and readers will have to evaluate whether or not bias may have influenced the trial’s outcomes or the co-treatments. This method is particularly useful when some side effects are much more frequent in the experimental group compared to the placebo group, which could affect the blinding. A clear example is the adverse reactions associated with peginterferon beta-1a, an injectable drug used in the treatment of relapsing multiple sclerosis. In fact, in a study announced as double-blinded, 62% of the patients injected with peginterferon beta-1a every 2 weeks and 56% of those injected every 4 weeks had an injection site erythema, whereas in the placebo group, this percentage was only 7% [2]. Furthermore, influenza-like symptoms are another adverse reaction commonly associated with the drug. In fact, in the same study, the percentage of patients that experienced this side effect was 47% in both experimental groups compared to only 13% in the control group. Another example of a side effect commonly associated with a drug is flatulence due to acarbose treatment. Indeed, a double blinded study comparing acarbose to placebo in controlling blood sugar in Asian type 2 diabetic patients treated with insulin showed that flatulence was observed in 28,6% of the patients in the experimental group, whereas this percentage was only 16,4% in the control group [3]. These are clear examples that certain drugs are associated with significant side effects that could give patients or investigators an idea that the real drug was administered. Other drugs may have more discreet side effects or other peculiarities that could affect the blinding, and the only way to assess this is by asking the patients and the investigators to guess which treatment was given and the reason motivating their choice.

Question 3

Please inform us of the following dates: Trial registration, first patient randomised, last patient randomised, database locked. If the principal and/or secondary outcomes are different in the registration and in the paper, please let us know the date and the reasons for this modification. Trust but verify, Ronald Reagan.

One of the aims of trial registration in ClinicalTrials.gov or other data bases is to reassure the reader that the protocol designed for the trial is identical to the one reported in the eventual publication, particularly concerning the outcomes.

Question 4

In order to better evaluate the degree of deviation caused by the modified intent to treat please provide, in a sensitivity analysis, the results of the hardcore intent to treat analysis: The many virtues of the intent to treat analysis are well recognized and accepted. Most people agree that it should be the basis of the main analysis. Unfortunately, it appears to be going the way of the Dodo bird and the dinosaurs and is replaced by a hardier sub-species better adapted to the present permissive environment: The modified intent to treat. In the hardcore intent to treat, all the randomized patients are included in the analysis. In the modified version, there are some postrandomization exclusions. Descriptions of modified intent to treat analysis are often difficult to interpret particularly if they contain more than one criterion.

Question 5

Please provide us with results of the per-protocol analysis: The fact that the intent to treat analysis has priority does not mean that the per-protocol is necessarily useless. If interpreted with caution, given the loss of the benefits of randomization, per protocol provides information on the benefits of the intervention on the patients who adhered to the protocol. Usually the intervention will appear to be more efficacious in the per-protocol than in the intent to treat analysis. If the difference is small, it indicates that the intervention is well tolerated. If it is large, it’s probably due to compliance issues, most likely caused by side effects. If that is the case, it opens an avenue to improve the intervention effectiveness by addressing the compliance issues.

Question 6

Please provideus with both, intent to treat and a per-protocol analysis of the unwanted side effects: Either analysis might be appropriate depending on the mechanism of action of the side effect under consideration. If the side effect only occurs while the drug is in the body (anticoagulant), then a per-protocol (while on drug) analysis is most appropriate. However, if the side effect can occur for a prolonged period of time after the drug has been discontinued (myocardial infarction from an atherogenic drug) then intent to treat analysis is the only way to detect it. If the mechanism of action is unknown, the wisest strategy consists in casting a wide net and does both analyses. The importance of doing these analyzes is highlighted by the results of a study showing that there was a significant underreporting of cardiovascular events in refecoxib trials. Once in the market, this drug showed almost a doubling of the Cardiovascular Thrombotic (CVT) events predicted by the initial studies. In fact, after an in-depth analysis of the results, it was clear that only a per-protocol analysis was performed, and this did not show any significant cardiovascular risk associated with the drug. However, intent to treat analysis subsequently made with the data of these studies showed that there was a tripling or quadrupling of the CVT mortality rate in the experimental group, a result that better reflects the risk in clinical practice. Ultimately, this error led to 50000 to 79000 cases of serious coronary heart diseases that could have been prevented if the drug had been withdrawn 39 months sooner, when the intent to treat analysis started to show a significant increase in the CVT events [4-6].

Question 7

Please provide us with the protocol definition of the clinical picture that would justify a hospitalization: Hospitalizations are increasingly being used as outcomes, often as part of the combined outcome together with death from all causes and other serious outcomes such as stroke and myocardial infarction. These are clearly not equivalent, at least from the patient’s point of view. From the trialist point of view this has two advantages. Firstly, it increases the number of events and facilitates adequate power. Secondly, since hospitalizations are expensive, these parameters facilitate the deployment of an economic study that would show that, in spite of its high acquisition price, the intervention produces some savings to the health care system. The hospitalizations undergone by patients are frequently assumed by researchers to be a surrogate for worsening disease. However, the reasons for hospitalizing a patient (social or economic) can be unrelated to the progression of the disease. Furthermore, the threshold for hospitalizing a patient varies in different geographic jurisdictions, clearly a problem since many large trials are multicenter and multinational. Another source of asymmetry is that while the other clinical outcomes, such as non-fatal myocardial infarction or stroke are carefully validated by an adjudication committee, this might not be the case for the hospitalizations.

Question 8

Please show us the censored patients as click marks on the Kaplan-Meyer curves: Kaplan-Meyer curves have become very popular since they have many statistical virtues in addition to being very 'photogenic'. Survival time for each subject is the time from entry into the trial until the occurrence of the event of interest, the censoring of the patient because the follow-up period ends or until he voluntarily leaves the study or is lost to follow up. The amount and distribution in time of censored subjects is important. If many patients are censored in the study, it raises questions about the validity of the study, how it was executed and the effects (good or bad) of the treatment on the patients. This is particularly true if many patients are censored early in the trial.

Question 9

Independently of the method actually used please also show us the results obtained with the mean, median and mode imputation method: To comply with the intent to treat principle it is sometimes necessary to imputed missing values due to withdrawals or patients lost to follow up. Independently of the method used, one of the inevitable problems with imputation is that given that the imputed values are estimates, the variance of the end result is underestimated. There are several methods to derive imputed values, some of them with folkloric names evocative of Las Vegas or Montecarlo such as the Hot-deck and the Cold-deck. A method which, while not perfect, is easy to understand and very transparent is to impute to the missing value the mean, median and mode of its corresponding population.

Question 10

Please provide us with a description of the clinical advantages of the product being tested, outside of its efficacy profile, which justify the size of the delta: As therapeutics improves, the use of placebos in clinical trials is becoming less ethical, which results in the increasing popularity of non-inferiority trials.

Conclusion

The aim of non-inferiority trials is to determine whether the efficacy of a new therapeutic intervention is not less than an active comparator by more than a previously defined noninferiority margin known as delta. The choice of the magnitude of the delta is thus crucial to the validity of the trial. The selection of the delta must be based on both statistical and clinical reasoning. Independently of statistical considerations, as a clinician, my main question are whether the purported benefits of the new therapeutic intervention, outside its efficacy, justifies taking the risk of a potential decrease in the expected benefit.

References

1. Smeeth L, Douglas I, Hall AJ, Hubbard R, Evans S. Effect of statins on a wide range of health outcomes: A cohort study validated by comparison with randomized trials. Br J Clin Pharmacol. 2009;67(1):99-109.

   [Cross ref] [Google scholar]

2. Calabresi PA, Kieseier BC, Arnold DL, Balcer LJ, Boyko A, Pelletier J, et al. Pegylated interferon beta-1a for relapsing-remitting multiple sclerosis (ADVANCE): A randomised, phase 3, double-blind study. Lancet Neurol. 2014;13(7):657-665.

   [Cross ref] [Google scholar]

3. Hwu CM, Ho LT, Fuh MMT, Siu SC, Sutanegara D, Piliang S, et al. Acarbose improves glycemic control in insulin-treated Asian type 2 diabetic patients: Results from a multinational, placebo-controlled study. Diabet Res Clin Practice. 2003;60(2):111-118.

   [Cross ref] [Google scholar]

4. Madigan D, Sigelman DW, Mayer JW, Furberg CD, Avorn J. Under-reporting of cardiovascular events in the rofecoxib Alzheimer disease studies. Am Heart J. 2012;164(2):186-193.

   [Cross ref] [Google scholar]

5. Begg CB. Significance tests of covariance imbalance in clinical trials. Contr Clin Trials. 1990;11(4):223-235.

   [Cross ref] [Google scholar]

6. Bothwell LE, Podolsky SH. The emergence of the randomized, controlled trial. New Eng J Med. 2016;375(6):501-504.

   [Cross ref] [Google scholar]