Cook CE and Thigpen CA. Five good reasons to be disappointed with randomized trials. JMMT: 2019
- RCTs are recognized to exhibit very high levels of evidence. They are useful in testing the efficacy and effectiveness of interventions between groups. However, there are limitations that can influence clinical practice
- Reason 1: Right question- wrong design: RCTs are a form of research design and not appropriate for all forms. For example, diagnostic accuracy studies are best analyzed using a case-based, case-control design.
- Reason 2: The marginal patient. A well-known limitation of an RCT is external validity- essentially the degree to which the conclusions in the study are applicable to your patient.
- Because subjects are so well ‘controlled,’ a homogenous sample of diagnostically similar patients may not represent the actual demographics and complexity in the clinic.
- Reason 3: mixed treatment effect- just because one group reports better outcomes does not mean that the intervention works for all individuals in that group or future groups.
- Some individuals in both groups improve while some individuals in both groups do not. An RCT only functions to show whether more people in one group improved compared to another, not why someone improved.
- Reason 4: treatment fidelity- the application of the intervention for the condition of interest. Were the interventions appropriately performed (application, dosage, intensity, etc) and whether the interventions adequately represent how the intervention is performed in clinical practice
- Reason 5: unmeasured bias- randomization is used to reduce errors but the post-randomization experience is where bias can play a role. There are other considerations including the Hawthorn effect, patient beliefs and expectations, personal equipoise, and contamination bias.