One of the most important yet difficult issues we all face with the proliferation of medications available and in development of new drugs, especially those affecting the central nervous system, is how to determine what is a meaningful drug effect.  How does one know which drugs are effective and which are not?  What is a meaningful treatment effect?  How do professionals decide which drugs to recommend?  Are newly available drugs better than older medications?

We rely on large studies called randomized clinical trials (RTCs) to answer these questions.  RTCs are scientifically rigorous studies designed to evaluate the safety and efficacy of medications (often compared to a placebo). The problem comes when we try to interpret the findings from these studies.  What constitutes a meaningful result depends upon whom you ask (Defining a clinically meaningful effect for the design and implementation of randomized control trials, Innovations in Clinical Neuroscience, 2013, 10, 4S-19S).

The issue from the clinicians’ and researchers’ perspective is actually simple.   The drug in question must be “significantly” better than the placebo or comparative medication.  The concept of what is significant is based on well-accepted decision rules from standardized statistical design and analysis.  The concept of statistical significance is derived from defining a priori the risk of making the mistake of saying that two drugs differ in effectiveness when in reality they do not.

The complication is that a treatment effect may be significantly different from a placebo but trivial and therefore not meaningful. This is especially true of many of today’s studies that rely on huge samples sizes.  For example, a well-designed study with a large sample, say 10,000 patients, will facilitate discovery of statistically significant results that may be trivial. The definition of meaningful is elusive and depends on whom you ask.

The perspective of who pays for the medication.   Representatives from the federal government and insurers define meaningful effects based on what medications should be covered.  Payers are not impressed with mechanisms of action, new pathways, or meeting unmet needs.   They define meaningful based on cost and economic value.

The perspective of investors.  Quit simply investors define meaningful in terms of how many people are willing to buy the new drug.

The perspective of the FDA.  Approval is a complex decision process based on “substantial” evidence. There are no definitions, regulations, or universal agreements for defining what constitutes substantial evidence.

The perspective of the patient.  The patient’s perspective on meaningful is the most neglected aspect of this formula.  Patients participate in clinical trials under the assumption that the study is designed to determine the most beneficial treatment, which is not the objective of an RTC.  The drug consumer defines beneficial based on the treatment’s ability to improve their quality of life without doing harm.

In short, there is no consensus of how to determine meaningful treatment effects.  This is a complex issue that I have thought about for years.  I still don’t know how to best answer the question of meaningful treatments.  My hope is that we do a better job in the future of finding a way to incorporate the person into future formulas.