Assessing Benefit/Risk of Antipsychotic Treatment: An Evidence-Based Approach

 

John Kane, MD
Chairman, Department of Psychiatry
The Zucker Hillside Hospital
Vice President for Behavioral Services
North Shore – Long Island Jewish Health System
Glen Oaks, New York

View Clinical Issues videoView Clinical Summary video

The expanding armamentarium of antipsychotic medications, while increasing treatment options, has introduced complexities with regards to choosing the optimal treatment. Clinical decision-making regarding choice of treatments cannot be made by simple comparisons of efficacy and adverse events of medications, as these can be misleading. For instance, sometimes relative results are reported without the absolute results, thus failing to put the data into perspective. Sometimes benefits are minimized because of excessive concerns about adverse events. Without assessing the benefits of drugs, without knowing the absolute risks and attributable risks of drugs, and without taking into consideration the specifics for each patient, a confident general decision regarding treatment cannot be made in advance. There is, therefore, a need for individualized benefit/risk considerations before making clinical decisions.

Our patient Tom presented to the emergency room with a number of different challenges—co-occurring substance use and comorbid medical conditions, in addition to psychotic symptoms. The clinician has to decide on the best strategies to treat Tom not only while he is in the hospital but also after his discharge to the outpatient setting. Tom's doctor also has to evaluate and reevaluate whether the treatments that he had initiated are optimal for Tom, and this requires a thorough understanding of evidence-based benefit/risk decision making.

This article will provide a framework for benefit/risk decision making in clinical practice. We will discuss some of the statistical strategies that are used in calculating the relative benefit and relative risk estimates that inform clinical judgments. It is hoped that this discussion will help clinicians educate their patients about the basis of benefit/risk judgments so as to promote shared decision making.

Benefit/Risk Decision Making—Past and Present

In the past, clinical psychiatry had not incorporated an operationalized approach to benefit/risk decision making, because treatment options were scarce and the stakes were high (harm to self or others) if psychotic conditions were not brought under rapid control. Thus, a lot of risks were tolerated in order to get immediate benefits. With current neuroscientific breakthroughs, treatment options have increased. It is now easier to be "choosier" and consider benefits and risks. Medico-legal forces in our society have forced the focus on benefit/risk of treatments. Additionally, consumers rightly expect a comprehensive benefit/risk approach in clinical decision making. Psychiatry, thus, has had in some respects to "catch up" with other areas of medicine. Traditional concepts used in benefit/risk decision making such as P-values and statistical significance are no longer adequate.

The P-value indicates whether or not a finding is likely to have occurred by chance.^[1,2] The smaller the P-value, the more likely that the finding is not just random. A P-value of < 0.05 is usually considered to be "statistically significant", which means that the finding has a less than 5% probability of occurring by chance alone. The P-value, however, does not state anything about the size or the importance of the nonrandom effect, the clinical significance of the finding, or the effect size.

Statistical significance does not necessarily equal clinical significance. In contrast, a difference that is not statistically significant may still be clinically meaningful, particularly if the trend is obvious, and the sample size is too small to draw statistical conclusions. Nonsignificance also does not necessarily mean that 2 treatments are equivalent. Nonsignificance can result from an inadequately powered study due to a sample size which is too small to detect a difference, or having had too many subjects drop out of the study. To demonstrate adequate power, a threshold of clinical significance—an effect size below which the treatment versus control difference would not be considered clinically significant—needs to be declared at the outset of a study. Thus, a finding of statistical significance in an adequately powered study should also indicate clinical significance.

Effect size is independent of P-value and does not provide information about the likelihood of the difference between treatments being due to chance alone. It does, however, tells us how clinically meaningful a statistically significant result is. Common effect size calculations include the odds ratio (relative risk) and the number needed to treat (NNT).

Clinicians need to be aware that although the P-value is probably the most common statistical measure reported in studies, it often does not provide sufficient information to make benefit/risk clinical decisions. Availability of absolute risk, relative risk, and attributable risk information concerning particular medications would be more informative in this regard.

Absolute Risk, Relative Risk, and Attributable Risk Concepts

Absolute risk, also known as incidence, is the number of cases of interest in a particular group over time (Table 1).^[2,3] Absolute risk is important because it tells the probability of encountering the event and is calculated by taking the number of times an event occurs over a period and dividing it by the total population at risk during that period. Relative risk, also referred to as odds ratio or hazard ratio, is the ratio of 2 absolute risks, and it describes the potential relative effect, for example, of a drug exposure on the risk of an event. Attributable risk is commonly referred to as absolute risk reduction, or ARR. It describes the excess potential risk associated with exposure to a particular drug or treatment. Attributable risk is calculated by taking the absolute risk of one drug or treatment and subtracting the absolute risk of the other drug or treatment.

Table 1. Understanding risk calculations^[2-4]

A fictitious example can best illustrate how to distinguish between these concepts. Say the risk of developing breast cancer in the next 5 years for a woman aged 55-65 years who does not take aspirin is 20/1000, or 2%. This would be the absolute risk. In other words, women aged 55-65 years who do not take aspirin have a 98% chance of remaining free of breast cancer over the next 5 years. However, if these women take an aspirin a day, the risk declines by 20% to 16/1000. This is a relative risk reduction of 20%, which now results in an absolute risk of 1.6%. This means that women who take an aspirin daily have a 98.4% chance of being cancer free over 5 years. We could also look at this in terms of overall relative risk, which would be 1.6%/2%, or 0.8. A relative risk of 0.8, which is less than one, indicates that the risk in the aspirin group is less than that of the nonaspirin group, so aspirin has a protective effect against developing breast cancer over the next 5 years, which sounds fairly significant. However, when we look at the ARR, or attributable risk, which is calculated by taking the absolute risk of 2% in the nonaspirin group minus the absolute risk of 1.6% in the aspirin group, we get 0.4%, which really doesn't sound as "significant" as we originally thought. So women aged 55-65 years who take an aspirin decrease their risk of developing breast cancer in the next 5 years by only 0.4%. This is not a big change at all. This example illustrates the importance of differentiating among absolute risk, relative risk, and attributable risk.

Epidemiological Benefit/Risk Concepts—NNT and NNH

The number needed to treat (NNT), derived from ARR, has become a popular measure of the effectiveness of interventions. It is considered the least misleading and the most clinically useful measure of treatment effectiveness.^[2] NNTs are also much easier to comprehend than some statistical descriptions, and NNTs for different agents can be easily compared. NNT indicates how many more patients one would need to treat with an experimental drug instead of the standard or control drug in order to see one more patient benefit (Table 1). NNT is calculated as the inverse of ARR (1/ARR). It is desirable to have a small NNT, as this would indicate that the relative advantage of one drug over another is large; thus, one has to treat fewer patients to see one additional case of response or benefit than if the alternative medication had been used. NNT may be presented as per 100 patients. Table 2 provides an example of how to perform an NNT calculation.

Table 2. Calculation of NNT using completion of therapy as the beneficial event (fictitious example)

If the NNT is negative, then the benefit is associated with the standard or control drug. A negative NNT value is also known as the number needed to harm (NNH).^[4] NNH is the reciprocal of the absolute risk increase (ARI) (1/ARI). It is a measure of the effect size of a difference in adverse events between 2 treatments. It indicates that for every so many patients treated with the experimental drug, one additional adverse event would be expected if treated with the control drug. It is desirable to have a large NNH because this would mean that one has to treat a large number of patients before one more adverse event with the experimental drug than with the control drug is seen. NNH may be presented as per 100 patients. Table 3 provides an example of how to perform an NNH calculation.

Table 3. Calculation of NNH using 7% weight gain as the harmful event (fictitious example)

NNT and NNH are, however, not without limitations. They are most valid when calculated from a randomized controlled trial with identical conditions for all drugs/interventions under study.^[5] As always, the overall quality and generalizability of the trial will determine the value of its findings. NNT or NNH or any other statistical technique cannot overcome the shortcomings of a particular trial. As NNTs are sensitive to factors that change the baseline risk, such as the outcome considered, patients' characteristics, secular trends in incidence and case fatality, and the clinical setting, NNTs cannot be compared across studies with different populations as in pooled analysis.^[6] Moreover, NNT and NNH are only calculable for binary or dichotomous events that are either present or absent, such as treatment response (yes/no), remission (yes/no), or avoidance of hospitalization (yes/no), and do not apply to continuous variables such as the value of a blood test. However, values with clinically significant thresholds, such as an adverse effect described as "weight gain > 7%," can be expressed as NNH because they are binary.^[5] Lastly, NNT and NNH also do not take into consideration the patient's values about the treatment and its potential risks and benefits.^[7]

Applying NNT to Clinical Decision Making in Schizophrenia

The relative ability of first and second generation antipsychotic drugs to prevent relapse rates in patients with schizophrenia was analyzed via a systemic review of randomized, controlled trials comparing the 2 agents with a study duration of at least a year.^[8] The analysis revealed that rates of relapse were modestly but statistically significantly lower with the SGA agents (15% vs 23%, P = 0.0001) (Figure). The ARR was 8% and the NNT was 13. An NNT of 13 indicates that if 13 more patients were treated for one year with SGA agents instead of FGA agents, one more relapse would be avoided. The absolute reduction of the relapse risk by 8%, ie, 80 relapses prevented per 1000 patients treated for one year, is as strong as the evidence supporting the use of aspirin to prevent vascular events,^[9] which is now a widespread clinical practice.

Figure. Relapse rates with second generation antipsychotic (SGA) agents compared with first generation antipsychotic (FGA) agents

Adapted from Leucht S et al. 2003.^[8] References cited in figure: Marder SR, et al. Am J Psychiatry. 2003;160:1405-1412; Csernansky JG, et al. N Engl J Med. 2002;346:16-22; Daniel DG et al. Psychopharmacol Bull. 1998;34:61-69; Speller JC, et al. Br J Psychiatry. 1997;171:564-568; Tamminga CA, et al. J Clin Psychiatry. 1994;55(suppl 9):102-106; Essock SM, et al. Psychopharmacol Bull. 1996;32:683-697; Rosenheck R, et al. Schizophr Bull. 1999; 25:709-719; Tran PV, et al. Br J Psychiatry. 1998;172:499-505.

A meta-analysis of randomized, controlled trials comparing SGA agents with FGA agents of one-year duration.

Presenting the data as NNT and ARR, thus, gives clinicians a better perspective of the benefit of using SGA agents, which could be used in clinical decision making.

CLINICAL COMMENTARY

The prevention of relapse is a key public health challenge that clinicians face in patients with schizophrenia. The consequences of relapse are devastating. Psychotic relapse—from a biological standpoint, from a psychosocial standpoint, and from a psychodynamic standpoint—has far reaching consequences and should be avoided.

Individualizing Clinical Trials Data to Patients—The Future

While the above discussion provides a basis for clinicians to understand and interpret clinical data, they need to judge how to apply the evidence to individual patients. Several studies, including a Cochrane Report, are now available that show how this can be done.^[4,10] Clinicians should refer to these studies to guide them in individualizing benefit/risk data to their patients.

A lot of factors need to be taken into consideration in individualizing benefit/risk decisions. For example, our patient Tom presented with a number of challenges—co-occurring substance use and comorbid medical conditions, in addition to psychotic symptoms. Antipsychotic treatment selection for Tom needs to take into consideration his comorbid conditions. At the same time, Tom's particular preferences for medication need to be considered so as to facilitate shared decision-making. All of these require that the clinician (treating Tom) has a good understanding of clinical evidence (benefit/risk) supporting recommended treatments, is able to communicate that evidence to Tom in a form that he can readily understand, and is prepared to continually reevaluate that clinical decision using established measurement tools. The clinician should also take care not to reach premature closure on any of the treatment decisions and should think about treating Tom's primary illness and his comorbid conditions utilizing an integrated treatment system. When bringing in other modalities of treatment for Tom in conjunction with antipsychotic medication, the clinician should not be sidetracked by the most salient problem that Tom might have on a particular day. Clinicians, in general, often make the mistake of focusing on a particular psychotic sign or symptom and not appreciating the potential side effects and the potential long-term consequences of the treatment. Clinicians should constantly remember they are dealing with a complex illness requiring numerous evidence-based and shared decisions that need to be made on an ongoing basis.

CLINICAL COMMENTARY

Shared decision making depends on presenting the relevant benefit/risk numbers in a form that patients can readily understand and on having a properly informed and educated patient. This is especially critical in schizophrenia, where patients may have cognitive difficulties that can make these tasks more challenging. Additionally, benefit/risk numbers should be relevant to the phase of the illness that the patient is in. For example, when dealing with Tom in the emergency room, clinicians are not necessarily going to have discussions about the long-term risks of antipsychotics. But, as Tom advances in his treatment and enters into a phase of resolution of acute symptoms or remission, discussions about long-term decision making may be more relevant.

Moving forward, clinical studies need to present data as effect size, NNT and ARR so as to provide a better perspective of the data for clinical decision making. Moreover, presenting data in these formats would also contribute to the growing awareness of these measures in clinical psychiatry.