Find A PhysicianHome  |  Library  |  myDownstate  |  Newsroom  |  A-Z Guide  |  E-mail  |  Contact Us  |  Directions
curve gif
Next Page
Course Contents
Section Contents
Next Section

CRITICAL ANALYSIS

THERAPY



The three main questions to ask yourself about any report before applying its findings to your patients are:

1) Are the results valid?
2) What are the results?
3) Will the patients help me with my individual patient?

The data required to answer these questions varies between the domains and, depending upon the type of question, the types of evidence used.

Are the results valid?

When reading the results of a single clinical trial, the meausure of validity is whether the trial lives up to the expectations of a randomized, double-blinded, placebo-controlled clinical trial.

RANDOMIZATION: Almost all reports of randomized clinical trials have a Table 1.  This table describes the characteristics of the participants in each arm of the trial.  By checking this table, you can make sure that patients were properly randomzied (that a particular characteristic is not represented more in one group than another).  In the event that pure chance has placed more people with one characteristic (e.g. more men) in one group than another, you want to make sure that these differences were accounted for.  The usual process for this is a sub-group analysis.  Researchers may recompute the necessary statistics accounting for the larger representation in one group.  If the researchers do not report doing a sub-group analysis or present the characteristics of their patients, the validity of the study may be called into question.

BLINDEDNESS:  It is vital that all participants in a study be blinded to the true nature of the "treatment" being received.  Clinicians who know whether their patients are taking the true treatment or the placebo may treat their patients differently and thus affect the outcome of the trial.  Patients who know they're getting the placebo may not wish to continue the trial.  Study personnel collecting data may be inclined to prompt for specific comments if they know the person they're interviewing is taking the treatment being studied.  Naturally, there are times when blinding is impossible.  It will be difficult to fool someone into believing they've undergone surgery when they haven't.  So allowances must be made.  That said, if there is no discussion of blinding, the validity of the study may be called into question.

FOLLOW-UP: All patients must be accounted for at the end of the trial. They must be analyzed in the groups to which they were randomized.  It happens, sometimes, that patients are lost to follow-up for reasons other than those related to the purpose of the trial (a death clearly counts as a failure on the part of the treatment).  These patients must still be accounted for.  An intention-to-treat analysis assigns those patients lost to follow-up to the group to which they were originally randomized.  It counts those patients as failures of the treatment.  If there is no mention of an intention-to-treat analysis, the validity of the study may be called into question.


What are the results?

The results typically reported in an article on therapy are Absolute Risk Reduction (or Number Needed to Treat) and the  Relative Risk Reduction.  At the very least, the data required to compute these numbers should be reported.   You will also want to check the precision of the results.  This is given by the confidence intervals for all both numbers.
For a further discussion of these statistics, see the Important Concepts section on Therapy.

Will the results help me in caring for my patients?

In addition to the concerns suggested earlier , there are two issues to be aware of when reading an article on therapy.

OUTCOMES:  When we discussed formulating a clinical question , we stressed the need to focus on clinically significant outcomes rather than lab tests.  However, if  the study used lab tests as their outcome, you need to be sure that these "substitute endpoints" are valid.

BENEFITS:  Any decision to institute therapy should weigh the possible benefits against the possible harms.  The data that most helps you to do this is the Number Needed to Treat.  

The University of Alberta provides a worksheet to analyze articles on therapy.



These criteria apply mainly to the report of a single clinical trial?  If the article you are reading is a systematic review or meta-analysis, then you will also need to take the following information into consideration.

COMPLETENESS: Were the criteria used to select article for inclusion appropriate?  We have already discussed the importance of only using clinical trials for questions about therapy.  If other types of articles were included (e.g. Cohort Studies), was there a good clinical reason for doing so?  Is it unlikely that important, relevant studies were missed?  The authors will have had to sufficiently describe the process for finding studies and the reasons for inclusion/exclusion to answer these questions.

VALIDITY: Were individual studies checked for validity?  Was the criteria for confirming validity adequately explained?

HOMOGENEITY: This is particularly important for meta-analyses.  Were the patients in each study sufficiently similar that conclusions from one study could be applied to the patients of another?

 The University of Alberta provides a worksheet to analyze systematic reviews.


Next Page
Course Contents
Section Contents
Next Section