Step 3: Appraise the Evidence
Why do you need to appraise the evidence? It is important to note that just because an article is published in a scientific peer review journal is not an assurance the study is of high quality. Likewise, a systematic review may have flaws or may not be appropriate to your question. Therefore, you want to be aware of factors that may impact the significance of your appraisal.
For instance, who is the author of the systematic review or individual research article? Are they biased? Do they have a vested interest? Is the authors question relevant to your question?
How do you know which articles provide the best evidence?
Clinicians often use a Critical Appraisal Form (CAT) as a strategy to appraise research articles with a systematic and organized mindset, keeping track of the evidence. A CAT review is an important bottom line summary of the research determining how effective and useful this research is to your PICO question (MacDermid & Law, 2008)[12]. However, this summary is only as good as your appraisal. At this time, you may want to go to the CAT tab and download the form here. You may find it helpful to print the form and follow along while reading this section.
To summarize the appraisal process, the overall objectives of your appraisal are to determine the quality and clinical utility of the study evidence. In order to achieve these objectives, you need to know the various research study designs and levels of evidence quality rankings. We will refer to the classic levels of evidence described by MacDermid and Law (2008)[12]. However you may be interested in the Oxford Centre For Evidence-Based Medicine website where you will find an updated detailed version. [47]
Why do you need to appraise the evidence? It is important to note that just because an article is published in a scientific peer review journal is not an assurance the study is of high quality. Likewise, a systematic review may have flaws or may not be appropriate to your question. Therefore, you want to be aware of factors that may impact the significance of your appraisal.
For instance, who is the author of the systematic review or individual research article? Are they biased? Do they have a vested interest? Is the authors question relevant to your question?
How do you know which articles provide the best evidence?
Clinicians often use a Critical Appraisal Form (CAT) as a strategy to appraise research articles with a systematic and organized mindset, keeping track of the evidence. A CAT review is an important bottom line summary of the research determining how effective and useful this research is to your PICO question (MacDermid & Law, 2008)[12]. However, this summary is only as good as your appraisal. At this time, you may want to go to the CAT tab and download the form here. You may find it helpful to print the form and follow along while reading this section.
To summarize the appraisal process, the overall objectives of your appraisal are to determine the quality and clinical utility of the study evidence. In order to achieve these objectives, you need to know the various research study designs and levels of evidence quality rankings. We will refer to the classic levels of evidence described by MacDermid and Law (2008)[12]. However you may be interested in the Oxford Centre For Evidence-Based Medicine website where you will find an updated detailed version. [47]
Classic Levels of Evidence (Adapted from MacDermid & Law, 2008, p. 124)
When you are ready, please peruse the levels of evidence and study design resources available on Appraising the Evidence Resource page for further detailed information.
After you are comfortable with study design and levels of evidence questions you will want to consider validity and reliability
factors in the evidence.
What is validity?
Internal validity is how well the research shows the cause-effect relationship between the independent (intervention provided) and dependent (outcomes) variables. In other words how strong is the evidence that the intervention caused the outcomes.
External validity refers to how well the evidence generalizes from the sample of participants to the specific population in the larger population. For instance, if a study population is people with multiple sclerosis, how well does this population generalize to the larger population as a whole.
Follow this link to learn more about validity
[13] http://guides.mclibrary.duke.edu/content.php?pid=431451&sid=3718368
What is reliability?
Basically, reliability is the range in which the results of the study are replicable. This means the researchers described their methodology in detail to enable the intervention to be repeated with similar results, by you for instance.
Follow this link to learn more about reliability
[38] http://www.socialresearchmethods.net/kb/reliable.php
Statistical and Clinical Significance
In your readings you will be looking for statistical significance. Generally, significance implies that there is a low probability
that the outcome results from the intervention did not happen by chance. Outcomes with statistical findings of p ≤ .05 are considered statistically significant. However, it is important to question if this significance is clinically significant. Clinical significance involves your judgment of all the factors around your client and the study. Does the outcome show real life
improvement for your client. For instance, we may know that doing a home exercise program may strengthen muscles however if this program does not improve function is it clinically significant?
Now that you have an understanding of appraising the evidence you are ready for Applying the Evidence.
