During the course of conducting your
systematic review, you should go through each of the following steps:
- Developing the topic and refining your
PICO
When you begin your systematic review, you will want to think about
the population, interventions, comparators, and outcomes that you are
interested in examining. You can then use these criteria to help develop your
Key Question(s), which will serve as the overall guiding force behind your
systematic review. To learn more about developing your PICO and Key Question,
check out the The MUSM
Skelton Medical LIbraries EBM site.
- Determining your study eligibility
criteria
In addition to your PICO, you will want to determine which other
criteria you want each included study to have, or which criteria will make a
study ineligible for your review. A few criteria to consider include study
design, dates of interest, whether you will include non-English studies, study
duration, and setting.
- Searching for studies
In this step, you
will design a comprehensive search strategy to capture the majority of the
literature on your given topic. You can use your PICO as the starting point
when brainstorming for terms to search. Once you have created your initial
search, you will also want to decide which databases (e.g. PubMed, Web of
Science, Cochrane, etc.) to search and translate your search for those
databases. In addition to the published literature, you will also want to
consider searching for unpublished or "grey" literature. Grey literature can be
found in many sources, including manufacturers websites, the FDA website for
drug information, and clinical trial registries like
ClinicalTrials.gov and the WHO International Clinical Trials
Registry Platform.
- Creating and registering your protocol
Your protocol should contain all of the steps outlined above and a plan for all
of the steps outlined below. You can register your protocol at
PROSPERO, to ensure that
someone else starting a review will not be duplicating your efforts.
- Screening and selecting studies
This
step includes both a preliminary review of all titles/abstracts to determine
which studies are potentially eligible, and an additional, more in-depth review
of the full text for all includes from the title/abstract phase to determine
final eligibility for the review. This step will require two people to review
each abstract and each full-text document, plus an additional reviewer to serve
as a tie breaker.
- Data extraction
You can use your PICO as a guide
for the data you want to extract, including information on the study
population, intervention or exposure, comparator (if applicable), outcomes
data, timing of data, and study design. There are a number of tools
that you can use for this step, ranging from a basic Excel setup to more
sophisticated tools designed for systematic review data abstraction, like AHRQs
Systematic Review Data Repository (SRDR)
and Covidence. To complete this step,
you will need at least one team member to go through and extract all relevant
data from each included study and one team member to review the data they have
extracted.
- Quality assessment
In this step, you
will assess the internal validity of each study, including systematic errors
(e.g. selection bias, confounding), nonsystematic errors (errors attributable
to chance), and inferential errors (problems in data analysis and
interpretation). Depending on the types of studies you are examining, you may
want to select a pre-existing quality rating tool for each study type. A sample
of quality rating tools can be found in the box to the right.
- Assessment of applicability
In this
step, you will assess the external validity of each study (how well the results
of the study are likely to reflect real-world outcomes), as well as the
external validity of the body of literatures as a whole. You can also use your
PICO to determine the most important factors affecting applicability. Some
issues to consider here are how narrow the exclusion criteria were within each
study, differences between patients in the study and in the community, whether
or not a medication regimen is reflective of current practice, and whether an
interventions intensity would be feasible for routine use.
- Presentation of findings
In this step,
you will develop your evidence tables, which give detailed information for each
study (using your PICO as a guide), and summary tables, which give a high level
overview of the findings of your review. You can create evidence and summary
tables to describe study characteristics, results, or both. These tables will
help you determine which studies, if any, are eligible for quantitative
synthesis.
- Quantitative synthesis
Quantitative
synthesis, also known as meta-analysis or pooling, is the process of combining
data from multiple studies in order to gain a sense of the overall effect of a
treatment or exposure. When determining whether or not a meta-analysis should
be undertaken, great care should be taken to determine whether or not the
studies are similar enough to analyze together, and whether or not an analysis
will provide potentially misleading results. There are a number of tools that
can be used to conduct a meta-analysis, including
RevMan,
CMA, and
Stata.
- Grading the strength of evidence
Strength of evidence grades are used to evaluate the strength of the entire
body of evidence for major outcomes and comparisons. Strength of evidence is a
single grade based on four domains: risk of bias, consistency, directness, and
precision. For more information on grading strength of evidence, check out
the Grading of Recommendations
Assessment, Development and Evaluation (GRADE) Working Group.
- Writing your review
Your final review
manuscript should contain the following sections: title, abstract,
introduction, methods, results, discussion, and disclosure of funding. In
addition to these main sections, you should also consider including appendices
with your complete evidence tables and quality assessment forms and a
line-by-line description of your search methodology.
For additional, step-by-step guidance, check
out the
AHRQ
Training Modules |