This article guides students through the first three critical steps of a systematic review: crafting a focused research question, developing a reproducible search strategy, and executing a transparent study selection process. Whether you are planning your first review or refining your methodology, these practical steps will help you build a strong foundation.
The Foundation: Formulating a Clear Research Question
A well-defined research question is the backbone of any systematic review. It directs every subsequent decision, from search terms to inclusion criteria. Without a precise question, your review risks becoming unfocused or unmanageable.
- Use a structured framework such as PICO (Population, Intervention, Comparison, Outcome) to break your topic into core components.
- For qualitative reviews, consider PICo (Population, phenomenon of Interest, Context) or SPIDER.
- Ensure the question is answerable through existing literature and is neither too broad nor too narrow.
- Test your question by running a quick preliminary search – if the result is overwhelming or empty, refine it.
Example: Applying PICO
Suppose you are interested in whether mindfulness-based therapy reduces anxiety in university students. Using PICO:
- Population: University students diagnosed with anxiety disorders.
- Intervention: Mindfulness-based cognitive therapy (MBCT).
- Comparison: Usual care or no intervention.
- Outcome: Reduction in anxiety scores measured by a validated scale.
This yields a focused question: “In university students with anxiety, does mindfulness-based cognitive therapy reduce anxiety scores compared to usual care?”
“A systematic review is only as good as the question it asks. Spend time sharpening your research question before you run a single search.” — Adapted from the Cochrane Handbook for Systematic Reviews of Interventions.
Constructing a Comprehensive Search Strategy
Your search strategy must be systematic, transparent, and reproducible. It is the step that distinguishes a systematic review from a simple literature summary. The goal is to retrieve all relevant studies, not just the ones you already know.
- Identify the key concepts from your research question (e.g., population, intervention, outcome).
- List synonyms, alternative spellings, and related terms for each concept.
- Use Boolean operators: AND to combine different concepts, OR to group synonyms, and NOT only when you are certain it will not remove relevant records.
- Select appropriate databases (e.g., PubMed, Scopus, Web of Science, PsycINFO) and adapt your search string for each platform’s syntax.
- Include a search for grey literature (preprints, conference abstracts, theses) to reduce publication bias.
Building the Search String
For the mindfulness and anxiety example, a basic search string might look like:
(“mindfulness-based cognitive therapy” OR “MBCT” OR “mindfulness therapy”) AND (university student* OR college student* OR undergraduate*) AND (anxiety OR anxious*)
Document every search exactly as run, including filters and date limits. This allows others (and your future self) to replicate the process.
Study Selection: Defining Eligibility Criteria
Eligibility criteria (inclusion/exclusion) operationalize your research question and determine which studies enter your review. These criteria must be objective, clearly stated, and applied consistently.
- Define criteria based on: study design, population characteristics, intervention/exposure details, comparators, outcomes measured, and publication date.
- Specify language restrictions only if you have the resources to manage them; otherwise, aim for no language limits.
- Exclude studies that are clearly irrelevant (e.g., opinion pieces, reviews without original data) but be careful not to pre-judge methodological quality at this stage.
- Pilot-test your criteria on a sample of 20–30 abstracts to ensure they capture what you intend.
Example Eligibility Criteria
| Criterion | Inclusion | Exclusion |
|---|---|---|
| Study design | Randomized controlled trials (RCTs) | Case reports, editorials |
| Population | University students (≥18 years) with anxiety diagnosis | Children, clinical patients outside university |
| Intervention | MBCT delivered in person or online | Other therapies combined with MBCT |
| Comparator | Control group (waitlist, placebo, no treatment) | Single-arm studies |
| Outcome | Anxiety measured by validated scale (e.g., GAD-7, BAI) | Only qualitative outcomes |
| Language | English only (if resources limited) | Non-English records |
Screening: Title/Abstract and Full-Text Stage
Screening is typically done in two rounds. First, you examine titles and abstracts to remove obviously irrelevant records. Then, you retrieve and screen the full text of the remaining studies against your eligibility criteria.
- Use at least two independent reviewers to reduce bias; calculate inter-rater agreement (e.g., Cohen’s kappa) to check consistency.
- Create a screening form or use systematic review software (Rayyan, Covidence) to track decisions.
- At the title/abstract stage, be inclusive – if there is any doubt about relevance, keep the record for full-text review.
- Document the reason for each exclusion at full-text stage (e.g., wrong population, wrong design).
- Update your search just before publication to capture any new studies.
“Screening is not about proving your hypothesis – it is about finding every study that meets your criteria, even the ones that might challenge your assumptions.” — Adapted from PRISMA 2020 statement.
Common Pitfalls and How to Avoid Them
Students often make avoidable mistakes during the question, search, and selection phases. Being aware of these can save weeks of rework.
- Pitfall: A question that is too broad leading to thousands of hits. Solution: Use PICO and pilot test.
- Pitfall: Relying on only one database. Solution: Search at least two core databases plus a specialized one.
- Pitfall: Not saving search strategies. Solution: Copy-paste the exact search strings into a methods appendix.
- Pitfall: Screening with only one reviewer. Solution: Recruit a colleague or supervisor as second reviewer.
- Pitfall: Applying eligibility criteria inconsistently. Solution: Develop a detailed codebook and calibrate with your team on 20 tests.
Conclusion
Mastering the research question, search strategy, and study selection process is the most demanding – and most rewarding – part of any systematic review. A clear question ensures your review answers something meaningful. A thorough search ensures you find all relevant evidence. And a transparent selection process ensures that your results are trustworthy and reproducible. By following these steps with care, you set yourself up for a review that can genuinely inform practice and policy.
Frequently Asked Questions
1. What is the difference between a systematic review and a literature review?
A systematic review follows a predefined, rigorous methodology to identify, appraise, and synthesize all available evidence on a specific question. A traditional literature review is often less structured and may selectively cite studies that support the author’s argument.
2. How do I choose between PICO and PICo for my question?
Use PICO when your review focuses on intervention effectiveness and quantitative outcomes. Use PICo (Population, phenomenon of Interest, Context) for qualitative or descriptive questions, as it does not require a comparison group.
3. How many databases should I search for a systematic review?
Most systematic reviews search at least three databases (e.g., PubMed, Scopus, and a subject-specific database like PsycINFO). For comprehensive coverage, additional databases plus grey literature sources are recommended.
4. What is a Boolean operator and how do I use it?
Boolean operators (AND, OR, NOT) connect search terms. AND narrows results by requiring that all terms appear, OR broadens by retrieving records with any of the terms, and NOT excludes terms that may cause irrelevant hits. Use parentheses to group synonyms together.
5. Should I include non-English studies in my systematic review?
Ideally, yes, to reduce language bias. If you lack translation resources, clearly state the language restriction in your methods and discuss the potential impact on your findings in the limitations section.
6. How do I find grey literature for my review?
Search preprint servers (medRxiv, arXiv), conference proceedings, clinical trial registries, and institutional repositories. Use sources like OpenGrey or contact experts in the field.
7. What is a PRISMA flow diagram and why do I need one?
The PRISMA flow diagram shows the number of records identified, screened, excluded, and finally included in your review. It is a standard requirement for systematic reviews to ensure transparency in the selection process.
8. How do I manage duplicate records from multiple database searches?
Use reference management software (e.g., EndNote, Zotero) to automatically detect duplicates, then manually review and remove unique copies. Some review tools (e.g., Rayyan, Covidence) have built-in deduplication features.
9. What should I do if I cannot find a second reviewer for screening?
If a second reviewer is unavailable, document this as a limitation. Consider using automated screening tools (e.g., ASReview, RobotAnalyst) to prioritize records, but still aim to have one human reviewer check a random sample.
10. How do I update my search after the initial screening?
Re-run your exact original search string on the same databases with an updated date filter (e.g., from last search date to present). Screen the new records using the same criteria and software, then add them to your review if they are eligible.
