The Postgraduate Student’s Guide to Systematic Reviews: Part 2

The Postgraduate Student’s Guide to Systematic Reviews : Part 2

Welcome to the deep end. If you are a postgraduate student embarking on a systematic review entirely on your own, I want to start by acknowledging the weight of the task. You don’t have a team of research assistants or a dedicated librarian at your beck and call. You are the investigator, the screener, the data extractor, and the writer. It is a lot to handle, and the temptation to rush through the “procedural” chapters—the Introduction and Methods—to get to the “exciting” Results is very real.

However, as someone who has supervised many reviews, let me let you in on a secret: these sections are your absolute best friends. In a solo review, the Methods section acts as your “shield against bias.” While a standard literature review is essentially a curated reading list, a systematic review is a robust, transparent, and replicable scientific process (Randles and Finnegan, 2023). Because you are working alone, your transparency is your credibility. By the time we finish this, you won’t just be “reading papers”; you’ll be building a rigorous piece of evidence-based science that can withstand the toughest academic scrutiny.

1. The Introduction: Setting the Stage and the Stakes

Your Introduction isn’t just a background summary; it is the “sales pitch” for why your review needs to exist. It must demonstrate that you have a “sufficient understanding” of the subject area and that you’ve done your homework on previous research (Randles and Finnegan, 2023).

As a postgrad writing a systematic review, if your results and discussion are the public-facing “architecture” of your research, then the Introduction and Methods sections are the “Engine Room.” For a solo postgraduate researcher, these sections are where the heavy lifting of scientific rigor occurs. It is here that you convince your examiners that your findings aren’t just a collection of opinions, but a reproducible, unbiased account of the available evidence.

In a traditional narrative literature review, you might summarize what is known about a topic, often with a high degree of subjectivity in selecting which studies to highlight. By contrast, a Systematic Review (SR) is a formal synthesis of evidence that is identified, critically appraised, and analyzed through a methodical and transparent process (Randles and Finnegan, 2023). It adheres to a strict scientific design based on explicit, pre-specified methods (CRD, 2009). For the solo student, the Methods section serves as a protective blueprint; it prevents you from getting lost in the “tsunami of data” and ensures your conclusions are defensible.

Highlighting the Problem

In many fields, individual studies reach conflicting conclusions or are methodologically flawed (CRD, 2009). This creates a problem for practitioners who need reliable evidence to make decisions. Your job in the Introduction is to highlight these gaps. Perhaps existing studies are too small to show a clear effect, or perhaps the findings are disparate because the study designs were inconsistent. By synthesizing these problems, you make the case that a systematic review is necessary to provide a more precise and comprehensive estimate of an intervention’s effectiveness than any single study could manage alone (CRD, 2009; Drucker, Fleming and Chan, 2016).

Justifying the Solution

The Introduction is where you establish the “why” of your study. Its primary purpose is to provide a clear rationale for the review and to identify existing gaps in the research (CRD, 2009). You are essentially building a case for your project’s existence, explaining why the world needs another review on this specific topic.

According to Drucker, Fleming, and Chan (2016), your Introduction must answer a fundamental question: What is the goal? Are you updating an older, outdated review? Are you resolving conflicting conclusions between two major schools of thought? Or are you identifying gaps where future primary research is needed?

The Mentor’s Niche Tip: Before you commit, you must check if a high-quality, recent review already exists. Search the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) first (CRD, 2009). If a robust review was published in the last 18 months, you might need to pivot. Don’t panic—you can often narrow your focus to a specific population (e.g., “older adults” instead of “the general public”) to find your own unique niche.

2. The Power of the Initial Scoping Search

One of the most common mistakes I see solo researchers make is skipping the “scoping” phase. Scoping is the pre-work that defines your inclusion years and keeps you from doing redundant work.

As Randles and Finnegan (2023) point out, scoping helps you understand the volume of existing literature. For example, in their work on “Veterans,” they initially found a massive body of literature on general mental health. Through scoping, they identified that a major systematic review on veteran mental health had been published in 2015. Rather than repeating that work, they used it as a justification to narrow their focus specifically to “Help-Seeking Behavior.” This narrowed scope made the review manageable for a solo researcher while ensuring it added fresh value to the field.

3. The Methods: Crafting Your PICOS Compass

The Methods section is the heart of your review. To keep your search focused and your screening workload manageable, you need a framework for setting your inclusion and exclusion criteria.

Framework Comparison

While PICO is the gold standard for clinical questions, it doesn’t fit every topic. As a mentor, I encourage you to feel empowered to adapt these frameworks (Munn et al., 2018). If your topic doesn’t fit, amend the framework to suit your needs (Randles and Finnegan, 2023).

Framework	Best For…	Components Explained
PICO	Quantitative/Clinical effectiveness	Population, Intervention, Comparison, Outcome
SPICE	Qualitative/Service evaluation	Setting, Perspective, Intervention, Comparison, Evaluation
SPIDER	Qualitative/Mixed methods	Sample, Phenomenon of Interest (the behavior/experience), Design, Evaluation, Research Type

The Retinoblastoma Case Study: Defining PICOS with Precision

To make your review replicable, every element of PICOS (Population, Intervention, Comparator, Outcome, and Study Design) must be crystal clear. Take the example of a review on childhood retinoblastoma (CRD, 2009):

Population: Participants aged 18 or under diagnosed with retinoblastoma. Why 18? Because that is the clinically justifiable cut-off for “childhood.”
Intervention: Any treatment (chemotherapy, radiotherapy, etc.).
Comparator: A different treatment or a control group.
Outcome: Survival rates, tumor response, or adverse effects.
Study Design: You might decide to include Randomized Controlled Trials (RCTs) and cohort studies with comparison groups. Crucially, in this example, you would exclude case reports and case series. Why? Because they have a high potential for bias and lack a comparison group, making them less reliable for determining effectiveness (CRD, 2009).

4. Why You Must “Publicly Declare Your Intent” (The Protocol)

Solo researchers often feel they can skip the formal protocol because they are “only working for themselves.” This is a dangerous trap. A protocol is a formal document that defines your methodology before you start.

Preventing Reporting Bias

A protocol stops “data-driven hypotheses.” If you don’t have a pre-written plan, you might be tempted to change your inclusion criteria or your primary outcomes once you see what the results look like—this is known as “reporting bias” (Drucker, Fleming and Chan, 2016). Using the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol) statement gives you a checklist to ensure you haven’t missed anything vital, from your search strategy to your planned analyses.

Registration: The PROSPERO Step

You should prospectively register your review on PROSPERO. This is an international database that tells the world what you are doing. It prevents other people from accidentally starting the same review and holds you accountable. If you have to deviate from your protocol later—perhaps your search was too narrow and yielded zero results—that’s okay! But you must explain and justify that deviation in your final Methods section (Drucker, Fleming and Chan, 2016).

5. The Search Strategy: Solving the “Goldilocks” Problem

Your search strategy needs to be highly sensitive. If it’s too narrow, you miss data; if it’s too broad, you’ll face “screening fatigue,” which is the nemesis of the solo researcher. You’re looking for the “Goldilocks” search—just right.

Boolean Operators and Syntax

To build a professional strategy, you must use Boolean operators and truncation (Randles and Finnegan, 2023):

AND: Combines different concepts (e.g., “Veterans” AND “Mental Health”).
OR: Broadens the search to synonyms (e.g., “Ex-forces” OR “Ex-military”).
Truncation (*): Finds all variations of a word (e.g., “educat*” finds educate, education, and educated).
Wildcards (?): Useful for spelling variations (e.g., “behavi?r” finds both UK and US spellings).

Example Boolean String: A search might look like this: (Veteran OR “ex-forces”) AND (“help seeking” OR “treatment seeking”). Using the “Title OR Abstract” field helps ensure you are finding papers where these terms are central to the work (Randles and Finnegan, 2022).

The Hunt for Grey Literature

To avoid publication bias, you cannot rely solely on PubMed or EMBASE. Studies with statistically significant results are more likely to be published than those without—this means “negative” findings are often hidden in the “Grey Literature” (Drucker, Fleming and Chan, 2016). You must check:

Trial Registries: Use clinicaltrials.gov or the World Health Organization’s clinical trials search portal (http://apps.who.int/trialsearch).
Regulatory Agency Websites: To find data submitted for drug or device approval.
Conference Abstracts: These are vital for identifying research that is currently in progress (Adler et al., 2016).

6. Screening and Extraction: Adapting for the Solo Researcher

Here is the hard truth: standard guidelines recommend two researchers screen papers independently because it minimizes errors. Research cited by CRD (2009) shows that a single researcher is likely to miss an average of 8% of eligible studies. Since you’re flying solo, you have to be extra-methodical to close that gap.

Software is Your Co-Pilot

Don’t try to manage this in a Word document. Use tools like Rayyan or EndNote for screening. For more complex reviews, EPPI-Reviewer is a fantastic web-based management program that handles everything from reference management to data synthesis (Randles and Finnegan, 2023). Other useful tools include NVivo for qualitative data or TrialStat SRS.

The Solo Strategy Checklist

Duplicate Removal: Use your software to automatically strip out identical records from different databases.
The “Generous” Sift: When scanning titles and abstracts, be generous. If you aren’t 100% sure a paper is irrelevant, keep it for the full-text stage.
Full-Paper Assessment: Read the remaining papers against your PICOS criteria.
Document “Near-Misses”: You must log why you excluded certain papers at the full-text stage (e.g., “wrong population” or “no comparison group”). You will need this for your PRISMA flow chart (Randles and Finnegan, 2023).
Pilot the Process: Before you do the whole set, test your screening on 5-10 papers to see if your criteria are actually workable.

7. Quality Assessment: Beyond the “Scale Trap”

Not all evidence is created equal. You must assess the Risk of Bias in the studies you include. This is where you differentiate between Internal Validity (is the study design sound?) and External Validity (are the results applicable to the real world?).

Avoid Numerical Quality Scales

Many students try to give studies a score (e.g., “This paper is 8/10 quality”). Try not to do this. Expert guidance warns that summary scores are misleading because it’s hard to know how different criteria should be weighted (CRD, 2009; Drucker, Fleming and Chan, 2016). Instead, use a domain-based checklist like the Cochrane Risk of Bias tool or the CASP (Critical Appraisal Skills Programme) checklist. These tools help you look for specific types of bias:

Selection Bias: Were the groups different at the start?
Performance Bias: Were participants treated differently (other than the intervention)?
Detection Bias: Did the person measuring the outcome know which group the participant was in?
Attrition Bias: Did too many people drop out of one group?
Reporting Bias: Did the authors only report the “good” results?

8. Guarding Against “Spin” and Competing Interests

As a solo reviewer, you are a detective. You must scrutinize the funding of the primary studies you include. Research has shown that industry-sponsored reviews are often of poorer quality than independent ones and are far less likely to express reservations about the intervention being studied (Jørgensen et al., 2006).

You also need to watch out for “Spin”—misleading reporting that overstates the benefits of a treatment. According to Yavchitz et al. (2016), spin often appears in the summaries and abstracts of industry-sponsored work. Look out for:

Misleading Reporting: Failing to mention the limitations or the full methods used.
Misleading Interpretation: Treating a “non-significant” result as if it were a success.
Inappropriate Extrapolation: Suggesting a drug works for everyone when it was only tested on a very specific group (Drucker, Fleming and Chan, 2016).

9. Final Pro-Tips for a Solid Methods Section

To make your journey smoother, keep these mentor-tested tips in mind:

Standardize Everything: Use a formal data extraction form to ensure you capture the same data (dose, population, outcomes) for every single paper (CRD, 2009).
Pilot Your Form: Test your extraction form on 2-3 papers. You will almost certainly find a field you forgot to include.
The Search Log: Keep a detailed log of every database searched, every filter used, and the exact date. Transparency is the hallmark of a master researcher.

10. Conclusion: The Foundation is Set

By the time you finish a rigorous Introduction and Methods section, you have done the heaviest lifting of your systematic review. These chapters provide the blueprint for your entire project. A clear, transparent Methods section makes the Results section almost write itself, and it ensures that your conclusions are robust enough to withstand the most intense academic scrutiny. You are no longer just “summarizing papers”—you are contributing high-quality, evidence-based science to your field.

Good luck—you’ve got this.

Reference List

Adler, B. L., Krausz, A. E., Minuti, A., Silverberg, J. I. and Lev-Tov, H. (2016) ‘Epidemiology and treatment of angiolymphoid hyperplasia with eosinophilia (ALHE): a systematic review’, Journal of the American Academy of Dermatology, 74(3), pp. 506–512.

Aromataris, E. and Munn, Z. (2020) JBI Manual for Evidence Synthesis. JBI.

Aromataris, E. and Pearson, A. (2014) ‘The systematic review: an overview’, American Journal of Nursing, 114(3), pp. 53–58.

CRD (2009) Systematic Reviews: CRD’s guidance for undertaking reviews in health care. University of York.

Drucker, A. M., Fleming, P. and Chan, A. W. (2016) ‘Research Techniques Made Simple: Assessing Risk of Bias in Systematic Reviews’, Journal of Investigative Dermatology, 136(11), pp. e109–e114.

Jørgensen, A. W., Hilden, J. and Gotzsche, P. C. (2006) ‘Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review’, BMJ, 333(7572), p. 782.

Munn, Z., Stern, C., Aromataris, E., Lockwood, C. and Jordan, Z. (2018) ‘What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences’, BMC Medical Research Methodology, 18(5).

Page, M. J. et al. (2021) ‘The PRISMA 2020 statement: an updated guideline for reporting systematic reviews’, BMJ, 372, p. n71.

Randles, R. and Finnegan, A. (2023) ‘Guidelines for writing a systematic review’, Nurse Education Today, 125, p. 105803.

Yavchitz, A. et al. (2016) ‘A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity’, Journal of Clinical Epidemiology, 75, pp. 56–65.

If you missed part 1, you can read it here

Follow on Instagram Instagram

See my videos on academic writing on my You Tube Neil Academic Tuition Channel

Spotify: Academic Writing in 3 Minutes (yes, this is me too)

@neilproofs https://neilacademictuition.com/