Even back when I was an undergraduate student (almost 20 years ago) doing my honours project, I trawled through dozens of papers to find that perfect one. I remember printing all of them out, sorting them, highlighting them, and in the end I only really worked on the content of two papers.

I still struggle through literature searches today. I’m afraid that I will miss that perfect paper that will address my research. It works when you’re in a niche area, and the body of literature is relatively small. Or when you have a professor to suggest reading for you.

Google has made my life somewhat easier over my early career researcher years. I set up notifications on my Google Scholar searches, and almost every day I get a few links to follow up. I can read the title and the extract, and determine whether I want to click on the link.

Recently, since moving up the career ladder into supervisory roles, that old feeling has come back. I feel the need to be ‘on top’ of the literature when moving into a tangential field, and Google Scholar notifications didn’t work for me so much anymore. Because I was no longer keeping track of my reading, I don’t have a high-level view of the literature.

It wasn’t until this year that I discovered the ‘PRISMA’ [1]. PRISMA, short for ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses,’ infers a methodology for systematic literature reviews. It was perfect for my FOMO, because it captured my process for limiting the search results, which was lost on my previous literature searches.

I have recently published an article using PRISMA [2], performing a meta-analysis of papers written about using atom probe tomography to investigate the structure of aluminium alloys in the year 2018. In this paper, I specified the assumptions I made while limiting my search results, and the questions I was able to answer using a systematic meta-analysis methodology. Even though I’m sure I missed some papers, my process is transparent and anyone can provide criticism.

The remainder of this article documents my process.

Exploratory literature search: developing your questions

You have a topic. You don’t know your specific questions yet, but you want to know what the state-of-the-art is. Searching for the topic and the words “review article” did not come up with anything useful, perhaps because the topics of the review articles were not exactly relevant.

A rigorous way is the following. Search for the last five years. Note that some search engines do not provide the keywords for a proportion of the articles. Therefore, looking at the titles gives a pretty good indication of what the authors think is the most important topic of the paper. Break up the title into the consitutent terms, group terms together and make a mind-map of the terms. This will give you an idea of the most popular topics, and where the potential gaps are. The gaps may inform your questions.

A quicker way of doing this is to start with a broader question, and by looking at random titles you may be able to answer parts of the question as well as develop more specific questions. Remember to document the development of your questions, as sometimes I go around in a big circle and re-ask a question that I had asked before.

Researching for your literature review: synthesis of the data

The PRISMA literature review is suitable for when you already have questions, and you want to limit the literature you want to read. For my paper, I selected a topic regarding aluminium alloys, atom probe tomography, in the year 2018, which resulted in 34 papers.

When I was reading the full texts for this meta-analysis, I was averaging only three articles a day for a month, and even that apparently small amount was exhausting. This includes active reading, coding for structured keywords by referring to the mindmap for the structure. I used zotero for keyword coding, although endnote and other reference management software will also be suitable. I also tried the excellent software NVivo, and even though it was ideal for coding sections of the full text,  there was no way to export the coding into a RIS or BIB file at the time.

Once all of my articles were coded with structured keywords, I was able to perform a meta-analysis and answer my specific questions. Synthesis of the raw data involved generating reports of the number of each structured keyword. I grouped the structured keywords together and at times cross-references them, and this generated a picture of the state-of-the-art in 2018.

Some tools for keeping up-to-date with literature

Use notifications provided by search engines such as Google Scholar, Scopus or Web of Science. By this stage you may know the specific search terms you want to use. The email notifications provide the title and an excerpt of where the search terms were found, and usually this combination will allow you to decide whether the full text is interesting or not.

But don’t make the same mistake I did! Make sure you are as rigorous with documenting what you’ve read, as in a literature review.

[1] https://doi.org/10.1371/journal.pmed.1000100

[2] https://doi.org/10.3390/met9101071