How to do a literature review: Managing the Review

Document the Process

• Document your plans, thoughts and choices as you start, and as you proceed. As you go ask what?, and why?, and how do I know? 
• Use a "paper" notebook, or use "folders" with "Word" documents on your computer. Or use software programs like Evernote, or Dropbox. Those programs support making notes and 'saving to the cloud' for access. As noted below, each of these research log tools might have sections for different kinds of comments.
• See Using Research Logs ideas below. These are kept so that you can know (and you can tell others) what you did as a basis for your review.
• Also see other boxes on documenting searches.

Save things

• Save references for items you think might have relevant content.
• Save articles pdfs, paper copies.
• Save links to information, etc. 
• Save screens or web sites with information. 
• Set up access to references and to articles you have downloaded. See Research Management Software at left.

Simple Logs or "Notebooks"

You might simply keep a notebook with any kind of comments or questions you have about anything related to your project. You may have sections on your initial understanding and goals for the project, on searches tried, on methods for analysis, on possible relevant considerations, on problems, etc.

More Involved Logs, "Notebooks", or a "File/Folder System". 

You may have notebook sections like the following for comments, etc. related to your review project.  These may also be separate "files" or "folders" in a "filing system" (again, paper or electronic).

• Background and Planning  Initial information, thoughts, and plans/timeline for completing. Also ideas concerning changes to plans.  
• Research topic/questions  Initial, updated and final drafts of your topic.
• Inclusion/Exclusion Criteria (IE Criteria) What are you looking for that will lead you to choose the sources that might have information you need. The criteria also indicate what information sources you will not choose to use (exclude). Also you might list what will guide your selection of actual content/information eventually used/incorporated into your review. 
• Scoping or Ad hoc searches & results  Initial "reconnaissance" for initial mapping of an area or for periodic update/checking, etc. It is possible to adjust your IE criteria.
• Full Search  Design/Search results/ Use of results/Search Modification, etc.
• Extraction  What information you need and in fact are choosing to pull out of sources to use.  
• Analysis or Synthesis  Ideas, sketches, thinking in response to the information you have identified in the information sources.
• Browsing  Information activities that are "not systematic", online, or in the "library stacks", etc.  
• "Extra" Ideas that don't fit elsewhere in notebook, thoughts that are kind of serendipitous. etc.
• ILL "Interlibrary Loan" or actions taken to get items from outside of KSU or Ohio, and results.   
• Email/communication 
• Drafts of review

In addition to a research log (or filing system) for documenting your "overall literature review" goals, activities, ideas, and comments, etc., there are more detailed approaches for documenting your literature search activities (e.g., document computer searches, article selection, and the selection of information for use from the articles). Thoughts and resources for this kind of documenting are below.

A step-by-step framework to consider for documenting your literature search process: 

• Provide a purpose statement.
• Document the databases or search engines used.
• Specify the limits applied to the search.
• List the inclusion and exclusion criteria for the search.
• List the search terms used.
• Document the search process for each search resource used.
• Assess retrieved articles for relevance and document assessment choices and reasoning.
• Document a summary table of included articles.
• Provide a statement specifying the number of retrieved articles.

These steps taken from: Kable, A. H., Pich, J., and Maslin-Prothero, S. (2012) A structured approach to documenting a search strategy for publication: A 12-step guideline for authors. Nurse Education Today, 32, 878-886.    Read the article online here. 

More on documenting:

• How & What to Document
• Reconstructing the giant: on the importance of rigour in documenting the literature search process.
• Methods for documenting systematic review searches: a discussion of common issues (an article).
  Guidance based on a survey of over 260 systematic review authors.

The "Matrix Method" is an approach to organizing, monitoring, and documenting your search activities. 

The Matrix Method of Literature Reviews 

Some questions to ask when appraising the literature!

1.   From Doing a Literature Review in Health and Social Care: A Practical Guide by Helen Aveyard

• What is the journal of publication?
• What is the research question and why was the study conducted?
• What method was selected to undertake the research?
• Has the appropriate sample been obtained?
• How were the data collected?
• How were the data analyzed?

The 2007 edition of this book is in the KSU Libraries collection. You can read the 2nd ed. (2010) of this book online. 

2.   Critical Appraisal Tools drawn from or related to the health sciences

It can be valuable to use "tools for appraising" the literature that is then used in a literature review.

"Critical appraisal is an integral process in Evidence Based Practice. Critical appraisal aims to identify methodological flaws in the literature and provide consumers of research evidence the opportunity to make informed decisions about the quality of research evidence..."  This site offers "a list of critical appraisal tools, linked to the websites where they were developed." - from the  International Centre for Allied Health Evidence (iCAHE).