Computer Search Report Checklist (CSRC): CSRC Project

2007/08

 Major guidelines were reviewed for item content. The assumption was that major organizations who specialize in "systematic reviews" (SR) have been very careful and thorough in the guidance they provide for using SR methods. That guidance was taken to contain high-quality recommendations, including recommendations regarding "computer searching" and the "reporting" of computer search steps. 

At this stage, the publications of the following organizations were consulted:
Cochrane Collaboration, the Campbell Collaboration, the working group for the Meta-analysis Of Observational Studies in Epidemiology (MOOSE), the Human Genome Epidemiology Network (HuGENet), the CONSORT Group, the NHS Centre for Reviews and Dissemination, and QUOROM/PRISMA.

 Non-redundant recommendations translated into 47 items.   Some of the 47 items were based on professional experience; these supplemented what was addressed in the SR guidelines just noted.    A similar approach to item development was found in Yoshii, et al.  2009.

2008/09

Trained and piloted test items.  A focus on "comprehensiveness" and "reproducibility" emerging. 

Assessed checklist item inter-rater agreement.  Kappas for the 36 items indicated moderate to perfect agreement < .00

2009/11

Added two items to original 47 giving 49 items total.

Evaluated psychology SR with "comprehensiveness items” (C)

Article published in journal Research Synthesis Methods.  See publication tab.

2012/present

Used "reproducibility”(R) items with sample of psychology SR.

Poster presentation for Campbell Collaboration meeting in Chicago, 2013. See Publication & Presentation.

Poster/paper regarding the CSRC presented at the American Psychological Association's Annual Convention, Toronto, August, 2015. See Publication & Presentation.

Results of work focusing on "reproducibility of computer searches" being submitted for publication. 

Ongoing development ideas/plans - emphasize collaboration

A survey is one approach being considered as a means for gathering input on possible directions and value of CSRC project.

Closely monitor and evaluate relation of CSRC to related resources such as AMSTAR, PRESS, and PRISMA. How might each be used by researchers in a supplementary manner to support literature review work?

Create other items, or refine, or reduce number of items.

Create a practical tool....a flexible/extensible resource.

Possible web site to support CSRC scoring & reports.