An empirical assessment of a systematic search process for systematic reviews

Abstract

Background: Systematic Literature Reviews (SLRs) have been gaining significant attention from Software Engineering (SE) researchers. Several researchers are also working on improving SLR methodology for SE. Objective: The study reported in this paper aims to validate the QGS-based search process for SLR, i.e. whether a more effective (sensitive) and/or productive (precise) search can be achieved by following the QGS-based process. Method: We used a dual-case study, in which each case includes two observations of SE literature search for the same SLR but using and not using the QGS-based approach. The overall sensitivity and precision were calculated for each observation that implemented different search design. Results: The use of QGS-based search process resulted in higher sensitivity and precision in this dual-case study. Conclusions: A systematic search process can help capture more relevant studies as well as save researchers' time spent on literature search activities. Our observations also support that an integrated search strategy is recommended for SLRs in SE to avoid the possible limitations of applying a single (manual or automated) search method.