Según el artículo Las revisiones sistemáticas (I) (2011) (y II (2011)), una revisión sistemática es:
Las revisiones sistemáticas son el mejor esfuerzo por recopilar y sintetizar evidencia científica sobre un tema, a través de un método que asegure que los sesgos y limitaciones a los que hemos hecho mención en el artículo precedente de esta serie, sean los mínimos posibles. Esto se logra utilizando los principios del método científico.
Las etapas de una revisión sistemática son:
- formulación de la pregunta de la revisión,
- definición de las fuentes y del método de búsqueda,
- definición y aplicación de criterios de selección de los estudios,
- análisis crítico de los estudios,
- extracción y síntesis de los resultados de los estudios,
- y conclusiones e inferencias.
En Systematic Review and Meta-analysis: When One Study Is Just not Enough (2008) se define una revisión sistemática como:
A systematic review uses a process to identify comprehensively all studies for a specific focused question (drawn from research and other sources), appraise the methods of the studies, summarize the results, present key findings, identify reasons for different results across studies, and cite limitations of current knowledge (8,9). In a systematic review, all decisions used to compile information are meant to be explicit, allowing the reader to gauge for him- or herself the quality of the review process and the potential for bias. In this way, systematic reviews tend to be more transparent than their narrative cousins, although they too can be biased if the selection or emphasis of certain primary studies is influenced by the preconceived notions of the authors or funding sources (10).
En What is a systematic review? se define una revisión sistemática como:
A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions. Judgments may be made about the evidence and inform recommendations for healthcare.
These reviews are complicated and depend largely on what clinical trials are available, how they were carried out (the quality of the trials) and the health outcomes that were measured. Review authors pool numerical data about effects of the treatment through a process called meta-analyses. Then authors assess the evidence for any benefits or harms from those treatments. In this way, systematic reviews are able to summarise the existing clinical research on a topic.
Una revisión sistemática se puede realizar a partir de un mapeo sistemático. Las diferencias que existen entre ambos tipos de estudios han sido analizadas en The value of mapping studies: a participant-observer case study (2010); en este artículo se presenta la siguiente tabla de diferencias entre un mapeo sistemático y una revisión sistemática:
| SLR process | Mapping Study | SLR |
|---|---|---|
| Research Question | General – related to research trends. Which researchers, how much activity, what type of studies etc. | Specific - related to outcomes of empirical studies. Of the form: Is technology/method A better or not than B? |
| Search process | Defined by topic area | Defined by research question |
| Search strategy requirements | Less stringent if only research trends are of interest | Extremely stringent – all relevant studies must be found |
| Quality evaluation | Not essential | Important to ensure that results are based on best quality evidence |
| Results | Set of papers related to a topic area and counts of the number of papers in various categories | Answer to specific research question, possible with qualifiers (e.g. results apply to novices only). |
En el artículo Systematic mapping studies in software engineering (2008), se explican algunas de las ventajas de un systematic review frente a un literature review:
Briefly, a systematic review (SR) goes through existing primary reports, reviews them in-depth and describes their methodology and results.
Compared to literature reviews common in any research project, a SR has several benefits:
a well-defined methodology reduces bias, a wider range of situations and contexts can allow more general conclusions,
and use of statistical meta-analysis can detect more than individual studies in isolation (Kitchenham & Charters 2007).
La mayor desventaja es que un systematic review requiere un mayor esfuerzo:
However, SRs also have several drawbacks, the main one being that they require considerable effort.
En el artículo Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement (2009), también disponible en Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement (2010), se presenta PRISMA (preferred reporting items for systematic reviews and meta-analyses), un método para realizar revisiones sistemáticas y metaanálisis. En este artículo, una revisión sistemática se define como:
A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select,
and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review.
Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies.
El método PRISMA se compone de 27 elementos y un proceso de aplicación compuesto de 4 fases.
Después del artículo original de PRISMA, se han publicado otros más que desarrollan o extienden el método:
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration (2009)
- PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts (2013)
También existen los frameworks SALSA y ReSiste-CSH.
En A Guide to Conducting a Standalone Systematic Literature Review (2015) se define una guía compuesta de 8 pasos para realizar una revisión sistemática:
The guide I present here has eight major steps (see Figure 1) that one needs to take to conduct a
systematic literature review. Indeed, these steps are very valuable for any kind of literature review;
however, for a review to be scientifically rigorous, all of the following steps are essential.
- Identify the purpose: the first step in any review requires the reviewers to clearly identify the
review’s purpose and intended goals, which is necessary for the review to be explicit to its
readers.
- Draft protocol and train the team: for any review that employs more than one reviewer,
reviewers need to be completely clear and in agreement about the procedure they will follow,
which requires both a written, detailed protocol document and training for all reviewers to ensure
consistency in how they execute the review.
- Apply practical screen: also called screening for inclusion, this step requires that the reviewers
be explicit about what studies they considered for review and which ones they eliminated without
further examination (a very necessary part of any literature review). For excluded studies, the
reviewers must state their practical reasons for not considering them and justify how the resulting
review can still be comprehensive given the practical exclusion criteria.
- Search for literature: the reviewers need to be explicit in describing the details of the literature
search and need to explain and justify how they assured the search’s comprehensiveness.
- Extract data: after reviewers have identified all the studies that should be included in the review,
they need to systematically extract the applicable information from each study.
- Appraise quality: also called screening for exclusion, the reviewers need to explicitly spell out
the criteria they use to judge which papers they will exclude for insufficient quality. Researchers
need to score all included papers, depending on the research methodologies they employ, for
their quality.
- Synthesize studies: also known as analysis, this step involves combining the facts extracted
from the studies by using appropriate techniques, whether quantitative, qualitative, or both.
- Write the review: in addition to the standard principles to be followed in writing research papers,
the process of a systematic literature review needs to be reported in sufficient detail such that
other researchers can independently reproduce the review’s results.
Revisión sistemática del estado de la cuestión (systematic literature review)
Revisiones bibliográficas sistematizadas: guía de recursos
How to... write a literature review
How to conduct a systematic or evidence-based literature review
Una revisión sistemática del estado de la cuestión es una revisión de la bibliografía referente a un tema realizada mediante un método que es explícito y transparente, que cumple un protocolo o un conjunto de fases.
Procedures for Performing Systematic Reviews
Cita: Barbara Kitchenham. Keele University and National ICT Australia Ltd, 2004.
Systematic review in software engineering
Cita: J. Biolchini,P.G. Mian, A.C.C. Natali, G.H. Travassos. System Engineering and Computer Science Department COPPE/UFRJ, Technical Report ES, 679(05), 45, May 2005.
Systematic Reviews in the Social Sciences: A Practical Guide
Cita: Mark Petticrew, Helen Roberts. Wiley-Blackwell, 2005.
Guidelines for performing Systematic Literature Reviews in Software Engineering
Cita: Barbara Kitchenham, Stuart Charters. Keele University and Durham University Joint Report, 2007.
Experiences Using Systematic Review Guidelines
Cita: Mark Staplesa, Mahmood Niazi. Journal of Systems and Software. Volume 80, Issue 9, September 2007, Pages 1425-1437.
Systematic literature reviews in software engineering: Preliminary results from interviews with researchers
Cita: M.A. Babar, He Zhang. 3rd International Symposium on Empirical Software Engineering and Measurement, Pages 346-355, 15-16 Oct. 2009.
A Guide to Conducting a Systematic Literature Review of Information Systems Research
Cita: Chitu Okoli, Kira Schabram. Sprouts: Working Papers on Information Systems, 10(26), 2010.
Systematic review in software engineering: where we are and where we should be going
Cita: Barbara A. Kitchenham. In Proceedings of the 2nd international workshop on Evidential assessment of software technologies (EAST '12). ACM, New York, NY, USA, 2012.
A Guide to Conducting a Standalone Systematic Literature Review
Cita: Okoli, C. Communications of the Association for Information Systems, 37, 2015.
Systematic Approaches to a Successful Literature Review
Cita: Andrew Booth, Anthea Sutton, Diana Papaioannou. Second Edition, Sage Publications, 2016.
Doing a Systematic Review
Cita: Angela Boland, Gemma Cherry, Rumona Dickson. Sage Publications, 2017.
An Introduction to Systematic Reviews
Cita: David Gough, Sandy Oliver, James Thomas. Second Edition, Sage Publications, 2017.
Ejemplos de revisión sistemática (en español)
Calidad en Cursos Abiertos Masivos y en Línea. Revisión de literatura del 2012 - 2016
Cita: Alejandra Meléndez, Mariela Román, Mar Pérez-Sanagustín, Jorge J. Maldonado. Actas de la Jornada de MOOCs en Español en EMOOCs 2017 (EMOOCs-ES 2017), Pages 72-80.
Los MOOC y la evaluación del alumnado: revisión sistemática (2012-2016)
Cita: María del Mar Sánchez Vera, Víctor González Calatayud, María Paz Prendes Espinosa. revista d'innovació educativa, 18, 2017.
Ejemplos de revisión sistemática (en inglés)
Requirements modeling languages for software product lines: A systematic literature review
Cita: Samuel Sepúlveda, Ania Cravero, Cristina Cachero. Information and Software Technology, Volume 69, January 2016. DOI: 10.1016/j.infsof.2015.08.007
A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015
Cita: George Veletsianos and Peter Shepherdson. International Review of Research in Open and Distributed Learning, Volume 17, Issue 2, February 2016.
A systematic review of research on the flipped learning method in engineering education
Cita: Aliye Karabulut-Ilgu, Nadia Jaramillo Cherrez, Charles T. Jahren. British Journal of Educational Technology, Volume 49, Issue 3, 2018. DOI: 10.1111/bjet.12548
Research challenges in accessible MOOCs: a systematic literature review 2008–2016
Cita: Sanchez-Gordon, S. & Luján-Mora, S. Universal Access in the Information Society, 17, 775-789, 2018. DOI: https://doi.org/10.1007/s10209-017-0531-2
Ver en línea: Research challenges in accessible MOOCs: a systematic literature review 2008–2016
Modeling and Management Big Data in Databases—A Systematic Literature Review
Cita: Diana Martinez-Mosquera, Rosa Navarrete, Sergio Lujan-Mora. Sustainability 2020, 12(2). https://doi.org/10.3390/su12020634
Rediscovering the use of chatbots in education: A systematic literature review
Cita: José Quiroga Pérez, Thanasis Daradoumis, Joan Manuel Marquès Puig. Computer Applications in Engineering Education. https://doi.org/10.1002/cae.22326
Empirical Studies on Web Accessibility of Educational Websites: A Systematic Literature Review
Cita: Milton Campoverde-Molina, Sergio Luján-Mora, Llorenç Valverde. IEEE Access, 8(1), p. 91676-91700, 2020. e-ISSN: 2169-3536. https://doi.org/10.1109/ACCESS.2020.2994288
Web-Based Serious Games and Accessibility: A Systematic Literature Review
Cita: Luis Salvador-Ullauri, Patricia Acosta-Vargas, Sergio Luján-Mora. Applied Sciences, 10(21), p. 1-25, 2020. ISSN: 2076-3417. https://doi.org/10.3390/app10217859
Systematic literature review on software architecture of educational websites
Cita: Milton Campoverde-Molina, Sergio Luján-Mora, Llorenç Valverde. IET Software, p. 1-21, 2021. ISSN: 1751-8814. https://doi.org/10.1049/sfw2.12024
Ejemplos de revisión (en inglés)
Educational Data Mining: A Review of the State of the Art
Cita: Cristóbal Romero, Sebastián Ventura. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). Volume 40, Issue 6, November 2010, Pages 601-618.
Este artículo tiene 304 citas que ocupan casi 8 páginas de las 18 páginas que tiene.
Herramientas
- StArt (State of the Art through Systematic Review)
- SciMAT (Science Mapping Analysis Tool)
- NVivo
- Rayyan QCRI
- Publish or Perish
- Parsifal