I+D+i

10 Revisión sistemática

Según el artículo Las revisiones sistemáticas (I) (y II), una revisión sistemática es:

Las revisiones sistemáticas son el mejor esfuerzo por recopilar y sintetizar evidencia científica sobre un tema, a través de un método que asegure que los sesgos y limitaciones a los que hemos hecho mención en el artículo precedente de esta serie, sean los mínimos posibles. Esto se logra utilizando los principios del método científico.

Las etapas de una revisión sistemática son:

  1. formulación de la pregunta de la revisión,
  2. definición de las fuentes y del método de búsqueda,
  3. definición y aplicación de criterios de selección de los estudios,
  4. análisis crítico de los estudios,
  5. extracción y síntesis de los resultados de los estudios,
  6. y conclusiones e inferencias.

En Systematic Review and Meta-analysis: When One Study Is Just not Enough se define una revisión sistemática como:

A systematic review uses a process to identify comprehensively all studies for a specific focused question (drawn from research and other sources), appraise the methods of the studies, summarize the results, present key findings, identify reasons for different results across studies, and cite limitations of current knowledge (8,9). In a systematic review, all decisions used to compile information are meant to be explicit, allowing the reader to gauge for him- or herself the quality of the review process and the potential for bias. In this way, systematic reviews tend to be more transparent than their narrative cousins, although they too can be biased if the selection or emphasis of certain primary studies is influenced by the preconceived notions of the authors or funding sources (10).

En What is a systematic review? se define una revisión sistemática como:

A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions. Judgments may be made about the evidence and inform recommendations for healthcare.

These reviews are complicated and depend largely on what clinical trials are available, how they were carried out (the quality of the trials) and the health outcomes that were measured. Review authors pool numerical data about effects of the treatment through a process called meta-analyses. Then authors assess the evidence for any benefits or harms from those treatments. In this way, systematic reviews are able to summarise the existing clinical research on a topic.

Una revisión sistemática se puede realizar a partir de un mapeo sistemático. Las diferencias que existen entre ambos tipos de estudios han sido analizadas en The value of mapping studies: a participant-observer case study (2010); en este artículo se presenta la siguiente tabla de diferencias entre un mapeo sistemático y una revisión sistemática:

SLR process Mapping Study SLR
Research Question General – related to research trends. Which researchers, how much activity, what type of studies etc. Specific - related to outcomes of empirical studies. Of the form: Is technology/method A better or not than B?
Search process Defined by topic area Defined by research question
Search strategy requirements Less stringent if only research trends are of interest Extremely stringent – all relevant studies must be found
Quality evaluation Not essential Important to ensure that results are based on best quality evidence
Results Set of papers related to a topic area and counts of the number of papers in various categories Answer to specific research question, possible with qualifiers (e.g. results apply to novices only).

En el artículo Systematic mapping studies in software engineering (2008), se explican algunas de las ventajas de un systematic review frente a un literature review:

Briefly, a systematic review (SR) goes through existing primary reports, reviews them in-depth and describes their methodology and results. Compared to literature reviews common in any research project, a SR has several benefits: a well-defined methodology reduces bias, a wider range of situations and contexts can allow more general conclusions, and use of statistical meta-analysis can detect more than individual studies in isolation (Kitchenham & Charters 2007).

La mayor desventaja es que un systematic review requiere un mayor esfuerzo:

However, SRs also have several drawbacks, the main one being that they require considerable effort.

En el artículo Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement, también disponible en Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement, se presenta PRISMA (preferred reporting items for systematic reviews and meta-analyses), un método para realizar revisiones sistemáticas. En este artículo, una revisión sistemática se define como:

A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies.

Por otro lado, un metaanálisis se define como:

Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies.

El método PRISMA se compone de 27 elementos y un proceso de aplicación compuesto de 4 fases.

Después del artículo original de PRISMA, se han publicado otros más que desarrollan o extienden el método:

En Systematic Review and Meta-analysis: When One Study Is Just not Enough se define metaanálisis como:

Depending on the nature of the data, the results of a systematic review can be summarized in text or graphic form. In graphic form, it is common for different trials to be depicted in a plot where the point estimate and 95% confidence interval for each study are presented on an individual line (11). When results are mathematically combined (a process sometimes referred to as pooling), this is referred to as meta-analysis. Graphically, the pooled result is often presented as a diamond at the bottom of the plot.

When performing a meta-analysis, a review team usually combines aggregate-level data reported in each primary study (point and variance estimate of the summary measure).

Una revisión sistemática y un metaanálisis se pueden combinar en un mismo estudio, por ejemplo Web-Based Digital Health Interventions for Weight Loss and Lifestyle Habit Changes in Overweight and Obese Adults: Systematic Review and Meta-Analysis, cuyo método fue:

We searched PubMed or Medline, SciELO, Lilacs, PsychNet, and Web of Science up to July 2018, as well as references of previous reviews for randomized trials that compared Web-based digital health interventions to offline interventions. Anthropometric changes such as weight, body mass index (BMI), waist, and body fat and lifestyle habit changes in adults with overweight and obesity were the outcomes of interest. Random effects meta-analysis and meta-regression were performed for mean differences (MDs) in weight. We rated the risk of bias for each study and the quality of evidence across studies using the Grades of Recommendation, Assessment, Development, and Evaluation approach.

Revisión sistemática del estado de la cuestión (systematic literature review)

Revisiones bibliográficas sistematizadas: guía de recursos

How to... write a literature review

How to conduct a systematic or evidence-based literature review

Una revisión sistemática del estado de la cuestión es una revisión de la bibliografía referente a un tema realizada mediante un método que es explícito y transparente, que cumple un protocolo o un conjunto de fases.

Procedures for Performing Systematic Reviews

Cita: Barbara Kitchenham. Keele University and National ICT Australia Ltd, 2004.

Systematic review in software engineering

Cita: J. Biolchini,P.G. Mian, A.C.C. Natali, G.H. Travassos. System Engineering and Computer Science Department COPPE/UFRJ, Technical Report ES, 679(05), 45, May 2005.

Guidelines for performing Systematic Literature Reviews in Software Engineering

Cita: Barbara Kitchenham, Stuart Charters. Keele University and Durham University Joint Report, 2007.

Experiences Using Systematic Review Guidelines

Cita: Mark Staplesa, Mahmood Niazi. Journal of Systems and Software. Volume 80, Issue 9, September 2007, Pages 1425-1437.

Systematic literature reviews in software engineering: Preliminary results from interviews with researchers

Cita: M.A. Babar, He Zhang. 3rd International Symposium on Empirical Software Engineering and Measurement, Pages 346-355, 15-16 Oct. 2009.

Systematic review in software engineering: where we are and where we should be going

Cita: Barbara A. Kitchenham. In Proceedings of the 2nd international workshop on Evidential assessment of software technologies (EAST '12). ACM, New York, NY, USA, 1-2.

A Guide to Conducting a Systematic Literature Review of Information Systems Research

Cita: Chitu Okoli, Kira Schabram. Sprouts: Working Papers on Information Systems, 10(26), 2010.

Systematic Reviews in the Social Sciences: A Practical Guide

Cita: Mark Petticrew, Helen Roberts. Wiley-Blackwell, 2005.

Ejemplos de revisión sistemática (en español)

Calidad en Cursos Abiertos Masivos y en Línea. Revisión de literatura del 2012 - 2016

Cita: Alejandra Meléndez, Mariela Román, Mar Pérez-Sanagustín, Jorge J. Maldonado. Actas de la Jornada de MOOCs en Español en EMOOCs 2017 (EMOOCs-ES 2017), Pages 72-80.

Los MOOC y la evaluación del alumnado: revisión sistemática (2012-2016)

Cita: María del Mar Sánchez Vera, Víctor González Calatayud, María Paz Prendes Espinosa. revista d'innovació educativa, 18, 2017.

Ejemplos de revisión sistemática (en inglés)

Reconfigurable computing: a survey of systems and software

Cita: Katherine Compton and Scott Hauck. ACM Computing Surverys. Volume 34, Issue 2, June 2002, Pages 171-210.

Empirical studies of agile software development: A systematic review

Cita: Tore Dybå, Torgeir Dingsøyr. Information and Software Technology. Volume 50, Issues 9-10, August 2008, Pages 833-859.

A Systematic Review of Usability Evaluation in Web Development

Cita: Emilio Insfran, Adrian Fernandez. Web Information Systems Engineering – WISE 2008 Workshops, LNCS 5176, 2008, Pages 81-91.

Software engineering researchers' attitudes on case studies and experiments: An exploratory survey

Cita: D. Tofan, M. Galster, P. Avgeriou, D. Weyns. 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE 2011), Pages 91-95, 11-12 April 2011.

A systematic review on the effectiveness of web usability evaluation methods

Cita: Adrian Fernandes, Silvia Abrahão, Emilio Insfran. 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012), 2012.

A systematic review on security in Process-Aware Information Systems - Constitution, challenges, and future directions

Cita: Maria Leitner, Stefanie Rinderle-Ma. Information and Software Technology. Volume 56, Issue 3, March 2014, Pages 273-293.

A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015

Cita: George Veletsianos and Peter Shepherdson. International Review of Research in Open and Distributed Learning, Volume 17, Issue 2, February 2016.

A systematic review of research on the flipped learning method in engineering education

Cita: Aliye Karabulut-Ilgu, Nadia Jaramillo Cherrez, Charles T. Jahren. British Journal of Educational Technology. DOI: 10.1111/bjet.12548

Research challenges in accessible MOOCs: a systematic literature review 2008–2016

Cita: Sanchez-Gordon, S. & Luján-Mora, S. Universal Access in the Information Society (2017). DOI: 10.1007/s10209-017-0531-2

Ver en línea: Research challenges in accessible MOOCs: a systematic literature review 2008–2016

Systematic reviews en la revista PLOS One.

Ejemplos de revisión (en inglés)

Educational Data Mining: A Review of the State of the Art

Cita: Cristóbal Romero, Sebastián Ventura. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). Volume 40, Issue 6, November 2010, Pages 601-618.

Este artículo tiene 304 citas que ocupan casi 8 páginas de las 18 páginas que tiene.

Ejemplos de metaanalisis (en inglés)

Estos artículos son una revisión sistemática de revisiones sistemáticas. Se podría decir que son una "metarevisión sistemática". También se puede decir que son "tertiary research", frente a primary research y secondary research.

Systematic literature reviews in software engineering - A systematic literature review

Cita: Barbara Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, Stephen Linkman. Information and Software Technology. Volume 51, Issue 1, January 2009, Pages 7-15. DOI: 10.1016/j.infsof.2008.09.009

Systematic literature reviews in global software development: A tertiary study

Cita: J. M. Verner, O. P. Brereton, B. A. Kitchenham, M. Turner, M. Niazi. 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012), Ciudad Real, 2012, Pages 2-11. DOI: 10.1049/ic.2012.0001

Relation between Alice software and programming learning: a systematic review of the literature and meta-analysis

Cita: Joana M. Costa, Guilhermina L. Miranda. (2017). British Journal of Educational Technology, 48(6), 1464-1474. DOI: 10.1111/bjet.12496

Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? – A systematic review of reviews: an update

Cita: Jin, Y., Sanger, N., Shams, I., Luo, C., Shahid, H., Li, G., Bhatt, M., Zielinski, L., Bantoto, B., Wang, M., Abbade, L. P., Nwosu, I., Leenus, A., Mbuagbaw, L., Maaz, M., Chang, Y., Sun, G., Levine, M. A., Adachi, J. D., Thabane, L., … Samaan, Z. (2018). Journal of multidisciplinary healthcare, 11, 495-510. DOI: 10.2147/JMDH.S155103

Herramientas

Revistas