8 Mapeo sistemático

Según el artículo Systematic mapping studies in software engineering (2008), un systematic mapping study:

[...] provides a structure of the type of research reports and results that have been published by categorizing them and often gives a visual summary, the map, of its results. It often requires less effort while providing a more coarse-grained overview.

En este artículo también se explica la necesidad de realizar un mapeo sistemático (o una revisión sistemática):

As a research area matures there is often a sharp increase in the number of reports and results made available, and it becomes important to summarize and provide overview. Many research fields have specific methodologies for such secondary studies, and they have been extensively used in for example evidence based medicine

En este artículo se proponen las siguientes etapas para un mapeo sistemático:

  1. Definition of Research Questions (Research Scope).
  2. Conduct Search for Primary Studies (All Papers).
  3. Screening of Papers for Inclusion and Exclusion (Relevant Papers).
  4. Keywording of Abstracts (Classification Scheme).
  5. Data Extraction and Mapping of Studies (Systematic Map).

El resultado final de este proceso es el mapeo sistemático, que suele hacer uso de diferentes tipos de visualizaciones para mostrar los resultados: diagrama de burbujas (bubble plot), mapa de calor (heat map), diagrama de Venn, tabla de características (characteristics table), etc.

En Guidelines for conducting systematic mapping studies in software engineering: An update (2015) se revisa el concepto de mapeo sistemático (también llamado estudio de alcance, scoping study) y se ofrecen nuevas pautas para su realización:

Systematic mapping studies or scoping studies are designed to give an overview of a research area through classification and counting contributions in relation to the categories of that classification. It involves searching the literature in order to know what topics have been covered in the literature, and where the literature has been published.

Y se explica la diferencia principal con una revisión sistemática:

Though, a systematic mapping study and a systematic literature review share some commonalities (e.g. with respect to searching and study selection), they are different in terms of goals and thus approaches to data analysis. While systematic reviews aim at synthesizing evidence, also considering the strength of evidence, systematic maps are primarily concerned with structuring a research area.

Las diferencias han sido analizadas en The value of mapping studies: a participant-observer case study (2010), donde se indica que un systematic mapping es una forma particular de systematic literature review:

Mapping studies use the same basic methodology as [systematic literature reviews] SLRs but aim to identify all research related to a specific topic rather than addressing the specific questions that conventional SLRs address.

Although at the extreme, mapping studies and systematic reviews have rather different goals, there is often an overlap. Some systematic reviews include a classification system to organise relevant literature followed by a more detailed description of the research within each category.

Un mapeo sistemático emplea los mismos métodos que una revisión sistemática, pero suele ser un estudio más sencillo de realizar ya que no tiene como objetivo analizar los resultados presentados en los artículos que conforman el mapeo. Un mapeo sistemático puede ser la base para realizar posteriormente una revisión sistemática.

En este artículo se presenta la siguiente tabla de diferencias entre un mapeo sistemático y una revisión sistemática:

SLR process Mapping Study SLR
Research Question General – related to research trends. Which researchers, how much activity, what type of studies etc. Specific - related to outcomes of empirical studies. Of the form: Is technology/method A better or not than B?
Search process Defined by topic area Defined by research question
Search strategy requirements Less stringent if only research trends are of interest Extremely stringent – all relevant studies must be found
Quality evaluation Not essential Important to ensure that results are based on best quality evidence
Results Set of papers related to a topic area and counts of the number of papers in various categories Answer to specific research question, possible with qualifiers (e.g. results apply to novices only).

En What is a “mapping study?” (2016) también se indica otra diferencia entre mapeo sistemático y revisión sistemática:

Mapping studies are also reviews, but they do not discuss the findings. They are based on the concept that published articles not only represent findings, but, indirectly, represent activity related to the finding.

It is a review that seeks to identify, not results, but linkages. Mapping focuses on characteristics such as where the activity took place, where the funding came from, and in what journal or other medium it was presented.

En Guidelines for snowballing in systematic literature studies and a replication in software engineering se propone el término systematic literature studies para agrupar los diferentes tipos de revisiones sistemáticas:

Here, we use systematic literature studies as a collective term for systematic literature reviews and systematic mapping studies.

El paso 2 del proceso de un mapeo sistemático es Conduct Search for Primary Studies (All Papers). En Systematic literature studies: database searches vs. backward snowballing se analizan y comparan dos de los métodos clásicos de búsqueda de artículos para realizar una revisión sistemática:

En Guidelines for snowballing in systematic literature studies and a replication in software engineering se explica la técnica de snowballing:

Snowballing refers to using the reference list of a paper or the citations to the paper to identify additional papers. However, snowballing could benefit from not only looking at the reference lists and citations, but to complement it with a systematic way of looking at where papers are actually referenced and where papers are cited. Using the references and the citations respectively is referred to as backward and forward snowballing.

La siguiente imagen muestra un proceso de selección de artículos en un mapeo sistemático:

'Selection process' del artículo 'Guidelines for conducting systematic mapping studies in software engineering: An update' de Petersen, Vakkalanka y Kuzniarz (2015)

El proceso de selección de los artículos influye enormemente en los resultados obtenidos. En Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources se analizó la efectividad para seleccionar los diferentes métodos que se emplean normalmente: protocol-driven search strategies, snowballing y personal knowledge or personal contacts.

Una vez seleccionados los artículos, se deben leer y clasificar. En Analyzing the Past to Prepare for the Future: Writing a Literature Review se explica que una revisión de la literatura se debe centrar en los conceptos tratados por los artículos seleccionados:

A literature review is concept-centric. Thus, concepts determine the organizing framework of a review. In contrast, some authors take an author-centric approach and essentially present a summary of the relevant articles. This method fails to synthesize the literature.

Y se propone el uso de una concept matrix para analizar los artículos:

[...] we recommend that you compile a concept matrix as you read each article, an idea we have adapted from Salipante et al. (1982). When your reading is complete, synthesize the literature by discussing each identified concept. Before commencing this step, take some time to develop a logical approach to grouping and presenting the key concepts you have uncovered.

Referencias básicas

Analyzing the past to prepare for the future: writing a literature review

Cita: Jane Webster, Richard T. Watson. MIS Quarterly, Volume 26 Issue 2, June 2002, Pages xiii-xxiii.

Procedures for Performing Systematic Reviews

Cita: Barbara Kitchenham. Keele University and National ICT Australia Ltd, 2004.

Guidelines for performing Systematic Literature Reviews in Software Engineering

Cita: Barbara Kitchenham, Stuart Charters. Keele University and Durham University Joint Report, 2007.

Systematic mapping studies in software engineering

Cita: Kai Petersen, Robert Feldt, Shahid Mujtaba, Michael Mattsson. Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering (EASE'08), Pages 68-77.

Guidelines for conducting systematic mapping studies in software engineering: An update

Cita: Kai Petersen, Sairam Vakkalanka, Ludwik Kuzniarz. Information and Software Technology, Volume 64, August 2015, Pages 1-18.

Ejemplos de mapeo sistemático (en español)

Evicencia empírica sobre mejoras en productividad y calidad en enfoques MDD: un mapeo sistemático

Cita: Y. Martínez, C. Cachero, S. Meliá, Revista Española de Innovación, Calidad e Ingeniería del Software, 7(2), 2011.

Mapeo sistemático de la literatura sobre la Adopción de Inspecciones de Software

Cita: D. Macchi, M. Solari, Conferencia Latinoamericana de Informática (CLEI 2012).

Modelos de Madurez de Green IT: un mapeo sistemático

Cita: J. David Patón-Romero, Mario Piattini. 2017 12th Iberian Conference on Information Systems and Technologies (CISTI). DOI: 10.23919/CISTI.2017.7975693.

Ejemplos de mapeo sistemático (en inglés)

Software product line testing – A systematic mapping study

Cita: Emelie Engström, Per Runeson, Information and Software Technology, Volume 53, Issue 1, January 2011, Pages 2–13.

Usability evaluation methods for the web: A systematic mapping study

Cita: Adrian Fernandez, Emilio Insfran, Silvia Abrahão, Information and Software Technology, Volume 53, Issue 8, August 2011, Pages 789–817.

The maturity of maturity model research: A systematic mapping study

Cita: Roy Wendler, Information and Software Technology, Volume 54, Issue 12, December 2012, Pages 1317–1339.

Quality of Web Mashups: A Systematic Mapping Study

Cita: Priscila Cedillo, Adrian Fernandez, Emilio Insfran, Silvia Abrahão, ICWE 2013 International Workshops ComposableWeb, QWE, MDWE, DMSSW, EMotions, CSE, SSN, and PhD Symposium, Aalborg, Denmark, July 8-12, 2013, Pages 66-78.