Systematic Literature Review & Mapping
Juan Cruz-Benito
GRIAL Research Group, Department of Computers and Automatics
University of Salamanca, Salamanca, Spain.
juancb@usal.es
@_juancb
Education in the Knowledge Society PhD programme.
University of Salamanca 7/11/2016
DOI: 10.5281/zenodo.165773
Outline
•  Introduction. What is a SLR & Mapping?
•  Goals 
•  Planning the SLR & Mapping
•  Methodology
•  SLR & Mapping, step by step
•  Where to publish a Literature Review
•  Conclusions
•  Acknowledgments
•  References
Introduction
What is a SLR & Mapping?
Introduction. What is a SLR & Mapping
The SLR is a type of literature review that collects and critically
analyzes multiple research studies or papers through a systematic
process.
The purpose of a SLR is to provide a exhaustive summary of the
available literature relevant to a research question.
Introduction. What is a SLR & Mapping
The SLR born in the field of Medicine and Health studies to get
expertise in a topic.
In Healthcare, exists the Cochrane Collaboration group composed by
more than 31000 members that work reviewing systematically
research related to prevention, treatments, rehabilitation and health
systems intervention.
This group publish their reviews in the Cochrane Database of
Systematic Reviews which has an impact factor of 6.103 and is ranked
12th in the “Medicine, General & Internal” in JCR (top 7%).
Introduction. What is a SLR & Mapping
The SLR is not currently restricted to Healthcare. There are many
researchers and organization involved in making SLR in other
knowledge fields.
I.E. the Campbell Collaboration is a sister initiative of Cochrane
Collaboration that deals with SLR in Social Sciences.
Also in other fields like Computer Sciences there is a strong
community that works with SLR and tries to standardize it and spread
its techniques and results in the knowledge area.
Introduction. What is a SLR & Mapping
The Mapping in Literature Reviews (a.k.a. Literature Mapping)
techniques are useful at the very beginning of the literature review as
a brainstorming and scoping tool1.
The literature mapping is broadly used to complement the SLR
1 https://as.exeter.ac.uk/media/universityofexeter/academicservices/educationenhancement/cascade/Mapping_in_literature_reviews.pdf
Introduction. What is a SLR & Mapping
The Mapping Literature techniques/outcomes are very different
depending on the purpose:
•  To write down words, phrases and sub-topics related to the
main topic in a white paper to gather key concepts and issues
•  Summarize key findings from journal, books and working
papers to create concept maps
•  Present a summary of the journals, conferences, publication
years, most important authors, etc. found in the SLR
•  Etc.
Introduction. What is a SLR & Mapping
Systematic Literature Review
≠
Mapping in Literature Reviews
Introduction. What is a SLR & Mapping
Systematic Literature Review
+
Mapping in Literature Reviews
=
Better results
Goals
Goals
•  Deeper knowledge in your knowledge field
•  Get insight about the current trends and future challenges
•  Identify the most important authors
•  Identify the most important journals & conferences
•  Get a (several?) good publication(s)
•  Get citations
Planning the SLR & Mapping
Planning the SLR & Mapping
Several aspects to keep in mind:
•  Scope
•  Time
•  Planned revenue
•  Where to publish
Methodology
Methodology
By Jacknunn - Own work, CC BY-SA 4.0, 
https://commons.wikimedia.org/w/index.php?curid=49168037
Methodology
•  Define research questions (and goals)
•  Define inclusion criteria for your SLR
•  Define exclusion criteria for your SLR
•  Search in scientific databases and extract relevant contents/data
(iterating the process in several stages).
•  Assess the quality of these results
•  Gather the most outstanding results in order to analyze, discuss
and learn from them.
SLR & Mapping, step by step
SLR & Mapping, step by step
1.  RQs
2.  ICs
3.  ECs
4.  PICOC
5.  Databases
6.  Queries
7.  Review phases
8.  Quality assessment
9.  Traceability
10.  Write results
SLR & Mapping, step by step
Research Questions
SLR & Mapping, step by step
Research questions: Mapping
SLR & Mapping, step by step
Research questions: SLR
SLR & Mapping, step by step
PICOC
SLR & Mapping, step by step
PICOC: defining the SRL scope. This scope helps in the papers analysis
to answer the research questions
•  Population (P)
•  Intervention (I)
•  Comparison (C)
•  Outcomes (O)
•  Context (C)
SLR & Mapping, step by step
PICOC
SLR & Mapping, step by step
Inclusion Criteria
SLR & Mapping, step by step
Inclusion Criteria
SLR & Mapping, step by step
Exclusion Criteria
SLR & Mapping, step by step
Exclusion Criteria
SLR & Mapping, step by step
Databases
SLR & Mapping, step by step
Databases:
WoS, Scopus, Google Scholar, IEEEXplore, ACM, Springer, ERIC,
Pubmed, ScienceDirect, Compendex, etc.
**Not limited only to major databases.
SLR & Mapping, step by step
Queries
SLR & Mapping, step by step
The queries between the different database where the researcher
search for results should be the same or equivalent
(if not, the results gathered would not be comparable)
SLR & Mapping, step by step
Review phases
SLR & Mapping, step by step
Review phases. Typical steps:
1.  Execute query
2.  Remove duplicates
3.  Review by regarding titles and abstracts (applying IC, EC)
4.  Review the full text & assess quality (applying also IC, EC)
5.  Include (if necessary) papers cited in your results and repeat
SLR & Mapping, step by step
Review phases
SLR & Mapping, step by step
Quality assessment
SLR & Mapping, step by step
Quality assessment: checklist
•  The researcher would assess quality using a checklist to evaluate
the aspects relevant for the SLR in each paper
•  Depending on the evaluation score, each paper would be included
or excluded in the final phase. The researcher will fix the cuttoff
point.
SLR & Mapping, step by step
Quality assessment: checklist
SLR & Mapping, step by step
Traceability
SLR & Mapping, step by step
Traceability
•  You must provide full explanations on how was carried the process
•  You should include the papers reviewed in each phase. Depending
the review phase you will required to specify the IC, EC used to
select or reject the paper in the SLR.
•  If you do not provide these explanations, the reviewer/thesis
supervisor will not be able to trust your research (and you).
SLR & Mapping, step by step
Traceability
•  Most of these information cannot be included in a journal/
conference paper.
•  Too much extension / visual fatigue (in the case of huge tables)
SLR & Mapping, step by step
Traceability
•  One solution: use Google Spreadsheets
•  https://docs.google.com/spreadsheets/d/
169RCtU7Q4Qq1eryq1d6c1XnMzDEhgyptYqZh1C8eaYA/edit#gid=0
•  Other solutions: use Websites
•  https://sites.google.com/site/francilaneiva/research/pragmatic-
interoperability-a-systematic-mapping
SLR & Mapping, step by step
Write results
SLR & Mapping, step by step
Write results:
•  Usually the resultant paper will have one section for the mapping
report and other for the systematic
•  Each one should respond the research questions and provide
insights about the paper and contents selected for that.
•  Use charts, tables and visual explanations
SLR & Mapping, step by step
Write results.
SLR & Mapping, step by step
Write results.
SLR & Mapping, step by step
Write results.
SLR & Mapping, step by step
Write results.
SLR & Mapping, step by step
Write results.
Where to publish a Literature Review & Mapping
Where to publish a Literature Review & Mapping
•  Conferences
•  Journals
•  Books
Where to publish a Literature Review & Mapping:
examples
Where to publish a Literature Review & Mapping
•  TEEM Conference
•  Other conferences (HCI International, Interacción, SIIE, AIDIPE?)
•  PLOS ONE
•  Education in the Knowledge Society (EKS)
•  International Journal of Knowledge Management (IJKM).
•  Health Education
Where to publish a Literature Review & Mapping
•  International Journal of Law and Management
•  International Journal of Productivity and Performance Management
•  Computers in Human Behavior
•  Information and Software Technology
•  IEEE Transactions on Software Engineering
•  Computers & Education
•  …
Conclusions
Conclusions
•  Strengths
•  This kind of reviews are regarded as the strongest in many
knowledge areas
•  Many organizations fund SLR processes for develop research
•  A good SLR in a journal that accepts this kind of research, has
real choices to be published
Conclusions
•  Weaknesses
•  Publisher bias
•  If the review takes too much time, you will need to re-do some
parts after a while
•  Should be extended usually to other databases apart of the main
ones. This will help the SLR effectiveness
•  Depending the publisher, you will need to cut some parts of your
SLR (regarding papers extension)
Conclusions
•  To publish your SLR & Mapping:
•  Find what journals/conference use to publish them and the
latest SLR papers published
•  Prepare your paper version of the SLR based on these latest
papers published previously to your submission
•  If you will make a huge SLR, only part of it would be published by
a journal/conference. The other part of the content/results
could be available in your full Thesis volume
Do a SLR!
Acknowledgments
Acknowledgments
I would like to thanks the European Social Fund and the Consejería de
Educación of the Junta de Castilla y León (Spain) for funding my
pre-doctoral fellow contract.
Also thanks to my PhD supervisors that finally convinced
me to start my SLRs
References
References
•  Fink, A. (1998). Conducting literature research reviews: from paper to the internet. Thousand Oaks, CA:
Sage
•  Neiva, F. W., David, J. M. N., Braga, R., & Campos, F. (2016). Towards pragmatic interoperability to support
collaboration: A systematic review and mapping of the literature. Information and Software Technology,
72, 137-150.
•  Kitchenham, B. A., Budgen, D., & Brereton, O. P. (2011). Using mapping studies as the basis for further
research–a participant-observer case study. Information and Software Technology, 53(6), 638-651.
•  Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University,
33(2004), 1-26.
•  B.A. Kitchenham, S. Charters, Guidelines For Performing Systematic Litera ture Reviews in Software
Engineering, EBSE Technical Report EBSE-2007-01, 2007.URL
http://www.rbsv.eu/courses/rmtw/mtrl/SLR.pdf.
•  Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. John
Wiley & Sons.
References
•  Cruz-Benito, J., Therón, R., & García-Peñalvo, F. J. (2016, July). Software architectures supporting human-computer
interaction analysis: A literature review. In International Conference on Learning and Collaboration Technologies (pp.
125-136). Springer International Publishing.
•  Neiva, F. W., David, J. M. N., Braga, R., & Campos, F. (2016). Towards pragmatic interoperability to support collaboration: A
systematic review and mapping of the literature. Information and Software Technology, 72, 137-150.
•  Radant, O., Colomo-Palacios, R., & Stantchev, V. (2014). Analysis of Reasons, Implications and Consequences of
Demographic Change for IT Departments in Times of Scarcity of Talent: A Systematic Review. International Journal of
Knowledge Management (IJKM), 10(4), 1-15.
•  Chilton, R., Pearson, M., & Anderson, R. (2015). Health promotion in schools: a scoping review of systematic reviews. Health
Education, 115(3/4), 357-376.
•  Sepúlveda, S., Cravero, A., & Cachero, C. (2016). Requirements modeling languages for software product lines: A
systematic literature review. Information and Software Technology, 69, 16-36.
•  Chei-Chang Chiou (2009) Effects of concept mapping strategy on learning performance in business and economics
statistics. Teaching in Higher Education. Feb2009, Vol. 14 Issue 1, p55-69.
This presentation is available in
http://www.slideshare.net/knowedgesociety/systematic-literature-review-mapping
http://repositorio.grial.eu/handle/grial/685
Systematic Literature Review & Mapping
Juan Cruz-Benito
GRIAL Research Group, Department of Computers and Automatics
University of Salamanca, Salamanca, Spain.
juancb@usal.es
@_juancb
Education in the Knowledge Society PhD programme.
University of Salamanca 7/11/2016
DOI: 10.5281/zenodo.165773

More Related Content

PPTX
Systematic literature review: An introduction
PPTX
PRISMA flow diagram in systematic literature review
PDF
Introduction to Systematic Literature Review method
PPTX
How to conduct systematic literature review
PDF
Systematic Literature Review
PPTX
Introduction to systematic literature review
PPT
Systematic Reviews: the process, quantitative, qualitative and mixed methods ...
PDF
Review of literature - systematic review
Systematic literature review: An introduction
PRISMA flow diagram in systematic literature review
Introduction to Systematic Literature Review method
How to conduct systematic literature review
Systematic Literature Review
Introduction to systematic literature review
Systematic Reviews: the process, quantitative, qualitative and mixed methods ...
Review of literature - systematic review

What's hot (20)

PDF
Research metrics Apr2013
PPTX
How to do a Literature Review
PPT
Session 2 Literature Review
PDF
Literature Review
PPTX
Research methods
PPTX
Research gap
PDF
Research Publications in Scopus
PPTX
Quantitative reseach method
PDF
4. review of literature
PPTX
Literature search techniques
PPT
literature review
PDF
How to Identify the Research Gap While Writing a PhD Dissertation Literature ...
PPT
Literature review (1)
PPTX
How to write and Publish Research
PDF
Updated - What makes systematic review systematic - Anna Sidorchuk
PPTX
Research Process: Selecting and Evaluating Sources
PPT
Literature Review
PPTX
Research Design
PPTX
Literature Review.pptx
Research metrics Apr2013
How to do a Literature Review
Session 2 Literature Review
Literature Review
Research methods
Research gap
Research Publications in Scopus
Quantitative reseach method
4. review of literature
Literature search techniques
literature review
How to Identify the Research Gap While Writing a PhD Dissertation Literature ...
Literature review (1)
How to write and Publish Research
Updated - What makes systematic review systematic - Anna Sidorchuk
Research Process: Selecting and Evaluating Sources
Literature Review
Research Design
Literature Review.pptx
Ad

Viewers also liked (20)

PPT
Systematic review
PDF
The Systematic Review of Literature in LIS: an approach
PPTX
Systematic review
PPTX
Desarrollo y evaluación de competencias emocionales para profesores mediante ...
PPT
Systematic review ppt
PDF
News Production Workflows in Data- driven, Algorithmic Journalism: A Systema...
PDF
Systematic Reviews and Research Synthesis, Part 2
PPTX
Awakening the Entrepreneurial Expertise
PPTX
Schreiben mit System I für Doktoranden der Medizin
PPTX
Sunday Morning - Effectuation & pitch
PDF
What makes entrepeneurs
PDF
digitizer // inspiration : 7 Erfolgsfaktoren. So gelingt die digitale Transfo...
PPTX
13 02-13 Dissertation
PPTX
Systematic Reviews: the researcher's perspective and the research question. E...
PDF
La revisión de la literatura como punto de partida de la investigación en Edu...
PPTX
Effectual thinking in entrepreneurship education - Nick Williams, Tim Vorley,...
PPTX
Effectuation presentation gc20151210 (4)
PDF
Systematic review: teams, processes, experiences
Systematic review
The Systematic Review of Literature in LIS: an approach
Systematic review
Desarrollo y evaluación de competencias emocionales para profesores mediante ...
Systematic review ppt
News Production Workflows in Data- driven, Algorithmic Journalism: A Systema...
Systematic Reviews and Research Synthesis, Part 2
Awakening the Entrepreneurial Expertise
Schreiben mit System I für Doktoranden der Medizin
Sunday Morning - Effectuation & pitch
What makes entrepeneurs
digitizer // inspiration : 7 Erfolgsfaktoren. So gelingt die digitale Transfo...
13 02-13 Dissertation
Systematic Reviews: the researcher's perspective and the research question. E...
La revisión de la literatura como punto de partida de la investigación en Edu...
Effectual thinking in entrepreneurship education - Nick Williams, Tim Vorley,...
Effectuation presentation gc20151210 (4)
Systematic review: teams, processes, experiences
Ad

Similar to Systematic Literature Review & Mapping (20)

PDF
1 - Systematic Literature Reviews: introduction and methods
PPTX
Literature review for a dissertation: a step-by-step guide
PPTX
Literature Review (Review of Related Literature - Research Methodology)
PDF
presentationlitreview-2ND COT PR2.pdf
DOCX
PPTX
Systematic literature review technique.pptx
PPT
HOW TO WRITING A REVIEW LITERATURES IN PPT.ppt
PDF
Sgd literature review-1
PPTX
Writing Research Proposals and Publications: Getting started
PPTX
Lecture 6 - Literature Review.pptx
PPT
Literature Review
PDF
systematic review info.pdf
PPT
HLTH4909_2022xzccdscaascsdcsdzxczccs.ppt
PDF
A Federated Search Approach to Facilitate Systematic Literature Review in Sof...
PDF
Part 1 Research workshop
PPT
Systematic Literature Reviews : Concise Overview
PDF
The Essential Guide to Doing a Health and Social Care Literature Review-Routl...
PPTX
7.REVIEW OF RELATED LITERATURE (RRL).pptx
PDF
Literature Survey-Lect. 4.pdf
PPSX
Chapter 2 incorporating theory and conducting literature search and review
1 - Systematic Literature Reviews: introduction and methods
Literature review for a dissertation: a step-by-step guide
Literature Review (Review of Related Literature - Research Methodology)
presentationlitreview-2ND COT PR2.pdf
Systematic literature review technique.pptx
HOW TO WRITING A REVIEW LITERATURES IN PPT.ppt
Sgd literature review-1
Writing Research Proposals and Publications: Getting started
Lecture 6 - Literature Review.pptx
Literature Review
systematic review info.pdf
HLTH4909_2022xzccdscaascsdcsdzxczccs.ppt
A Federated Search Approach to Facilitate Systematic Literature Review in Sof...
Part 1 Research workshop
Systematic Literature Reviews : Concise Overview
The Essential Guide to Doing a Health and Social Care Literature Review-Routl...
7.REVIEW OF RELATED LITERATURE (RRL).pptx
Literature Survey-Lect. 4.pdf
Chapter 2 incorporating theory and conducting literature search and review

More from Education in the Knowledge Society PhD (20)

PPTX
Presentación de la defensa de la tesis de Sandra Liliana Torres Taborda
PPTX
Presentación de la defensa de la tesis de Martín Adalberto Tena Espinoza de l...
PPTX
Presentación de la defensa de la tesis de Li Yang
PDF
Presentación de la defensa de la tesis de Luis González Rodero
PDF
Defensa de la Tesis de Alberto Ortiz López
PDF
Kick-off Meeting del Programa de Doctorado curso 2023-2024
PDF
Evaluación de las necesidades del profesorado de Educación Infantil y Primari...
PDF
Kick-off Meeting del Programa de Doctorado curso 2022-2023
PPTX
University Teachers and Students’ Digital Competence: A Case Study of Gansu A...
PDF
La brecha de género y etnicidad en estudios pre y universitarios del sector S...
PDF
La formación en competencia digital dirigida a los profesionales de Salud en ...
PDF
Modelo evaluativo de formación docente basada en la práctica para potenciar m...
PDF
EL PODER DEL AUDIO DIGITAL Radio y podcast nuevas oportunidades de negocio
PDF
Kick-off PhD Programme 2021-2022
PDF
Aporte del Análisis Estadístico Implicativo a Learning Analytics
PPTX
Bibliotecas invisibles: sistemas, ciudades y representaciones sociales de la...
PDF
La comunicación de datos en España
PDF
Desarrollo de nuevas férulas de inmovilización sensorizadas para la generació...
PDF
Modelo de Educomunicación Intercultural para la Formación de Emprendedoras de...
PDF
Competencia digital docente como contribución a estimular procesos de Innovac...
Presentación de la defensa de la tesis de Sandra Liliana Torres Taborda
Presentación de la defensa de la tesis de Martín Adalberto Tena Espinoza de l...
Presentación de la defensa de la tesis de Li Yang
Presentación de la defensa de la tesis de Luis González Rodero
Defensa de la Tesis de Alberto Ortiz López
Kick-off Meeting del Programa de Doctorado curso 2023-2024
Evaluación de las necesidades del profesorado de Educación Infantil y Primari...
Kick-off Meeting del Programa de Doctorado curso 2022-2023
University Teachers and Students’ Digital Competence: A Case Study of Gansu A...
La brecha de género y etnicidad en estudios pre y universitarios del sector S...
La formación en competencia digital dirigida a los profesionales de Salud en ...
Modelo evaluativo de formación docente basada en la práctica para potenciar m...
EL PODER DEL AUDIO DIGITAL Radio y podcast nuevas oportunidades de negocio
Kick-off PhD Programme 2021-2022
Aporte del Análisis Estadístico Implicativo a Learning Analytics
Bibliotecas invisibles: sistemas, ciudades y representaciones sociales de la...
La comunicación de datos en España
Desarrollo de nuevas férulas de inmovilización sensorizadas para la generació...
Modelo de Educomunicación Intercultural para la Formación de Emprendedoras de...
Competencia digital docente como contribución a estimular procesos de Innovac...

Recently uploaded (20)

PDF
Diabetes Mellitus , types , clinical picture, investigation and managment
PDF
LATAM’s Top EdTech Innovators Transforming Learning in 2025.pdf
PPTX
IT infrastructure and emerging technologies
PPTX
Neurology of Systemic disease all systems
PPTX
MMW-CHAPTER-1-final.pptx major Elementary Education
PDF
FYJC - Chemistry textbook - standard 11.
PDF
Physical pharmaceutics two in b pharmacy
PDF
WHAT NURSES SAY_ COMMUNICATION BEHAVIORS ASSOCIATED WITH THE COMP.pdf
PDF
Laparoscopic Imaging Systems at World Laparoscopy Hospital
PPSX
namma_kalvi_12th_botany_chapter_9_ppt.ppsx
PDF
Unleashing the Potential of the Cultural and creative industries
DOCX
THEORY AND PRACTICE ASSIGNMENT SEMESTER MAY 2025.docx
PPTX
Power Point PR B.Inggris 12 Ed. 2019.pptx
PPT
hemostasis and its significance, physiology
PDF
African Communication Research: A review
PDF
GIÁO ÁN TIẾNG ANH 7 GLOBAL SUCCESS (CẢ NĂM) THEO CÔNG VĂN 5512 (2 CỘT) NĂM HỌ...
PDF
Horaris_Grups_25-26_Definitiu_15_07_25.pdf
PPTX
2025 High Blood Pressure Guideline Slide Set.pptx
PDF
anganwadi services for the b.sc nursing and GNM
PDF
Disorder of Endocrine system (1).pdfyyhyyyy
Diabetes Mellitus , types , clinical picture, investigation and managment
LATAM’s Top EdTech Innovators Transforming Learning in 2025.pdf
IT infrastructure and emerging technologies
Neurology of Systemic disease all systems
MMW-CHAPTER-1-final.pptx major Elementary Education
FYJC - Chemistry textbook - standard 11.
Physical pharmaceutics two in b pharmacy
WHAT NURSES SAY_ COMMUNICATION BEHAVIORS ASSOCIATED WITH THE COMP.pdf
Laparoscopic Imaging Systems at World Laparoscopy Hospital
namma_kalvi_12th_botany_chapter_9_ppt.ppsx
Unleashing the Potential of the Cultural and creative industries
THEORY AND PRACTICE ASSIGNMENT SEMESTER MAY 2025.docx
Power Point PR B.Inggris 12 Ed. 2019.pptx
hemostasis and its significance, physiology
African Communication Research: A review
GIÁO ÁN TIẾNG ANH 7 GLOBAL SUCCESS (CẢ NĂM) THEO CÔNG VĂN 5512 (2 CỘT) NĂM HỌ...
Horaris_Grups_25-26_Definitiu_15_07_25.pdf
2025 High Blood Pressure Guideline Slide Set.pptx
anganwadi services for the b.sc nursing and GNM
Disorder of Endocrine system (1).pdfyyhyyyy

Systematic Literature Review & Mapping

  • 1. Systematic Literature Review & Mapping Juan Cruz-Benito GRIAL Research Group, Department of Computers and Automatics University of Salamanca, Salamanca, Spain. [email protected] @_juancb Education in the Knowledge Society PhD programme. University of Salamanca 7/11/2016 DOI: 10.5281/zenodo.165773
  • 2. Outline •  Introduction. What is a SLR & Mapping? •  Goals •  Planning the SLR & Mapping •  Methodology •  SLR & Mapping, step by step •  Where to publish a Literature Review •  Conclusions •  Acknowledgments •  References
  • 3. Introduction What is a SLR & Mapping?
  • 4. Introduction. What is a SLR & Mapping The SLR is a type of literature review that collects and critically analyzes multiple research studies or papers through a systematic process. The purpose of a SLR is to provide a exhaustive summary of the available literature relevant to a research question.
  • 5. Introduction. What is a SLR & Mapping The SLR born in the field of Medicine and Health studies to get expertise in a topic. In Healthcare, exists the Cochrane Collaboration group composed by more than 31000 members that work reviewing systematically research related to prevention, treatments, rehabilitation and health systems intervention. This group publish their reviews in the Cochrane Database of Systematic Reviews which has an impact factor of 6.103 and is ranked 12th in the “Medicine, General & Internal” in JCR (top 7%).
  • 6. Introduction. What is a SLR & Mapping The SLR is not currently restricted to Healthcare. There are many researchers and organization involved in making SLR in other knowledge fields. I.E. the Campbell Collaboration is a sister initiative of Cochrane Collaboration that deals with SLR in Social Sciences. Also in other fields like Computer Sciences there is a strong community that works with SLR and tries to standardize it and spread its techniques and results in the knowledge area.
  • 7. Introduction. What is a SLR & Mapping The Mapping in Literature Reviews (a.k.a. Literature Mapping) techniques are useful at the very beginning of the literature review as a brainstorming and scoping tool1. The literature mapping is broadly used to complement the SLR 1 https://as.exeter.ac.uk/media/universityofexeter/academicservices/educationenhancement/cascade/Mapping_in_literature_reviews.pdf
  • 8. Introduction. What is a SLR & Mapping The Mapping Literature techniques/outcomes are very different depending on the purpose: •  To write down words, phrases and sub-topics related to the main topic in a white paper to gather key concepts and issues •  Summarize key findings from journal, books and working papers to create concept maps •  Present a summary of the journals, conferences, publication years, most important authors, etc. found in the SLR •  Etc.
  • 9. Introduction. What is a SLR & Mapping Systematic Literature Review ≠ Mapping in Literature Reviews
  • 10. Introduction. What is a SLR & Mapping Systematic Literature Review + Mapping in Literature Reviews = Better results
  • 11. Goals
  • 12. Goals •  Deeper knowledge in your knowledge field •  Get insight about the current trends and future challenges •  Identify the most important authors •  Identify the most important journals & conferences •  Get a (several?) good publication(s) •  Get citations
  • 13. Planning the SLR & Mapping
  • 14. Planning the SLR & Mapping Several aspects to keep in mind: •  Scope •  Time •  Planned revenue •  Where to publish
  • 16. Methodology By Jacknunn - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=49168037
  • 17. Methodology •  Define research questions (and goals) •  Define inclusion criteria for your SLR •  Define exclusion criteria for your SLR •  Search in scientific databases and extract relevant contents/data (iterating the process in several stages). •  Assess the quality of these results •  Gather the most outstanding results in order to analyze, discuss and learn from them.
  • 18. SLR & Mapping, step by step
  • 19. SLR & Mapping, step by step 1.  RQs 2.  ICs 3.  ECs 4.  PICOC 5.  Databases 6.  Queries 7.  Review phases 8.  Quality assessment 9.  Traceability 10.  Write results
  • 20. SLR & Mapping, step by step Research Questions
  • 21. SLR & Mapping, step by step Research questions: Mapping
  • 22. SLR & Mapping, step by step Research questions: SLR
  • 23. SLR & Mapping, step by step PICOC
  • 24. SLR & Mapping, step by step PICOC: defining the SRL scope. This scope helps in the papers analysis to answer the research questions •  Population (P) •  Intervention (I) •  Comparison (C) •  Outcomes (O) •  Context (C)
  • 25. SLR & Mapping, step by step PICOC
  • 26. SLR & Mapping, step by step Inclusion Criteria
  • 27. SLR & Mapping, step by step Inclusion Criteria
  • 28. SLR & Mapping, step by step Exclusion Criteria
  • 29. SLR & Mapping, step by step Exclusion Criteria
  • 30. SLR & Mapping, step by step Databases
  • 31. SLR & Mapping, step by step Databases: WoS, Scopus, Google Scholar, IEEEXplore, ACM, Springer, ERIC, Pubmed, ScienceDirect, Compendex, etc. **Not limited only to major databases.
  • 32. SLR & Mapping, step by step Queries
  • 33. SLR & Mapping, step by step The queries between the different database where the researcher search for results should be the same or equivalent (if not, the results gathered would not be comparable)
  • 34. SLR & Mapping, step by step Review phases
  • 35. SLR & Mapping, step by step Review phases. Typical steps: 1.  Execute query 2.  Remove duplicates 3.  Review by regarding titles and abstracts (applying IC, EC) 4.  Review the full text & assess quality (applying also IC, EC) 5.  Include (if necessary) papers cited in your results and repeat
  • 36. SLR & Mapping, step by step Review phases
  • 37. SLR & Mapping, step by step Quality assessment
  • 38. SLR & Mapping, step by step Quality assessment: checklist •  The researcher would assess quality using a checklist to evaluate the aspects relevant for the SLR in each paper •  Depending on the evaluation score, each paper would be included or excluded in the final phase. The researcher will fix the cuttoff point.
  • 39. SLR & Mapping, step by step Quality assessment: checklist
  • 40. SLR & Mapping, step by step Traceability
  • 41. SLR & Mapping, step by step Traceability •  You must provide full explanations on how was carried the process •  You should include the papers reviewed in each phase. Depending the review phase you will required to specify the IC, EC used to select or reject the paper in the SLR. •  If you do not provide these explanations, the reviewer/thesis supervisor will not be able to trust your research (and you).
  • 42. SLR & Mapping, step by step Traceability •  Most of these information cannot be included in a journal/ conference paper. •  Too much extension / visual fatigue (in the case of huge tables)
  • 43. SLR & Mapping, step by step Traceability •  One solution: use Google Spreadsheets •  https://docs.google.com/spreadsheets/d/ 169RCtU7Q4Qq1eryq1d6c1XnMzDEhgyptYqZh1C8eaYA/edit#gid=0 •  Other solutions: use Websites •  https://sites.google.com/site/francilaneiva/research/pragmatic- interoperability-a-systematic-mapping
  • 44. SLR & Mapping, step by step Write results
  • 45. SLR & Mapping, step by step Write results: •  Usually the resultant paper will have one section for the mapping report and other for the systematic •  Each one should respond the research questions and provide insights about the paper and contents selected for that. •  Use charts, tables and visual explanations
  • 46. SLR & Mapping, step by step Write results.
  • 47. SLR & Mapping, step by step Write results.
  • 48. SLR & Mapping, step by step Write results.
  • 49. SLR & Mapping, step by step Write results.
  • 50. SLR & Mapping, step by step Write results.
  • 51. Where to publish a Literature Review & Mapping
  • 52. Where to publish a Literature Review & Mapping •  Conferences •  Journals •  Books
  • 53. Where to publish a Literature Review & Mapping: examples
  • 54. Where to publish a Literature Review & Mapping •  TEEM Conference •  Other conferences (HCI International, Interacción, SIIE, AIDIPE?) •  PLOS ONE •  Education in the Knowledge Society (EKS) •  International Journal of Knowledge Management (IJKM). •  Health Education
  • 55. Where to publish a Literature Review & Mapping •  International Journal of Law and Management •  International Journal of Productivity and Performance Management •  Computers in Human Behavior •  Information and Software Technology •  IEEE Transactions on Software Engineering •  Computers & Education •  …
  • 57. Conclusions •  Strengths •  This kind of reviews are regarded as the strongest in many knowledge areas •  Many organizations fund SLR processes for develop research •  A good SLR in a journal that accepts this kind of research, has real choices to be published
  • 58. Conclusions •  Weaknesses •  Publisher bias •  If the review takes too much time, you will need to re-do some parts after a while •  Should be extended usually to other databases apart of the main ones. This will help the SLR effectiveness •  Depending the publisher, you will need to cut some parts of your SLR (regarding papers extension)
  • 59. Conclusions •  To publish your SLR & Mapping: •  Find what journals/conference use to publish them and the latest SLR papers published •  Prepare your paper version of the SLR based on these latest papers published previously to your submission •  If you will make a huge SLR, only part of it would be published by a journal/conference. The other part of the content/results could be available in your full Thesis volume
  • 62. Acknowledgments I would like to thanks the European Social Fund and the Consejería de Educación of the Junta de Castilla y León (Spain) for funding my pre-doctoral fellow contract. Also thanks to my PhD supervisors that finally convinced me to start my SLRs
  • 64. References •  Fink, A. (1998). Conducting literature research reviews: from paper to the internet. Thousand Oaks, CA: Sage •  Neiva, F. W., David, J. M. N., Braga, R., & Campos, F. (2016). Towards pragmatic interoperability to support collaboration: A systematic review and mapping of the literature. Information and Software Technology, 72, 137-150. •  Kitchenham, B. A., Budgen, D., & Brereton, O. P. (2011). Using mapping studies as the basis for further research–a participant-observer case study. Information and Software Technology, 53(6), 638-651. •  Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1-26. •  B.A. Kitchenham, S. Charters, Guidelines For Performing Systematic Litera ture Reviews in Software Engineering, EBSE Technical Report EBSE-2007-01, 2007.URL http://www.rbsv.eu/courses/rmtw/mtrl/SLR.pdf. •  Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. John Wiley & Sons.
  • 65. References •  Cruz-Benito, J., Therón, R., & García-Peñalvo, F. J. (2016, July). Software architectures supporting human-computer interaction analysis: A literature review. In International Conference on Learning and Collaboration Technologies (pp. 125-136). Springer International Publishing. •  Neiva, F. W., David, J. M. N., Braga, R., & Campos, F. (2016). Towards pragmatic interoperability to support collaboration: A systematic review and mapping of the literature. Information and Software Technology, 72, 137-150. •  Radant, O., Colomo-Palacios, R., & Stantchev, V. (2014). Analysis of Reasons, Implications and Consequences of Demographic Change for IT Departments in Times of Scarcity of Talent: A Systematic Review. International Journal of Knowledge Management (IJKM), 10(4), 1-15. •  Chilton, R., Pearson, M., & Anderson, R. (2015). Health promotion in schools: a scoping review of systematic reviews. Health Education, 115(3/4), 357-376. •  Sepúlveda, S., Cravero, A., & Cachero, C. (2016). Requirements modeling languages for software product lines: A systematic literature review. Information and Software Technology, 69, 16-36. •  Chei-Chang Chiou (2009) Effects of concept mapping strategy on learning performance in business and economics statistics. Teaching in Higher Education. Feb2009, Vol. 14 Issue 1, p55-69.
  • 66. This presentation is available in http://www.slideshare.net/knowedgesociety/systematic-literature-review-mapping http://repositorio.grial.eu/handle/grial/685
  • 67. Systematic Literature Review & Mapping Juan Cruz-Benito GRIAL Research Group, Department of Computers and Automatics University of Salamanca, Salamanca, Spain. [email protected] @_juancb Education in the Knowledge Society PhD programme. University of Salamanca 7/11/2016 DOI: 10.5281/zenodo.165773