How to Write a Systematic Review of the Literature Health Environments Research & Design Journal 1-16 ª The Author(s) 2017 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1937586717747384 journals.sagepub.com/home/her Debajyoti Pati, PhD, FIIA, IDEC, LEED AP1, and Lesa N. Lorusso, MBA, MS Arch2 Abstract This article provides a step-by-step approach to conducting and reporting systematic literature reviews (SLRs) in the domain of healthcare design and discusses some of the key quality issues associated with SLRs. SLR, as the name implies, is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. The Cochrane Collaboration is the most well-known and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), like the Cochrane Collaboration, was created by an international network of health-based collaborators and provides the framework for SLR to ensure methodological rigor and quality. The PRISMA statement is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses. Keywords systematic literature review, healthcare design, evidence based design As the name suggests, systematic literature review (SLR) is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. It is “systematic” since it adopts a consistent, widely accepted, methodology. The methodology should address concerns regarding quality issues, such as bias, replicability, credibility, et cetera. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. Imagine a single cross-sectional study as a single snapshot in time and a longitudinal study as a single video. As the number of snapshots and videos available on a question or topic increases, it offers a better and more accurate grasp on the phenomena of interest. Since a single research is just one snapshot or video, 1 2 Texas Tech University, Lubbock, TX, USA University of Florida, Gainesville, FL, USA Corresponding Author: Debajyoti Pati, PhD, FIIA, IDEC, LEED AP, Texas Tech University, Lubbock, TX 79409, USA. Email: d.pati@ttu.edu 2 Health Environments Research & Design Journal XX(X) basing design or policy decisions on a single research is not as robust as basing decisions on findings from multiple studies. There are, of course, issues related to the quality of snapshots or videos, and the viewpoint from which those were taken. Poor quality could result in less clarity and, hence, less confidence. Changes in viewpoint could offer a different perspective of a phenomena or introduce bias. Even with appropriate measures for quality, findings from individual studies may vary owing to differences in population, setting, instruments, random chance, and so on. A properly conducted SLR considers the quality of issues and incorporates multiple viewpoints, providing a broader and more complete picture. SLRs are particularly gaining importance in the healthcare design field. Starting from about 600 published studies in 2004 (Ulrich, Zimring, Quan, Joseph, & Choudhary, 2004), the number of studies in the field has increased to a level that facilitates SLRs on important questions. This growing body of knowledge is making SLRs in the healthcare design field both possible and critically important. From single-patient rooms to decentralization, there is a wide range of topics of importance to efficiency, safety, care quality, cost, patient/family/staff experience, and so on, which could benefit from SLRs. Articles on SLRs are available in other fields of studies. The purpose of this article is to provide a step-by-step approach to conducting and reporting SLRs in the domain of healthcare design and discuss some of the key quality issues associated with SLRs. The purpose of this article is to provide a step-by-step approach to conducting and reporting SLRs in the domain of healthcare design and discuss some of the key quality issues associated with SLRs. Types of SLRs SLR began within the realm of healthcare and can be traced back to the work of Scottish naval surgeon James Lind who, in 1753, conducted the first randomized controlled trial (RCT) and recognized systematic methods as a key component in avoiding bias, which remains a pivotal goal of systematic research (Dunn, 1997). By the 1970s, metaanalysis was formalized as a process of research synthesis (Glass, 1976) and evidence-based policy and practice gained momentum in the 1990s. SLRs are only one type of review of literature. The type of review one needs to conduct depends on the objective of the review. Broadly speaking, the two most commonly known types of literature reviews are the “traditional” or narrative review and the “gold standard” or SLR (Booth, Sutton, & Papaioannou, 2016). Traditional reviews, also known as conventional or nonsystematic reviews (Greenhalgh, 2014), are generally faster and easier to conduct and are sometimes appropriate due to a short time frame or lack of resources. Grant and Booth (2009) provide a more comprehensive classification of 14 different types of reviews in the fields of medicine and library sciences. In general, reviews differ in the question being asked, including, among others: What is the current theoretical framework/understanding on a topic/ domain—based on either just peer-reviewed publications or all sources? What are some gaps, in theory or evidence, in existing published literature? Is there sufficient justification for conducting a study? What is the most appropriate study question/topic? Which research question/topic will provide the best return on investment? What is the effect size of an intervention, based on quantitative findings from previous studies? How did a certain past intervention (design, policy, practice, etc.) work? What is the volume of literature available on a topic/domain of interest? The key variations among these review types are in the primary audience group being targeted (intended end use of the review), scope, level of interpretation, level of evaluation of individual study quality, type of information included (published literature vs. other information sources), type of data included (qualitative, quantitative, or both), depth of analysis of published data/findings, level of synthesis, level of investigator bias, level of publication bias, extent of time commitment by the review team, and the type and amount of resources needed to conduct a review, among others. Among these 14 different review types articulated by Grant and Booth (2009), 3 are of relevance to the current discussion, namely: (1) meta-analysis, (2) qualitative systematic review or qualitative Pati and Lorusso evidence synthesis, and (3) systematic review. These three types of reviews share several common attributes: need for systematic search, need for comprehensiveness, need for study quality assessment, need for interpretation, and, most importantly, the need for transparency of the methodology such that the study can be replicated. The variations between the three types of review originate from the long history and maturation of science and publication in medicine and library sciences, resulting in more targeted review types. Meta-analysis uses statistical analysis procedures on quantitative data to estimate the effect size of a specific intervention, where the input data are the end products of the statistical procedures of a number of studies with vastly similar questions, setting, population, intervention, and time period. The authors believe that the healthcare design research field does not currently enjoy sufficient volume of quantitative studies with the above similarities, which may, however, change as the field matures. The intent of a metaanalysis is similar to those of a systematic review. The objective of a qualitative systematic review or a qualitative evidence synthesis is similar to meta-analysis, except that these reviews deal primarily with qualitative studies, and hence the outcomes and resulting end uses are different. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. It attempts to capture the broadest set of available literature on the topic of interest. After assessing the quality of individual studies, SLRs may eliminate low-quality studies from further consideration. Moreover, owing to the extensive documentation and reporting of the steps and assumptions, SLRs renders itself amenable for replication. It demands a team effort (at least two) to eliminate bias, among other issues. SLRs are inherently time intensive. Owing to the aforementioned quality demands, SLRs is regarded as the most robust level of evidence available on a topic or question of interest. SLRs is regarded as the most robust level of evidence available on a topic or question of interest. 3 Key Steps in Conducting SLRs The first and most crucial step in conducting an SLR is to familiarize oneself with generally accepted standards on the subject. Today, the Cochrane Collaboration is the most wellknown and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Founded in 1993, Cochrane consists of over 37,000 contributors from more than 130 countries who make up over 50 subject review groups (Cochrane, n.d.). Together, this network of contributors has transformed the healthcare industry by publishing SLRs that gather and summarize high-quality evidence to aide researchers, physicians, caregivers, and others in making well-informed decisions that positively impact human health globally. Cochrane provides guidance on methodological standards which are available at methods.cochrane.org/mecir and are compliant with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) which provides the framework for SLR (Moher, 2009; PRISMA, n.d.). PRISMA, like the Cochrane Collaboration, was created by an international network of healthcare-based collaborators seeking to strengthen and streamline the methodological rigor and quality of SLRs. First published in 2009 as the PRISMA statement, it is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses (Moher, 2009; PRISMA, n.d.). Several extensions to the original document were published in 2015 and can all be found at www.prisma-sta tement.org/extensions/default.aspx. Although originally created for the review of randomized trials, PRISMA is broadly applicable to myriad types of research. The actual SLR then proceeds systematically through various sequential steps, including question formulation, team formulation, identification of search domains and publication sources, systematic search, systematic critical analysis, systematic interpretation, and systematic reporting. Table 1 outlines key steps in the SLR process. 4 Health Environments Research & Design Journal XX(X) Table 1. Key Steps in Conducting an SLR. Step Description 1 2 3 4 5 6 7 8 9 10 Familiarize yourself with the PRISMA framework. Develop your study question. It should be clear and unambiguous. Develop your study team. SLRs work best when the team has at least three members, where disagreement between two members is resolved by a third member. Make sure that each member has the necessary qualification and experience to conduct your specific SLR. Identify the concepts to be included in your search. For SLRs focused on interventional and/or comparative effectiveness studies, complete the PICO search model. For other study questions, in general, one should articulate at least three concepts of interest: (1) population(s) of interest, (2) setting(s) of interest, and (3) issue(s) of interest. Define each term in the study question and the identified concepts of interest. Do not assume that any term is unequivocally understood either by the team members or by the readers. Select databases relevant to your topical area to conduct your search (at least one, typically no more than five). Typical healthcare-based databases are listed on the right. You may also include other databases as well, depending on your topic, such as ERIC, JSTOR, etc. It is a good idea to enlist the expertise of a reference librarian at this point. Guided by the terms identified and defined in Step 5, now finalize your plan for the systematic search. The first substep is to develop a series of key words to be used for the search. Next, decide the inclusion and exclusion criteria. All team members must understand these criteria. Terms in the inclusion/exclusion criteria must also be defined. Define the limits you will use for your systematic search. Decide whether or not you will include gray literature (non-peerreviewed publications, dissertations, thesis, conference proceedings, etc.) and whether or not you will incorporate reference list searching (snowballing) or hand searches. See Appendix for example. Next, go into each database individually to tailor your key words and search terms specifically to each search engine. Each database will have its own system for terms. Some use MeSH and truncated terms and some do not. MeSH are Medical Subject Headings and is a vocabulary thesaurus managed by the National Library of Medicine and are tied to key words embedded within published manuscripts. When used, they allow the researcher to search very specifically. This step has an ebb and flow to it as you develop your search terms and key words. Too specific and your search will yield nothing, too broad and your search will yield will be unmanageable. Use the key word generators within the database to create a formula of your search terms, using “or,” “and” as appropriate between terms. Document in a table for transparency and replicability. See Appendix for example. Title review: Now conduct your search for all databases. Your initial review can be done using titles within the actual database to screen out irrelevant articles. Be generous in your inclusion at this point, making sure to keep your yield manageable based on your team, time, and resources. Export the article yield into a reference citation manager and then from the citation manager into an excel spreadsheet as a .csv file. This will become your journal citation report (JCR) which is a vital tool for the screening process and overall search documentation. Helpful Hints If definitions already exist in published literature, one could adopt those. PubMed, CINAHL, PsycINFO, MEDLINE, Web of Science Limits may include Specific date ranges, human subjects, participant descriptive, publication type, language, etc. The overall yield amount considered manageable will depend on the size of your analytic team. Aim for initial yields no more than 500 per database as a rule of thumb. Reference citation managers: Zotero, EndNote, RefWorks, etc. (continued) Pati and Lorusso 5 Table 1. (continued) Step Description Helpful Hints 11 You can use viewing panes within Excel to help condense the cells to only what is relevant during this stage (abstract and screening questions). 12 13 13 14 15 Within the JCR, create a tab for each database, including snowballing, hand searching, and gray literature on their own tabs, if included in the search. Organize headers for relevant information including author name, abstract, publication journal, DOI, volume, year of publication, etc. Include columns at the right pertaining to your exclusion criteria that can be answered by yes or no. Remove any duplicates by striking through and hiding the cells. This way if you make a mistake, the information is still there and you can reincorporate it into your JCR if needed. In case you are not sure whether to include or exclude an article based on the title, either retain it or consult the other team members. Abstract review: Up to this point you have screened only titles, so now you will begin to read through the abstracts of each article and screen them against the exclusion criteria. Instead of inserting “yes” or “no,” it is helpful to create separate columns for the two and insert a number “1” (for yes or for no, as applicable) so that you can run a sum total at the bottom of each spreadsheet to keep track of how many duplicates were removed, how many were excluded and how many were included per database. Once you have completed this for each separate database, create a tab for all included articles. Copy and paste all the included articles into this sheet and sort by title to quickly find and remove any remaining duplicates. This information will be important data for the PRIMSA flowchart. Abstract phase screening quality check: To strengthen validity and reliability for your search results, work with at least one other researcher to conduct an inter-rater reliability assessment. For instance, you can give the second reviewer 20% of the total number of abstracts you reviewed, randomly chosen, and leave your exclusion fields blank so that the second reviewer can run a parallel screening process on the subset of abstracts. Compare your findings and aim for at least 90% agreement. Discuss any areas of disagreement with the third team member reviewer until you reach an inclusion/exclusion agreement. On the last tab in the Excel spreadsheet, organize your process totals in the JCR and update your PRISMA flowchart with the information. Keep the JCR as a tool and record of your process. Critical review of full articles: Once you have the list of final included articles, conduct in-depth critical review of each article. Extract all relevant information from each article and insert in the appropriate place in the Excel document. As part of the critical review, rate the level of quality of evidence for each article using the framework you have adopted for your study. Critical review assessment quality check: To strengthen validity and reliability for your critical review findings, work with at least one other researcher to conduct an interrater reliability assessment. For instance, you can give the secondary reviewer 20% of the total number of articles you critically reviewed, randomly chosen, and leave your assessment fields blank, so that the second reviewer can run a parallel critical evaluation process on the subset of articles. Compare your findings and aim for complete agreement. Closer the agreement to 100%, more robust is the critical review process. A level of at least 90% agreement may be acceptable. Discuss any areas of disagreement with the third team member. Note, once you have deleted articles from inclusion in the JCR you can remove them from your reference citation manager for consistency. You may review the research question, exclusion criteria, and definitions again to ensure uniformity of understanding. PRISMA flowchart can be found at http://www.prismastatement.org/ You may review the research question, the quality of evidence scale, quality issues, and definitions again to ensure uniformity of understanding. (continued) 6 Health Environments Research & Design Journal XX(X) Table 1. (continued) Step Description 16 Helpful Hints You can also calculate Update your PRISMA flowchart with any changes from the inter-rater interrater reliability with reliability analysis. Now you will create a new, clean spreadsheet that will the level of quality of become your matrix of included studies. This spreadsheet will be simpler evidence to further than the JCR with headings that are tailored specifically to the purpose of strengthen validity and your overall SLR. These might include first author, year, level of evidence, reliability of your SLR. methods, participants, environmental intervention, outcomes, setting, assessments, results, etc. This will likely be published with your article and should be well organized. Now you are ready to compose your report, following the guidelines from the Cochrane Collaboration and the PRISMA checklist, and using Table 2. Note. SLRs ¼ systematic literature reviews; PRISMA ¼ Preferred Reporting Items for Systematic Reviews and Meta-Analysis. Contents within the 27 PRISMA checklist items include guidelines regarding key steps in the process and are broken into title, abstract, methods, results, discussion, and funding related to the SLR (Moher, 2009; PRISMA, n.d.). According to the PRISMA checklist, authors should first clearly identify the report as a systematic review, metaanalysis, or both. This is the only aspect of PRISMA not required by the Cochrane Collaboration but is a good rule to follow for healthcare design researchers who wish to make the nature of their work easily identifiable for publishers and readers. The following is a summary of the PRISMA checklist, which can be found and downloaded at http:// www.prisma-statement.org/ for use as a guide. Introduction. The two critical elements within the introduction are the author’s rationale and objectives. The rationale for the review should be written in a way that explains the topic in light of what is currently understood in the existing literature. The objectives should follow and clearly identify the question(s) to be addressed and how they relate to the various concepts of interest. In interventional studies, a typical articulation of concepts of interest is participants/interventions/comparisons/outcomes (PICO) and study design (Figure 1). The PICO process comes from the field of evidence-based medicine (EBM) to address a healthcare-based question. In SLRs, PICO can be used to frame the search terms and key words to the specific topic being studied (Centre for Evidence-Based Medicine, 2014; Centre for Reviews and Dissemination, 2009; Lorusso & Bosch, 2016). Investigators should identify or develop their specific list of concepts of interest, such as population/setting/issue, based on the study objective and report it in the introduction. Abstract. The abstract of the report should consist of a detailed summary including the following elements, as applicable to each report: background, objectives, data sources, study eligibility criteria, participants and interventions, study appraisal and synthesis methods, results, limitations, conclusions and implications of key findings, and inclusion of a systematic review registration number if applicable. The overall goal is to be clear and concise in the abstract so as to transparently communicate the contents of the paper to readers. Methods. The methods section should outline the overall process followed within the SLR. It is important to maintain clarity and transparency throughout the description of the process followed in the SLR in order to reduce bias and improve the translation of the overall findings. This section will include information on the protocol and registration, eligibility criteria, information sources, search strategy, study selection and data collection process, data items studied, risk of bias in individual studies, a summary of A Suggested Model for Systematically Composing an SLR Report Key Sections in Writing an SLR Following the PRISMA Checklist Pati and Lorusso 7 P Patient, Population or Problem e.g. Dementia I Intervention e.g. Multisensory Environments C Comparison of Intervention (if appropriate) NA O Outcome to Measure or Achieve e.g. Patient Behavior Figure 1. Explanation of participants/interventions/comparisons/outcomes (PICO) search model (Center for Evidence-Based Medicine). measures, synthesis of results, risk of bias across studies, and any additional analysis. This section provides an overview explanation preceding the results section which will provide information culminating from a deeper dive into many of the same topics for the studies included in the final review. Protocol and registration, if it exists and where it can be found, should be reported. Eligibility criteria should be provided of included studies such as study characteristics, time frame of studies, language, publication, and so on, and a rationale for why the author used these specific eligibility criteria. Information sources included in the review should be clearly described. These are typically the databases targeted for the review with an explanation of the dates used within the review and an explanation of any contact with authors for additional included studies. The search strategy should be clearly defined for one or more databases. This should be provided in great detail to allow for future updates or replication and should include any limits used by the author. The study selection process should be outlined regarding screening of findings, ratings of eligibility, and a breakdown of what articles were included in the review. This is supported by the PRISMA flowchart, which provides a clear, graphic representation of the process from included articles through to the final selection of included articles (Figure 2). A template is available at http://www.prisma-statement.org/PRISMAState ment/FlowDiagram.aspx The data collection process and precise method used by the authors to gather information from the literature should be clearly provided. This is often done through an excel document known as the journal citation report (JCR) that collates information extracted from reference managers in an organized way (author name, publication, publication date, volume, abstract, etc.). It is often used as a tool by the author or team writing the SLR to rate the found articles against the inclusion criteria and keeps a record of duplicates, excluded and included articles. Data items are listed within the methods section and include variables and all assumptions or simplifications made by the SLR author(s). Risk of bias in individual studies is provided and discussed in terms of how the risk was assessed for each article and how it will be used in the data synthesis of the SLR. A summary of measures should be outlined in terms of the primary measures used within the articles included in the SLR. This helps to clarify the overall analysis and helps readers understand how outcomes were achieved. A synthesis of results is explained by a description of the methods used by the SLR author(s) to review data and evaluate the level of evidence. Risk of bias across studies is discussed that may impact the overall findings. This is different from the risk of bias within individual studies because it discusses a broader range of risk including publication bias or limited reporting of data throughout studies. Additional analyses are important particularly in SLRs, which might include sensitivity or subgroup analyses, metaregression, and so on. It may also include any other additional analysis conducted by the SLR author or team relevant to the synthesis of findings. Results. The results section of the SLR is presented in descriptive, graphic, and detailed tabular formats. Combined, these representations should carefully describe the study selection, characteristics, risk of bias within and of individual studies, synthesis of results, risk of bias across studies, and any additional analysis of findings. Whereas the methods section provides an introduction, and outline of how these topics will be discussed, the 8 Health Environments Research & Design Journal XX(X) Figure 2. Example of Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram. results section provides great detail specifically on the articles included in the final review. Study selection, supported by the PRISMA flowchart, provides detailed information on the number of studies screened, assessed for eligibility, and excluded or included for review with an explanation for exclusion throughout the process. Study characteristics are provided for each included article along with information on the data that were extracted. This could include participant or site descriptions, intervention, method, time frames, and so on. This information is supported graphically in a matrix of the included studies. Risk of bias within studies is identified with outcome-level assessments. Results of individual studies are provided for all outcomes including information on the intervention and any group differences. This can be done in written format, separated into common themes or areas of affinity, and supported graphically by the SLR in a matrix of the included studies. A synthesis of results will provide information on meta-analysis of included articles and Pati and Lorusso assessed levels of evidence. Risk of bias across studies and additional analysis provides information on broad risk of bias across all included studies and detail any additional analysis conducted on included literature. Discussion. Lastly, the discussion section of the article will bring all the information together into a final summary of the evidence, listing limitations of the SLR and offering conclusions stemming from the review. In healthcare design SLRs, this section may also include a description of possible implications for practice that can be used by healthcare designers, researchers, and administrators in future applications. The summary of evidence will restate the main findings and discuss the overall quality level and magnitude of evidence and how each finding relates to key user groups within the healthcare industry. A discussion of limitations is an important component of the SLR to overcome bias. All limitations including any risk at any level of the review process should be identified. All reviews will have some form of risk of bias and so this is not a section that should be avoided or seen as a weakness to the overall report. To the contrary, an open and straightforward discussion of any risk of bias serves to increase the credibility of an SLR. Conclusions. The concluding section of the discussion will include the author(s) perspective on the findings and their role related to other reported findings and how they may impact future research or healthcare design overall. Any sources of funding should be reported including a description of any other support from sources supplying data and the subsequent role of funding sources related to the SLR. Table 2 provides a suggested step-by-step, generic model for composing an SLR report. It includes important questions that need to be responded to in each section of an SLR report. One may ignore questions that do not apply to one’s specific SLR study. Explanation of Key Elements Within the Review As discussed earlier, critical factors separating SLR from traditional reviews are clarity, validity, and auditability (Booth et al., 2016). These 9 elements are best achieved by following the Cochrane Collaboration’s guidelines and the PRISMA statement checklist closely. Adhering to these frameworks helps ensure a level of uniformity in the SLRs, supporting transparency, and reducing overall risk of bias. This uniformity incorporates key elements including an assessment of the evidence, the concepts of interest and PRISMA flow diagrams, a JCR, and a matrix of included studies. Both the quality level and magnitude of evidence need to be analyzed in an SLR. The quality level of evidence pertains to the quality of an individual study. The magnitude of evidence pertains to the volume of quality evidence available on a topic or question of interest. Referring back to our analogy of snapshots in the introduction, the quality level of evidence can be viewed as the quality of each photograph whereas the magnitude of evidence can be viewed as the number of good quality photos available for stitching together. The purpose of critically assessing the evidence in an SLR is important because it provides the reader with a description of its quality. This assessment is often tied directly to the inclusion/exclusion criteria when an author will state that only certain quality of evidence will be included. This may be done in an SLR of a topic where the author(s) goal is to report only the highest quality of evidence and there is enough available literature to be selective. Sometimes, however, this is not possible particularly in the healthcare design domain, currently, because the body of knowledge regarding a topic may not yet be so extensive as to allow it. Whatever the decision of the author or team conducting the SLR, it is important to rate the quality level of evidence. Examples can be found in EBM (Balshem et al., 2011) and also in healthcare design literature (Hamilton, 2011; Marquardt, Bueter, & Motzek, 2014; Pati, 2011; Stichler, 2010) and are commonly described in a simple table format outlining the descriptions of each quality level. Other key elements within the review are the concepts of interest (such as PICO) and PRISMA flowchart both of which relate to the overall search strategy of the SLR. It is important to note that when designing the specific search strategy for each SLR, it is incredibly valuable to enlist the help of a research librarian with expertise within 10 Health Environments Research & Design Journal XX(X) Table 2. A Step-by-Step Model for Composing an SLR Report. Main Sections Subtopics Abstract Introduction Method Team composition Search preparation Search process Critical review Results Description of final list Content of final list Content of each thematic area Key Questions to Respond to [as required by the targeted journal] What is the SLR study question? Background: What was already known on the topic/question of interest? What drove the need for conducting this specific SLR? What is the objective of this SLR? What is the significance of this SLR study? How many investigators were on the team? What were their qualifications, expertise, and levels of experience? What databases were targeted? What key words/phrases were used in each database search? What were the inclusion and exclusion criteria? How and when were the database searches conducted? How many phases of search were involved? Was a hand search conducted? How was it conducted? How was it determined whether to include or exclude a specific publication? How many steps were involved in arriving at the final list of publications? What was involved in each step? How was interrater reliability ensured during the abstract review phase? How was the critical review conducted? What information was extracted and recorded? How was the quality level of each study determined? What quality framework was used? How were qualitative, quantitative, and mixedmethod studies evaluated? What method was used to develop thematic areas of relevance? How were studies within each thematic area synthesized? Were any statistical analysis/meta-analysis conducted on quantitative findings? If not, why? How was interrater reliability ensured during the critical review phase? How many articles were found in each sequential search step? How many were retained in each step and how many were discarded? What was the final number of articles on which critical review was performed? How many studies were empirical? Of the empirical studies, how many were causal/experimental, how many were relational, and how many were descriptive? Of the empirical studies, how many were qualitative, how many were quantitative, and how many used mixed methods? In which countries were the studies conducted? How many studies were classified at each level on the quality of evidence scale? What thematic areas were identified? For each thematic area (separately), what are the key descriptions of the contents (respond to the same questions in the “content of final list,” but for each thematic area)? What were the key findings in each thematic area? What are some patterns observable? Are there any converging/diverging findings? What is the overall confidence (magnitude of evidence) within each thematic area? Within each thematic area, how many studies and what proportion of studies fall under each level of the quality of evidence scale? (continued) Pati and Lorusso 11 Table 2. (continued) Main Sections Subtopics Key Questions to Respond to Discussions Summarization What new information was generated from this systematic review? What is the general level (quality and magnitude) of evidence in various thematic areas? Which of the thematic areas enjoy higher quality and magnitude of evidence? Which areas currently lack quality and/or magnitude of evidence? What are the limitations of this SLR? What topics/questions should be examine in future studies? What are the key take-away messages of this SLR for your target readers? [include complete list of articles reviewed and/or cited in the SLR, denote articles included in the SLR matrix with an asterisk*] [as appropriate] Limitations Future studies Conclusions References Appendix the specific topic to be reviewed. Research librarians with a focus on healthcare, for example, have extensive knowledge of the broad range of literature and are adept in assisting in the development of the PICO model serving as a guide for the search terms and key words relating the patient, population or problem, intervention, and outcome to be evaluated in the SLR, if the SLR is focused on interventional studies. Once established, the PICO search model for the SLR is often illustrated with a table and it is helpful to document the exact key words and terms used in a separate appendix for future revision or replication of the search. In case of SLRs not restricted to interventional studies, librarians can also assist in developing the right search terms for population, settings, and issues of interest. The PRISMA flow diagram is also used to document the search strategy. A template is available from the PRISMA statement’s website that can be edited and inserted directly into a Word document. The flowchart is simple and easy to follow and provides a consistent way to communicate the results of the search process from the initial findings, inclusion of gray or snowballed literature, exclusions, and final inclusion counts for the SLR. One of the most important elements within the SLR is the matrix of included studies. The matrix is a tabular representation of the results section of the paper made in a spreadsheet document that summarizes the details of each included study and the overall findings of the SLR. It is important to note that the articles included in the final review are included in the reference list of the paper and are marked with an asterisk to clearly identify them for the reader. The matrix will be tailored specifically to each SLR based on the unique objectives of the paper. Typical components of the matrix include the last name of the first author and year of publication, assigned quality level of evidence, methods, participants’ description, intervention summary, list of outcomes, description of setting and assessments, and a brief recap of the results of each study. The intention of the matrix is to serve as a tool providing an easily digestible format for readers to understand the purpose and outcomes of the SLR. Important Issues in Conducting SLRs There are several issues that warrant close attention while conducting and reporting an SLR. The key question one should ask while conducting an SLR is whether the intended audience will consider the findings credible (irrespective of the actual nature of SLR findings). If trust is not generated among the readers, then findings may not be perceived as credible, which will be counterproductive to the objectives of the SLR study. The following subsections outline some of the issues that investigators need to accord special attention to in order to generate trust and confidence. The key question one should ask while conducting an SLR is whether the intended audience will consider the findings credible. 12 Systematicity One distinguishing feature of SLRs that separates it from most other types of reviews is systematicity. Being systematic includes a systematic process of searching for literature, filtering results of literature searches, critical analysis of the studies, evaluating quality of identified studies, evaluating the level of evidence, interpretive analysis of findings across multiple studies, and reporting for the intended audience. Documentation One simple strategy for generating trust and confidence is documentation. Each step and results should be documented and reported in a clear, unambiguous manner. Study Question The most critical aspect of any research study is the study question. It should be noted that SLR, owing to its interpretive component, could be deemed as a research study. The study questions impact every subsequent step in a study. It should be clear and precise. Definitions One way of achieving clarity and precision in the study question is developing clear, unambiguous, and documented definitions of all terms associated with the study. Definitions are important in bringing all the investigators and the audience on the same page. Table 3 provides the definitions of key terms used in this article. Search Strategy The objective of a typical SLR is to include all literature associated with a study question, including literature in peer-reviewed publications and other sources, such as dissertations. Numerous non-peer-reviewed sources, also referred to as “gray literature” should also be brought into scrutiny, such as white papers and industry/trade articles. Search may (or should) include manual review of the reference section of key articles. In some fields, quantitative studies that do not result in a statistically significant findings face Health Environments Research & Design Journal XX(X) greater difficulties in getting published in peerreviewed journals (also known as “publication bias”), whereas from a theoretical perspective and in numerous real-life situations knowledge regarding nonsignificant findings could also be valuable. As a result, it is essential to search and retrieve associated literature from all types of sources. How a search is conducted could lead to bias, whether intentional or otherwise. The appropriate way to develop trust in the audience of an SLR is to demonstrate that a broad, complete, and thorough search was conducted, wherein the probability of missing any important work related to the study question is minimized. Inclusion and Exclusion Criteria Another source of potential bias and ambiguity is in the way literature found in the searchers are included or excluded from further analysis. Clear documentation of criteria used for inclusion or exclusion of a study will result in a higher trust and credibility in the SLR findings. Critical Review Producing a good SLR is dependent on critical review of the quality of each study. There are two approaches to assessing study quality. One approach focuses purely on the research design, wherein studies with higher internal validity are considered higher in quality. Quantitative studies have typically been considered higher in quality than qualitative studies since assessing internal validity is easier in quantitative study design. For instance, an experimental study design would be considered higher in quality than a relational or descriptive study or a qualitative study design. However, research design is not the only factor (and should not be) to assess in determining study quality. Perhaps, when time is limited, a classification based just on research design is justified. It should never be assumed that a good research design necessarily results in a good quality study. Ideally, each study should be assessed deeper on all quality issues—construct, internal, external, and conclusion validity as well as reliability. Issues such as definitions, study setting, tools and measures, sampling strategy, sample characteristics, Pati and Lorusso 13 Table 3. Definition of Terms Found in Article. Term Definition Areas of affinity Auditability Bias Clarity Cochrane Collaboration Conformity Credibility Cross-sectional Study Databases Dependability Dissertations Effect size Eligibility criteria Gray literature Industry/trade articles Instruments Intervention Investigator bias Journal citation report Limits Longitudinal study Magnitude MeSH Meta-analysis Mixed-methods Studies Peer-reviewed publications Population PRISMA Protocol Publication bias Qualitative Quality level Quantitative Randomized controlled trial Reference manager Replicability Research question Sample characteristics Sample size Sampling strategy Setting Snowballed literature Statistically significant Subgroup analyses Synthesis Themes Theoretical framework Topic of interest Transferability Common themes found in the SLR literature. Ability for the SLR to be externally reviewed and replicated. Prejudice, predisposition, or partiality to a topic or viewpoint. Related to the level of transparency, openly displaying the process. Global healthcare research organization producing SLRs. Consistency in design a method or process. Reliability or trustworthiness of a method or process. Study that analyzes data at a specific point in time. Online archives (e.g., PubMed, CINAHL, MEDLINE, etc.) Related to the soundness of the findings. A five-chapter report written as a requirement for a PhD degree. Statistical measure of strength of relationship between variables. Criteria or boundaries that must be shared by all articles included. Unpublished or published noncommercially, not peer reviewed. Articles published in a commercial magazine, targeting industry. Measurement tools to gather data (e.g., survey, observations, etc.) That which is being tested in an experimental study. Occurs when investigators influence results due to predisposition. Document used in an SLR to organize and analyze literature found. Boundaries within which search is conducted (e.g., date range, species, etc.) Data is gathered from the same subjects over a long period of time. Relating to the size or extent of existing, published literature. Medical Subject Headings, thesaurus terms embedded in literature. Statistical approach combining results from multiple quantitative studies. Studies that employ both qualitative and quantitative methods. Evaluation prior to publication by others within the same field. Describes the people at the focus of a study. Preferred Reporting Items for Systematic Review and Meta-Analyses A document outlining rationale and detailed procedure of a study. Occurs when a publication has a known partiality to a topic or findings. Research involving nonnumeric data. Relating to the value and perceived excellence of a study. Research involving numeric data. Studies where subjects are assigned randomly to receive one of several interventions where one is a base-line of comparison. Online software used to organize and manage literature findings. The ability for a study to be repeated with similar results. The question that a researcher attempts to find an answer. The defining features of a study sample (e.g., gender, race, age, etc.) The number of participants in a study. The researcher’s approach to finding participants for a study. The location of a particular study (e.g., hospital, nursing home, etc.) Literature found through the reference list of included articles. A result or finding between variables not attributed to chance. Exploration of how people respond differently to an intervention. Combination of findings. Similarities or commonalities found between studies. Interrelated concepts. Subject matter that a researcher seeks to find deeper understanding on. How easily a finding can be applied to other settings. (continued) 14 Health Environments Research & Design Journal XX(X) Table 3. (continued) Term Definition Transparency Truncated terms Validity Relates to the clarity and openness in the design of a study. Search technique that broadens to various spellings with an asterisk Closeness to the truth. sample size, analytical procedures, measures adopted to remove bias, among many others, should be evaluated in order to accurately, systematically, and transparently gauge the quality of a study. One fundamental requirement for conducting critical quality review of a study at this depth is to develop a team with appropriate qualifications to enable it. One should carefully select the team of a proposed SLR to achieve a robust level of critical review. It should also be noted that qualitative methods also have quality issues similar to those in quantitative methods, such as credibility, transferability, dependability, and conformity (Healey, 2014). In case of qualitative or mixedmethod studies, team members need to have sufficient knowledge and experience to evaluate the deeper quality issues in both types of methods. Reporting Reporting findings from SLRs should not be a simple additive task, where findings from various studies are reported as a summative narrative. One of the strengths of SLRs is in the interpretive analysis of findings from multiple studies. This requires in-depth domain knowledge and expertise of the investigating team. A critical issue in reporting is investigator bias. As mentioned previously, any hint of investigator bias could lead to an erosion of trust and credibility of SLR findings. Since every individual has some degree of bias, it is necessary to take precautionary measures to eliminate any. Having a team (instead of one investigator) is one of the strategies to eliminate or reduce bias in the literature search and selection phases. The ideal solution is to include team members who have little bias on a topical area. In the reporting phase, if one’s strong bias is known, it may be advisable to exclude that individual from the interpretation and reporting phases, or subject the interpretation and reporting to scrutiny by multiple team members or external experts. Conclusion With a growing body of evidence pertaining to the influence of physical design on clinical, behavioral, operational, and business processes outcomes, there will be a growing demand for SLRs on healthcare design topics. Key factors supporting SLRs are clarity, validity, and auditability (Booth et al., 2016). It is important to note that although bias is significantly reduced in SLRs, its existence is not eradicated. Also, despite their merits, SLRs are the most demanding types of literature reviews to conduct and involve a structured process for conducting and reporting. Although the process is significantly more involved than a traditional literature review, there is a great deal of support available through organizations like the Cochrane Collaborative and the PRISMA statement. As with any literature review, SLRs are undoubtedly strengthened through the additional support of a research librarian with expertise in your field of study. Overall, adhering to the requirements described in this article, and/or other similar SLR guides available in medical literature, will ensure a credible, high-quality SLR and contribute vital information to the ever-growing healthcare design knowledge base. Implications for Practice SLRs provide more comprehensive and holistic evidences than single research studies. Use SLRs to inform decision making when available. Use SLRs published in peer-reviewed journals only. SLRs not subjected to peer-review may or may not have investigators’ biases. Always check that your definition/understanding of terms used in practice are identical to those used in the SLR. Frequently, differences in the way terms are defined lead to erroneous conclusions, with potentially undesirable downstream outcomes. Pati and Lorusso 15 Appendix Table A1. Search Strategy Template. Research Question(s) Databases, Resources, and Limits Core Databases þReference List Searching þHand Searching Journals Table B1. Primary Literature Searching Template. Population Intervention Outcome Concept: Concept Concept Mesh/ subheadings Text words Note. Database: (Do this for each of your databases and gray literature). PubMed Search Strategies POPULATION (Example below, do the same for intervention/outcome as well) (“Dementia/drug therapy”[Mesh] OR “Dementia/ rehabilitation”[Mesh] INTERVENTION OUTCOME Yield: Duplicates Removed: Total Added Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) received no financial support for the research, authorship, and/or publication of this article. References Balshem, H., Helfand, M., Schunemann, H., Oxman, A., Kunz, R., Brozek, J. . . . Guyatt, G. (2011). GRADE Gray Literature -Clinical Trial Registries -National Clearinghouse -Google Scholar -Dissertation Sources -Conference Abstracts Limits 1. Language: 2. Years: 3. Publication Types: 4. Species: guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology, 64, 401–406. Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. Centre for Evidence-Based Medicine. (2014, June 10). Asking focused questions. Retrieved February 22, 2016, from http://www.cebm.net/asking-focusedquestions/ Centre for Reviews and Dissemination. (Ed.). (2009). CRD’s guidance for undertaking reviews in healthcare (3rd ed.). York, England: York Publishing Services. Cochrane. (n.d.). Cochrane Collaboration. Retrieved October 10, 2017, from www.cochrane.org/about-us Dunn, P. M. (1997). James Lind (1716-94) of Edinburgh and the treatment of scurvy. Archives of Disease in Childhood—Fetal and Neonatal Edition, 76, F64–F65. Retrieved from https://doi.org/10.1136/ fn.76.1.F64 Glass, G. V. (1976). Primary, secondary, and metaanalysis of research. Educational Researcher, 5, 3–8. Retrieved from https://doi.org/10.3102/ 0013189X005010003 Grant, M., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26, 91–108. Greenhalgh, T. (2014). How to read a paper: The basics of evidence-based medicine (5th ed.). Chichester, England: John Wiley. Hamilton, D. K. (2011). What constitutes best practice in healthcare design? Health Environments Research & Design Journal, 4, 121–126. Retrieved from https://doi.org/10.1177/193758671100400210 Healey, J. (2014). Statistics: A tool for social research (10th ed.). Belmont, CA: Wadsworth. 16 Lorusso, L., & Bosch, S. (2016). Impact of multisensory environments on behavior for people with dementia: A systematic literature review. The Gerontologist. Retrieved from https://doi.org/10.1093/ geront/gnw168 Marquardt, G., Bueter, K., & Motzek, T. (2014). Impact of the design of the built environment on people with dementia: An evidence-based review. Health Environments Research & Design Journal, 8, 127–157. Moher, D. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264. Retrieved from https://doi.org/10.7326/0003-4819151-4-200908180-00135 Pati, D. (2011). A framework for evaluating evidence in evidence-based design. Health Environments Research & Design Journal, 4, 50–71. PRISMA. (n.d.). PRISMA transparent reporting of systematic reviews and meta-analyses. Retrieved October 10, 2017, from www.prisma-statement.org Stichler, J. (2010). Weighing the evidence. Health Environments Research & Design Journal, 3, 3–7. Ulrich, R., Zimring, C., Quan, X., Joseph, A., & Choudhary, R. (2004). The role of the physical environment in the hospital of the 21st century: A oncein-a-lifetime opportunity. Concord, CA: The Center for Health Design. Health Environments Research & Design Journal XX(X) Author Biographies Debajyoti Pati is currently a professor of environmental design and Rockwell Endowment Chair at Texas Tech University. He has over 28 years of experience in research, practice, and teaching in the United States, Canada, and India. He was twice voted among the 25 most influential people in healthcare design and has received over ten awards for research excellence. His areas of interest include health, healthcare, healthy environments, sustainability, public architecture, judicial facilities, performance-based design, organizational learning, and developing indicators of facility performance. He is on the editorial board of HERD and on the board of directors of the NIBS-AIA BRIK program. Lesa Lorusso is a licensed interior designer, with over 15 years of experience in both academic and professional practice. Her educational background includes a BS in interior design, MBA and MS in architecture. She practices in Florida and is currently a doctoral candidate and research assistant within the College of Design, Construction and Planning at the University of Florida and researcher with the Geriatric Research Education and Clinical Center (GRECC) at the Malcolm Randall Veteran’s Administration Medical Center. The focus of her doctoral research is a collaboration between UF and the VA and investigates the impact of multisensory environments on behavior for Veterans with dementia.