Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Scoping studies: advancing the methodology

Scoping studies: advancing the methodology Background: Scoping studies are an increasingly popular approach to reviewing health research evidence. In 2005, Arksey and O’Malley published the first methodological framework for conducting scoping studies. While this framework provides an excellent foundation for scoping study methodology, further clarifying and enhancing this framework will help support the consistency with which authors undertake and report scoping studies and may encourage researchers and clinicians to engage in this process. Discussion: We build upon our experiences conducting three scoping studies using the Arksey and O’Malley methodology to propose recommendations that clarify and enhance each stage of the framework. Recommendations include: clarifying and linking the purpose and research question (stage one); balancing feasibility with breadth and comprehensiveness of the scoping process (stage two); using an iterative team approach to selecting studies (stage three) and extracting data (stage four); incorporating a numerical summary and qualitative thematic analysis, reporting results, and considering the implications of study findings to policy, practice, or research (stage five); and incorporating consultation with stakeholders as a required knowledge translation component of scoping study methodology (stage six). Lastly, we propose additional considerations for scoping study methodology in order to support the advancement, application and relevance of scoping studies in health research. Summary: Specific recommendations to clarify and enhance this methodology are outlined for each stage of the Arksey and O’Malley framework. Continued debate and development about scoping study methodology will help to maximize the usefulness and rigor of scoping study findings within healthcare research and practice. Background summarize and disseminate research findings, or identify Scoping studies (or scoping reviews) represent an gaps in the existing literature [6]. As such, researchers increasingly popular approach to reviewing health can use scoping studies to clarify a complex concept research evidence [1]. However, no universal scoping and refine subsequent research inquiries [1]. Scoping study definition or purpose exists (Table 1) [1,2]. Defini- studies may be particularly relevant to disciplines with tions commonly refer to ‘mapping,’ a process of sum- emerging evidence, such as rehabilitation science, in marizing a range of evidence in order to convey the which the paucity of randomized controlled trials makes breadth and depth of a field. Scoping studies differ from it difficult for researchers to undertake systematic systematic reviews because authors do not typically reviews. In these situations, scoping studies are ideal assess the quality of included studies [3-5]. Scoping stu- because researchers can incorporate a range of study dies also differ from narrative or literature reviews in designs in both published and grey literature, address that the scoping process requires analytical reinterpreta- questions beyond those related to intervention effective- tion of the literature [1]. ness, and generate findings that can complement the Researchers can undertake a scoping study to examine findings of clinical trials. the extent, range, and nature of research activity, deter- In an effort to provide guidance to authors undertaking mine the value of undertaking a full systematic review, scoping studies, Arksey and O’Malley [6] developed a six- stage methodological framework: identifying the research question, searching for relevant studies, selecting studies, * Correspondence: [email protected] 1 charting the data, collating, summarizing, and reporting School of Rehabilitation Science, McMaster University, 1400 Main Street West, Room 403, Hamilton, Ontario, Canada the results, and consulting with stakeholders to inform or Full list of author information is available at the end of the article © 2010 Levac et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Levac et al. Implementation Science 2010, 5:69 Page 2 of 9 http://www.implementationscience.com/content/5/1/69 Table 1 Definitions and purposes of scoping studies Authors Definition Purpose(s) Ehrich et al. (2002) None provided. ’The purpose of a scoping exercise is both to map a wide range of literature, and to envisage where gaps and innovative approaches may lie"’ [[11] p. 28]. Arksey and O’Malley ’Aim to map rapidly the key concepts underpinning a 1. To examine the extent, range, and nature of research (2005) research area and the main sources and types of evidence activity. available’ [[14] p. 194], as cited in [6] 2. To determine the value for undertaking a full systematic review. 3. To summarize and disseminate research findings. 4. To identify research gaps in the existing literature. [[6] p. 21] Anderson et al. (2008) ’Scoping studies are concerned with contextualizing 1. Literature mapping: ‘is a map of the relevant literature. knowledge in terms of identifying the current state of These vary in scope from general accounts of the literature understanding; identifying the sorts of things we know and to studies that are just short of systematic reviews. do not know; and then setting this within policy and Literature scoping studies often also involve the syntheses practice contexts’ [2] of findings from different types of study.’ 2. Conceptual mapping: ‘a scoping study designed to establish how a particular term is used in what literature, by whom and for what purpose.’ 3. Policy mapping: ‘a scoping study designed to identify the main documents and statements from government agencies and professional bodies that have a bearing on the nature of practice in that area.’’ 4. Stakeholder consultation: ‘Do[es] not constitute scoping studies in their own right, but they do have an important part to play in scoping studies concerned with the identification of research priorities, in helping to target research questions, and in validating the outcomes of scoping studies through peer-review’ [2]. Grant et al. (2009) ’Preliminary assessment of potential size and scope of ’Aims to identify the nature and extent of research research literature.’ [[4] p.95] evidence (usually including ongoing research’ [[4] p.95]. Davis et al. (2009) ’Scoping involves the synthesis and analysis of a wide ’We propose that a common synthesising construct range of research and non-research material to provide emerges to explain the purpose of scoping, namely that of greater conceptual clarity about a specific topic or field of ‘reconnaissance’. It is generally synonymous with a evidence’ [[1] p.1386]. preliminarily investigation in which information is systematically gathered and examined in order to establish strengths and weakness and guide in which ever context, future decision-making’ [[1] p. 1396]. National Institute for None provided. 1.’Clarification of working definitions and conceptual Health Research (NIHR) boundaries of a topic area, developed in the form of Service Delivery and systematic overview (narrative review) of the literature but Organisation specifically excluding a systematic review, to determine a Research and frame of reference; Development 2. Outline what is already known and identify gaps in Programme (SDO) existing research, and; 4. Conceptual analysis may include the ‘mapping’ of existing empirical evidence to describe and interpret issues that will inform further research and development opportunities.’ [[1] p. 1387] Canadian Institutes of ’Scoping reviews are exploratory projects that systematically None provided. Health Research map the literature available on a topic, identifying the key http://www.cihr-irsc.ca concepts, theories, sources of evidence, and gaps in the research. They are often preliminary to full syntheses, undertaken when feasibility is a concern – either because the potentially relevant literature is thought to be especially vast and diverse (varying by method, theoretical orientation or discipline) or there is suspicion that not enough literature exists. These entail the systematic selection, collection and summarization of existing knowledge in a broad thematic area for the purpose of identifying where there is sufficient evidence to conduct a full synthesis or where insufficient evidence exists and further primary research is necessary.’ [15] Levac et al. Implementation Science 2010, 5:69 Page 3 of 9 http://www.implementationscience.com/content/5/1/69 validate study findings (Table 2). While this framework on scoping study methodology. We searched CINAHL, provided an excellent methodological foundation, pub- MEDLINE, PubMed, ERIC, PsycInfo, and Web of lished scoping studies continue to lack sufficient metho- Science databases using the search terms ‘scoping,’ dological description or detail about the data analysis ‘scoping study,’‘scoping review,’ and ‘scoping methodol- process, making it challenging for readers to understand ogy’ for papers published in English between January how study findings were determined [1]. Arksey and 1990 and May 2010. Reference lists of pertinent papers O’Malley [6] encouraged other authors to refine their fra- were also searched. This search yielded seven citations mework in order to enhance the methodology. that reflected on scoping study methodology, which In this paper, we apply our experiences using the Arksey were reviewed by one author (DL). After independently and O’Malley framework to build on the existing metho- considering our own experiences utilizing the Arskey dological framework. Specifically, we propose recommen- and O’Malley [6] framework, we met on seven occasions dations for each stage of the framework, followed by to discuss the challenges and develop recommendations considerations for the advancement, application, and rele- for each stage of the methodological framework. vance of scoping studies in health research. Continual refinement of the framework stages may provide greater Recommendations to enhance scoping study clarity about scoping study methodology, encourage methodology researchers and clinicians to engage in this process, and We outline the challenges and recommendations asso- help to enhance the methodological rigor with which ciated with each stage of the methodological framework authors undertake and report scoping studies [1]. (Table 3). Discussion Framework stage one: Identifying the research question We each completed a scoping study in separate areas of Scoping study research questions are broad in nature as rehabilitation using the Arksey and O’Malley framework the focus is on summarizing breadth of evidence. Arksey [6]. Goals of these studies included: identifying research and O’Malley [6] acknowledge the need to maintain a priorities within HIV and rehabilitation [7], applying broad scope to research questions, however we found motor learning strategies within pediatric physical and our research questions lacked the direction, clarity, and occupational therapy intervention approaches [8], and focus needed to inform subsequent stages of the exploring the use of theory within studies of knowledge research process, such as identifying studies and making translation [9]. The amount of literature reviewed in our decisions about study inclusion. To clarify this stage, we studies ranged from 31 (DL) to 146 (KO) publications. recommend that researchers combine a broad research Upon discovering that we had similar challenges imple- question with a clearly articulated scope of inquiry. This menting the scoping study methodology, we decided to includes defining the concept, target population, and use our experiences to further develop the existing health outcomes of interest to clarify the focus of the framework. We conducted an informal literature search scoping study and establish an effective search strategy. Table 2 Overview of the Arksey and O’Malley methodological framework for conducting a scoping study Arksey and O’Malley Framework Description Stage 1: Identifying the research question Identifying the research question provides the roadmap for subsequent stages. Relevant aspects of the question must be clearly defined as they have ramifications for search strategies. Research questions are broad in nature as they seek to provide breadth of coverage. 2: Identifying relevant studies This stage involves identifying the relevant studies and developing a decision plan for where to search, which terms to use, which sources are to be searched, time span, and language. Comprehensiveness and breadth is important in the search. Sources include electronic databases, reference lists, hand searching of key journals, and organizations and conferences. Breadth is important; however, practicalities of the search are as well. Time, budget and personnel resources are potential limiting factors and decisions need to be made upfront about how these will impact the search. 3: Study selection Study selection involves post hoc inclusion and exclusion criteria. These criteria are based on the specifics of the research question and on new familiarity with the subject matter through reading the studies. 4: Charting the data A data-charting form is developed and used to extract data from each study. A ‘narrative review’ or ‘descriptive analytical’ method is used to extract contextual or process oriented information from each study. 5: Collating, summarizing, and An analytic framework or thematic construction is used to provide an overview of the breadth of the literature reporting results but not a synthesis. A numerical analysis of the extent and nature of studies using tables and charts is presented. A thematic analysis is then presented. Clarity and consistency are required when reporting results. 6: Consultation (optional) Provides opportunities for consumer and stakeholder involvement to suggest additional references and provide insights beyond those in the literature. Levac et al. Implementation Science 2010, 5:69 Page 4 of 9 http://www.implementationscience.com/content/5/1/69 Table 3 Summary of challenges and recommendations for scoping studies Framework Stage Challenges Recommendations for clarification or additional steps #1 Identifying the 1. Scoping study questions are broad. 1. Clearly articulate the research question that will guide the research question 2. Establishing scoping study purpose is not associated scope of inquiry. Consider the concept, target population, and with a framework stage. health outcomes of interest to clarify the focus of the scoping 3. The four purposes of scoping studies lack clarity. study and establish an effective search strategy. 2. Mutually consider the purpose of the scoping study with the research question. Envision the intended outcome (e.g., framework, list of recommendations) to help determine the purpose of the study. 3. Consider rationale for conducting the scoping study to help clarify the purpose. #2 Identifying 1. Balancing breadth and comprehensiveness of the 1a. Research question and purpose should guide decision- relevant studies scoping study with feasibility of resources can be making around the scope of the study. challenging. 1b. Assemble a suitable team with content and methodological expertise that will ensure successful completion of the study. 1c. When limiting scope is unavoidable, justify decisions and acknowledge the potential limitations to the study. #3 Study selection 1. The linearity of this stage is misleading. 1. This stage should be considered an iterative process 2. The process of decision making for study selection is involving searching the literature, refining the search strategy, unclear. and reviewing articles for study inclusion. 2a. At the beginning of the process, the team should meet to discuss decisions surrounding study inclusion and exclusion. At least two reviewers should independently review abstracts for inclusion. 2b. Reviewers should meet at the beginning, midpoint and final stages of the abstract review process to discuss challenges and uncertainties related to study selection and to go back and refine the search strategy if needed. 2c. Two researchers should independently review full articles for inclusion. 2d. When disagreements on study inclusion occur, a third reviewer can determine final inclusion. #4 Charting the data 1. The nature and extent of data to extract from included 1a. The research team should collectively develop the data- studies is unclear. charting form and determine which variables to extract in 2. The ‘descriptive analytical method’ of charting data is order to answer the research question. poorly defined. 1b. Charting should be considered an iterative process in which researchers continually extract data and update the data- charting form. 1c. Two authors should independently extract data from the first five to ten included studies using the data-charting form and meet to determine whether their approach to data extraction is consistent with the research question and purpose. 2. Process-oriented data may require extra planning for analysis. A qualitative content analysis approach is suggested. #5 Collating, 1. Little detail provided and multiple steps are summarized Researchers should break this stage into three distinct steps: summarizing, and as one framework stage. 1a. Analysis (including descriptive numerical summary analysis reporting the results and qualitative thematic analysis); 1b. Reporting the results and producing the outcome that refers to the overall purpose or research question; 1c. Consider the meaning of the findings as they relate to the overall study purpose; discuss implications for future research, practice and policy. #6 Consultation 1. This stage is optional. 1. Consultation should be an essential component of scoping 2. Lack of clarity exists about when, how and why to study methodology. consult with stakeholders and how to integrate the 2a. Clearly establish a purpose for the consultation. information with study findings. 2b. Preliminary findings can be used as a foundation to inform the consultation. 2c. Clearly articulate the type of stakeholders to consult and how data will be collected, analyzed, reported and integrated within the overall study outcome. 2d. Incorporate opportunities for knowledge transfer and exchange with stakeholders in the field. Levac et al. Implementation Science 2010, 5:69 Page 5 of 9 http://www.implementationscience.com/content/5/1/69 For example, in one author’s (KO) scoping study, the conclusions about the nature and extent of those gaps if research question was broadly ‘what is known about the quality of the evidence is not assessed. The purpose HIV and rehabilitation?’ Defining the concept of ‘rehabi- ‘to identify the key research priorities in HIV and reha- litation’ was essential in order to establish a clear scope bilitation to advance policy and practice for people liv- to the study, guide the search strategy, and establish ing with HIV in Canada’ does not explicitly align with parameters around study selection in subsequent stages one of the four Arskey and O’Malley purposes [7]. How- of the process [7]. ever, it appears authors inherently first summarized the Although Arskey and O’Malley [6] outline four main extent, range, and nature of research (purpose one) and purposes for undertaking a scoping study, they do not identified gaps in the existing literature (purpose four) articulate that purpose be specified within a specific fra- in order to subsequently identify the key research priori- mework stage. We recommend researchers simulta- ties in HIV and rehabilitation (author purpose). This neously consider the purpose of the scoping study when suggests authors might have an overall study purpose articulating the research question. Linking a clear pur- with multiple objectives articulated by Arksey and pose for undertaking a scoping study to a well-defined O’Malley that are required in order to help achieve their research question at the first stage of the framework will overall purpose. help to provide a clear rationale for completing the study and facilitate decision making about study selec- Framework stage two: Identifying relevant studies tion and data extraction later in the methodological pro- A strength of scoping studies includes the breadth and cess. A helpful strategy may be to envision the content depth, or comprehensiveness, of evidence covered in a and format of the intended outcome that may assist given field [1]. However, practical issues related to researchers to clearly determine the purpose at the time, funding, and access to resources often require beginning of a study. In the abovementioned HIV study, researchers to consider the balance between feasibility, authors linked the broadly stated research question with breadth, and comprehensiveness. Brien et al. [5] amorespecificpurpose ‘to identify the key research reported that their search strategy yielded a vast priorities in HIV and rehabilitation to advance policy amount of literature, making it difficult to determine and practice for people living with HIV in Canada’ [7]. how in depth to carry out the information synthesis. The envisioned outcome was a thematic framework that Although Arksey and O’Malley [6] identify these con- represented strengths and opportunities in HIV rehabili- cerns and provide some suggestions to support these tation research, followed by a list of the key research decisions, we also struggled with the trade-off between priorities to pursue in future work. breadth and comprehensiveness and feasibility in our Finally, the purposes put forth by Arksey and O’Mal- scoping studies. As such, we recommend that research- ley [6] require more debate. We concur with Anderson ers ensure decisions surrounding feasibility do not et al. [2] and Davis et al. [1], who state that researchers compromise their ability to answer the research ques- may benefit from further clarification of the purposes tion or achieve the study purpose. Second, we recom- for undertaking a scoping study. The first purpose, as mend that a scoping study team be assembled whose articulated by Arksey and O’Malley [6], is to summarize members provide the methodological and context the extent, range, and nature of research activity; how- expertise needed for decisions regarding breadth and ever, researchers are not required to reflect on their comprehensiveness. When limiting scope is unavoid- underlying motivation for doing so. We recommend able, researchers should justify their decisions and that researchers consider the rationale for why they acknowledge the potential limitations of their study. should summarize the activity in a field and the implica- tions that this will have on research, practice, or policy. Framework stage three: Study selection The second purpose is to assess the need for a full sys- Arksey and O’Malley [6] provide suggestions to manage tematic review. However, it is difficult to determine the time-consuming process of determining which stu- whether a systematic review is advantageous when a dies to include in a scoping study. We experienced this scoping study does not involve methodological quality stage as more iterative and requiring additional steps assessment of included studies. Furthermore, it is than implied in the original framework. While Arksey unclear how this purpose differs from existing methods and O’Malley [6] do not indicate a team approach is of determining feasibility for a systematic review. The imperative, we agree with others and suggest scoping third purpose is to summarize and disseminate research studies involve multidisciplinary teams using a transpar- findings, but we question how this differs from other ent and replicable process [2,10]. In two of our studies narrative or systematic literature reviews. Lastly, the (HC and DL) where decision making was primarily fourth purpose of undertaking a scoping study – to completed by a single author, we faced several chal- identify gaps in the existing literature – may yield false lenges, including uncertainty about which studies to Levac et al. Implementation Science 2010, 5:69 Page 6 of 9 http://www.implementationscience.com/content/5/1/69 include, variables to extract on the data-charting form, the included studies into three areas –HIV disability, and the nature and extent of detail to conduct the data interventions, and roles of rehabilitation professionals in extraction process. This raised questions related to rigor HIV care – to help determine the nature and extent and led to our recommendations for undertaking a sys- of information to extract from each of the types of stu- tematic team approach to conducting a scoping study. dies [7]. Specifically, we recommend that the team meet to dis- Arksey and O’Malley [6] refer to a ‘descriptive analyti- cuss decisions surrounding study inclusion and exclu- cal method’ that involves summarizing process informa- sion at the beginning of the scoping process. Refining tion, such as the use of a theory or model in a the search strategy based on abstracts retrieved from the meaningful format. Our experiences indicated that this search and reviewing full articles for study inclusion is is a highly valuable, though challenging aspect of scop- also a critical step. We recommend that at least two ing studies, as we struggled to chart and summarize researchers each independently review abstracts yielded complex concepts in a meaningful way. Arksey and from the search strategy for study selection. Reviewers O’Malley [6] indicate that synthesis of material is critical should meet at the beginning, midpoint, and final stages as scoping studies are not a short summary of many of the abstract review process to discuss any challenges articles. We agree, and feel that additional direction in or uncertainties related to study selection and to go the framework might help to navigate this crucial but back and refine the search strategy if needed. This can challenging stage. Perhaps synthesizing process informa- help to alleviate potential ambiguity with a broad tion may benefit from utilization of qualitative content research question and to ensure that abstracts selected analysis approaches to make sense of the wealth of are relevant for full article review. Next, two reviewers extracted data [11]. This issue also highlights the overlap should independently review the full articles for inclu- with the next analytical stage. The role and relevance of sion. When disagreements occur, a third reviewer can analyzing process data and using qualitative content be consulted to determine final inclusion. analysis within scoping study methodology requires further discussion. Framework stage four: Charting the data This stage involves extracting data from included stu- Framework stage five: Collating, summarizing, and dies. Based on our experiences, we were uncertain about reporting the results the nature and extent of information to extract from the Stage five is the most extensive in the scoping process, included studies. To clarify this stage, we recommend yet it lacks detail in the Arksey and O’Malley frame- that the research team collectively develop the data- work. Scoping studies have been criticized for rarely charting form to determine which variables to extract providing methodological detail about how results were that will help to answer the research question. Secondly, achieved [1]. We appreciate the importance of breaking we recommend that charting be considered an iterative the analysis phase into meaningful and systematic steps process in which researchers continually update the so that researchers can provide this undertake scoping data-charting form. This is particularly true for process- studies and report on findings in a rigorous manner. As oriented data, such as understanding how a theory or a result, we recommend three distinct steps in frame- model has been used within a study. Uncertainty about work stage five to increase the consistency with which the nature and extent of data that should be extracted researchers undertake and report scoping study metho- may be resolved by researchers beginning the charting dology: analyzing the data, reporting results, and apply- process and becoming familiar with study data, and then ing meaning to the results. As described in the existing meeting again to refine the form. We recommend an framework, analysis (otherwise referred to as collating additional step to charting the data in which two and summarizing) should involve a descriptive numeri- researchers independently extract data from the first five cal summary and a thematic analysis. Arksey and to ten studies using the data-charting form and meet to O’Malley [6] describe the need to provide a descriptive determine whether their approach to data extraction is numerical summary, stating that researchers should consistent with the research question and purpose. describe the characteristics of included studies, such as Researchers may review one study several times within the overall number of studies included, types of study this stage. The number of researchers involved in the design, years of publication, types of interventions, char- data extraction process will likely depend upon the acteristics of the study populations, and countries where number of included studies. For example, in one study, studies were conducted. However, the description of authors had difficulty developing one data-charting form thematic analysis requires additional detail to assist that could apply to all included studies representing a authors in understanding and completing this step. In range study designs, reviews, reports, and commentaries our experience, this analytical stage resembled qualita- [7]. As a preliminary step, authors decided to classify tive data analytical techniques, and researchers may Levac et al. Implementation Science 2010, 5:69 Page 7 of 9 http://www.implementationscience.com/content/5/1/69 consider using qualitative content analytical techniques stakeholders with whom they wish to consult, how they [10] and qualitative software to facilitate this process. will collect the data (e.g., focus groups, interviews, sur- Second, when reporting results, we recommend that veys), and how these data will be analyzed, reported, researchers consider the best approach to stating the and integrated within the overall study outcome. outcome or end product of the study and how the scop- Finally, given that consultation requires researchers to ing study findings will be articulated to readers (e.g., orient stakeholders on the scoping study purpose, through themes, a framework, or a table of strengths research question, preliminary findings, and plans for and gaps in the evidence). This product should be tied dissemination, we recommend that this stage addition- to the purpose of the scoping study as recommended in ally be considered a knowledge transfer mechanism. framework stage one. This may address Brien et al.’s [5] concern about the Finally, in order to advance the legitimacy of scoping usefulness of scoping studies for stakeholders and how study methodology, we must consider the implications to translate knowledge about scoping studies. Given the of findings within the broader context. As a result, we importance of knowledge transfer and exchange in the recommend that researchers consider the meaning of uptake of research evidence [12,13], the consultation their scoping study results and the broader implications stage can be used to specifically translate the prelimin- for research, policy, and practice. For example, for the ary scoping study findings and develop effective dissemi- question ‘how are motor-learning strategies used within nation strategies with stakeholders in the field, offering contemporary physical and occupational therapy inter- additional value to a scoping study. vention approaches for children with neuromotor condi- One scoping study included a consultation phase com- tions?,’ the author (DL) presented themes that described prised of focus groups and interviews with 28 stake- strategy use. Results yielded insights into how research- holders including people living with HIV, researchers, ers should better describe interventions in their publica- educators, clinicians, and policy makers [7]. Authors tions and provided further considerations for clinicians shared preliminary findings from the literature review to make informed decisions about which therapeutic phase of the scoping study with stakeholders and asked approach might best fit their clients’ needs. Considering whether they may be able to identify any additional emer- the overall implications of the results as an explicit fra- ging issues related to HIV and rehabilitation not yet pub- mework stage will help to ensure that scoping study lished in the evidence. The team proceeded to conduct a results have practical implications for future clinical second consultation with 17 new and returning stake- practice, research, and policy. This recommendation holders whereby the team presented a preliminary frame- leads to the final stage of the framework. work of HIV and rehabilitation research and stakeholders refined the framework to further identify six key research Optional stage six: Consultation priorities on HIV and rehabilitation. This series of con- Arksey and O’Malley [6] suggest that consultation is an sultations engaged community members in the develop- optional stage in conducting a scoping study. Although ment of the study outcome and provided opportunities only one of our three scoping studies incorporated this for knowledge transfer about HIV and rehabilitation stage, we argue that it adds methodological rigor and research. This process offered an ideal mechanism to should be considered a required component. Arksey and enhance the validity of the study outcome while translat- O’Malley [6] suggest that the purposes of consulting ing findings with the community. Nevertheless, further with stakeholders are to offer additional sources of development of steps for undertaking knowledge transla- information, perspectives, meaning, and applicability to tion as a part of the scoping study framework is required. the scoping study. However, it is unclear when, how, and why to consult with stakeholders, and how to ana- Additional considerations for scoping studies to support lyze and integrate these data with the findings. We the advancement, application, and relevance of scoping recommend researchers clearly establish a purpose for studies in health research the consultation, which may include sharing preliminary Scoping study terminology findings with stakeholders, validating the findings, or Discrepancies in nomenclature between ‘scoping informing future research. We suggest researchers use reviews,’‘scoping studies,’‘scoping literature reviews,’ preliminary findings from stage five (either in the form and ‘scoping exercises’ lead to confusion. Despite our of a framework, themes, or list of findings) as a founda- collective use of the Arksey and O’Malley framework, tion from which to inform the consultation. This will two authors (DL, HC) titled their studies as ‘scoping enable stakeholders to build on the evidence and offer a reviews’ while the other used ‘scoping study.’ In this higher level of meaning, content expertise, and perspec- paper, we use ‘scoping studies’ for consistency with Ark- tive to the preliminary findings. We also recommend sey and O’Malley’s original framework. Nevertheless, the that researchers clearly articulate the type of potential differences (if any) among the terms merit Levac et al. Implementation Science 2010, 5:69 Page 8 of 9 http://www.implementationscience.com/content/5/1/69 clarification. Lack of a universal definition for scoping Summary studies is also problematic to researchers trying to Scoping studies present an increasingly popular option clearly articulate their reasons for undertaking a scoping for synthesizing health evidence. Brien et al. [5] argue study. Finally, we advocate for labeling the methodology that guidelines are required to facilitate scoping review as the ‘Arksey and O’Malley framework’ to provide con- reporting and transparency. In this paper, we build on sistency for future use. the existing methodologicalframework for scoping stu- Quality assessment dies outlined by Arksey and O’Malley [6] and provide Another consideration for scoping study methodology is recommendations to clarify and enhance each stage, the potential need to assess included studies for metho- which may increase the consistency with which dological quality. Brien et al. [5] state that this lack of researchers undertake and report scoping studies. quality assessment makes the results of scoping studies Recommendations include: clarifying and linking the more challenging to interpret. Grant and Booth [4] purpose and research question; balancing feasibility with imply that a lack of quality assessment limits the uptake breadth and comprehensiveness of the scoping process; of scoping study findings into policy and practice. While using an iterative team approach to selecting studies our research questions did not directly relate to any and extracting data; incorporating a numerical summary quality assessment debate, we recognize the challenges and qualitative thematic analysis; identifying the implica- in assessing quality among the vast range of published tions of the study findings for policy, practice, or and grey literature that may be included in scoping stu- research; and adopting consultation as a required com- dies. This also raises the question of whether and how ponent of scoping study methodology. Ongoing consid- evidence from stakeholder consultation is evaluated in erations include: establishing a common accepted the scoping study process. It remains unclear whether definition and purpose(s) of scoping studies; defining the lack of quality assessment impacts the uptake and methodological rigor for the assessment of scoping relevance of scoping study findings. study quality; debating the need for quality assessment A final consideration for legitimization of scoping of included studies; and formalizing knowledge transla- study methodology includes the development of a criti- tion as a required element of scoping methodology. cal appraisal tool for scoping study quality [5]. Anderson Continued debate and development about scoping study et al. [2] offer criteria for assessing the value and utility methodology will help to maximize the usefulness of of a commissioned scoping study in health policy con- scoping study findings within healthcare research and texts, but these criteria are not necessarily applicable to practice. scoping studies in other areas of health research. Devel- oping a critical appraisal tool would require the ele- Acknowledgements ments of a methodologically rigorous scoping study to DL is supported by a Doctoral Award from the Canadian Child Health be defined. This could include, but would not be limited Clinician Scientist Program, a strategic training initiative of the Canadian Institutes of Health Research (CIHR), and the McMaster Child Health Research to, the minimum level of analysis required and the Institute. HC is supported by a Doctoral Award from the CIHR, the CIHR requirements for reporting results. Overall, the issues Quality of Life Strategic Training Program in Rehabilitation Research and the surrounding quality assessment of included studies and Canadian Occupational Therapy Foundation. KO is supported by a Fellowship from the CIHR, HIV/AIDS Research Program and a Michael subsequent scoping studies require further discussion. DeGroote Postdoctoral Fellowship (McMaster University). The authors acknowledge the helpful feedback of Dr. Cheryl Missiuna on an earlier draft of this manuscript. Limitations This paper responds to Arksey and O’Malley’s[6] Author details request for feedback to their proposed methodological School of Rehabilitation Science, McMaster University, 1400 Main Street West, Room 403, Hamilton, Ontario, Canada. Department of Physical framework. However, the recommendations that we pro- Therapy, University of Toronto, 160-500 University Avenue, Toronto, Ontario, pose are derived from our subjective experiences under- Canada. taking scoping studies of varying sizes in the Authors’ contributions rehabilitation field, and we recognize that they may not DL and HC conceived of this paper. DL undertook the literature review represent the opinions of all scoping study authors. process. DL, HC and KO developed challenges and recommendations. All Other than our individual experiences with our own stu- authors drafted the manuscript. All authors read and approved the final manuscript. dies, we have not yet implemented the full framework recommendations. Hence, readers can determine how Authors’ information strongly to interpret and implement these recommenda- DL is a physical therapist and doctoral candidate in the School of Rehabilitation Science at McMaster University. HC is an occupational tions in their scoping study research. We invite others therapist and doctoral candidate in the School of Rehabilitation Science at to trial our recommendations and continue the process McMaster University. KO is a clinical epidemiologist, physical therapist, and of refining and improving this methodology. postdoctoral fellow in the School of Rehabilitation Science at McMaster Levac et al. Implementation Science 2010, 5:69 Page 9 of 9 http://www.implementationscience.com/content/5/1/69 University. She is also a Lecturer in the Department of Physical Therapy at the University of Toronto. Competing interests The authors declare that they have no competing interests. Received: 11 June 2010 Accepted: 20 September 2010 Published: 20 September 2010 References 1. Davis K, Drey N, Gould D: What are scoping studies? A review of the nursing literature. Int J Nurs Stud 2009, 46:1386-1400. 2. Anderson S, Allen P, Peckham S, Goodwin N: Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Sys 2008, 6:7. 3. Rumrill P, Fitzgerald S, Merchant W: Using scoping literature reviews as a means of understanding and interpreting existing literature. Work 2010, 35:399-404. 4. Grant M, Booth A: A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009, 26:91-108. 5. Brien S, Lorenzetti D, Lewis S, Kennedy J, Ghali W: Overview of a formal scoping review on health system report cards. Implement Sci 2010, 5:2. 6. Arksey H, O’Malley L: Scoping studies: Towards a Methodological Framework. Int J Soc Res Methodol 2005, 8:19-32. 7. O’Brien K, Wilkins A, Zack E, Solomon P: Scoping the field: identifying key research priorities in HIV and rehabilitation. AIDS Behav 2010, 14:448-458. 8. Levac D, Wishart L, Missiuna C, Wright V: The application of motor learning strategies within functionally based interventions for children with neuromotor conditions. Peds Phys Ther 2009, 21:345-355. 9. Colquhoun H, Letts L, Law M, MacDermid J, Missiuna C: A scoping review of the use of theory in studies of knowledge translation. Can J Occup Ther . 10. Ehrich K, Freeman G, Richards S, Robinson I, Shepperd S: How to do a scoping exercise: continuity of care. Res Pol Plan 2002, 20:25-29. 11. Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res 2005, 15:1277-1288. 12. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: time for a map? J Contin Educ Health Prof 2006, 26:13-24. 13. Schuster M, McGlynn E, Brook R: How good is the quality of healthcare in the United States? Milbank Q 2005, 83:843-895. 14. Mays N, Roberts E, Popay J: Synthesising research evidence. In Studying the organization and delivery of health services: research methods. Edited by: Fulop N, Allen P, Clarke A, Black N. London: Routledge; 2001:194. 15. Canadian Institutes of Health Research: Knowledge Translation. [http:// www.cihr-irsc.gc.ca/e/29418.html]. doi:10.1186/1748-5908-5-69 Cite this article as: Levac et al.: Scoping studies: advancing the methodology. Implementation Science 2010 5:69. Submit your next manuscript to BioMed Central and take full advantage of: • Convenient online submission • Thorough peer review • No space constraints or color figure charges • Immediate publication on acceptance • Inclusion in PubMed, CAS, Scopus and Google Scholar • Research which is freely available for redistribution Submit your manuscript at www.biomedcentral.com/submit http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Implementation Science Springer Journals

Scoping studies: advancing the methodology

Download PDF
9 pages

Loading next page...
 
/lp/springer-journals/scoping-studies-advancing-the-methodology-QgpPycl73e

References (34)

Publisher
Springer Journals
Copyright
Copyright © 2010 by Levac et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; Health Promotion and Disease Prevention; Health Administration; Health Informatics; Public Health
eISSN
1748-5908
DOI
10.1186/1748-5908-5-69
pmid
20854677
Publisher site
See Article on Publisher Site

Abstract

Background: Scoping studies are an increasingly popular approach to reviewing health research evidence. In 2005, Arksey and O’Malley published the first methodological framework for conducting scoping studies. While this framework provides an excellent foundation for scoping study methodology, further clarifying and enhancing this framework will help support the consistency with which authors undertake and report scoping studies and may encourage researchers and clinicians to engage in this process. Discussion: We build upon our experiences conducting three scoping studies using the Arksey and O’Malley methodology to propose recommendations that clarify and enhance each stage of the framework. Recommendations include: clarifying and linking the purpose and research question (stage one); balancing feasibility with breadth and comprehensiveness of the scoping process (stage two); using an iterative team approach to selecting studies (stage three) and extracting data (stage four); incorporating a numerical summary and qualitative thematic analysis, reporting results, and considering the implications of study findings to policy, practice, or research (stage five); and incorporating consultation with stakeholders as a required knowledge translation component of scoping study methodology (stage six). Lastly, we propose additional considerations for scoping study methodology in order to support the advancement, application and relevance of scoping studies in health research. Summary: Specific recommendations to clarify and enhance this methodology are outlined for each stage of the Arksey and O’Malley framework. Continued debate and development about scoping study methodology will help to maximize the usefulness and rigor of scoping study findings within healthcare research and practice. Background summarize and disseminate research findings, or identify Scoping studies (or scoping reviews) represent an gaps in the existing literature [6]. As such, researchers increasingly popular approach to reviewing health can use scoping studies to clarify a complex concept research evidence [1]. However, no universal scoping and refine subsequent research inquiries [1]. Scoping study definition or purpose exists (Table 1) [1,2]. Defini- studies may be particularly relevant to disciplines with tions commonly refer to ‘mapping,’ a process of sum- emerging evidence, such as rehabilitation science, in marizing a range of evidence in order to convey the which the paucity of randomized controlled trials makes breadth and depth of a field. Scoping studies differ from it difficult for researchers to undertake systematic systematic reviews because authors do not typically reviews. In these situations, scoping studies are ideal assess the quality of included studies [3-5]. Scoping stu- because researchers can incorporate a range of study dies also differ from narrative or literature reviews in designs in both published and grey literature, address that the scoping process requires analytical reinterpreta- questions beyond those related to intervention effective- tion of the literature [1]. ness, and generate findings that can complement the Researchers can undertake a scoping study to examine findings of clinical trials. the extent, range, and nature of research activity, deter- In an effort to provide guidance to authors undertaking mine the value of undertaking a full systematic review, scoping studies, Arksey and O’Malley [6] developed a six- stage methodological framework: identifying the research question, searching for relevant studies, selecting studies, * Correspondence: [email protected] 1 charting the data, collating, summarizing, and reporting School of Rehabilitation Science, McMaster University, 1400 Main Street West, Room 403, Hamilton, Ontario, Canada the results, and consulting with stakeholders to inform or Full list of author information is available at the end of the article © 2010 Levac et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Levac et al. Implementation Science 2010, 5:69 Page 2 of 9 http://www.implementationscience.com/content/5/1/69 Table 1 Definitions and purposes of scoping studies Authors Definition Purpose(s) Ehrich et al. (2002) None provided. ’The purpose of a scoping exercise is both to map a wide range of literature, and to envisage where gaps and innovative approaches may lie"’ [[11] p. 28]. Arksey and O’Malley ’Aim to map rapidly the key concepts underpinning a 1. To examine the extent, range, and nature of research (2005) research area and the main sources and types of evidence activity. available’ [[14] p. 194], as cited in [6] 2. To determine the value for undertaking a full systematic review. 3. To summarize and disseminate research findings. 4. To identify research gaps in the existing literature. [[6] p. 21] Anderson et al. (2008) ’Scoping studies are concerned with contextualizing 1. Literature mapping: ‘is a map of the relevant literature. knowledge in terms of identifying the current state of These vary in scope from general accounts of the literature understanding; identifying the sorts of things we know and to studies that are just short of systematic reviews. do not know; and then setting this within policy and Literature scoping studies often also involve the syntheses practice contexts’ [2] of findings from different types of study.’ 2. Conceptual mapping: ‘a scoping study designed to establish how a particular term is used in what literature, by whom and for what purpose.’ 3. Policy mapping: ‘a scoping study designed to identify the main documents and statements from government agencies and professional bodies that have a bearing on the nature of practice in that area.’’ 4. Stakeholder consultation: ‘Do[es] not constitute scoping studies in their own right, but they do have an important part to play in scoping studies concerned with the identification of research priorities, in helping to target research questions, and in validating the outcomes of scoping studies through peer-review’ [2]. Grant et al. (2009) ’Preliminary assessment of potential size and scope of ’Aims to identify the nature and extent of research research literature.’ [[4] p.95] evidence (usually including ongoing research’ [[4] p.95]. Davis et al. (2009) ’Scoping involves the synthesis and analysis of a wide ’We propose that a common synthesising construct range of research and non-research material to provide emerges to explain the purpose of scoping, namely that of greater conceptual clarity about a specific topic or field of ‘reconnaissance’. It is generally synonymous with a evidence’ [[1] p.1386]. preliminarily investigation in which information is systematically gathered and examined in order to establish strengths and weakness and guide in which ever context, future decision-making’ [[1] p. 1396]. National Institute for None provided. 1.’Clarification of working definitions and conceptual Health Research (NIHR) boundaries of a topic area, developed in the form of Service Delivery and systematic overview (narrative review) of the literature but Organisation specifically excluding a systematic review, to determine a Research and frame of reference; Development 2. Outline what is already known and identify gaps in Programme (SDO) existing research, and; 4. Conceptual analysis may include the ‘mapping’ of existing empirical evidence to describe and interpret issues that will inform further research and development opportunities.’ [[1] p. 1387] Canadian Institutes of ’Scoping reviews are exploratory projects that systematically None provided. Health Research map the literature available on a topic, identifying the key http://www.cihr-irsc.ca concepts, theories, sources of evidence, and gaps in the research. They are often preliminary to full syntheses, undertaken when feasibility is a concern – either because the potentially relevant literature is thought to be especially vast and diverse (varying by method, theoretical orientation or discipline) or there is suspicion that not enough literature exists. These entail the systematic selection, collection and summarization of existing knowledge in a broad thematic area for the purpose of identifying where there is sufficient evidence to conduct a full synthesis or where insufficient evidence exists and further primary research is necessary.’ [15] Levac et al. Implementation Science 2010, 5:69 Page 3 of 9 http://www.implementationscience.com/content/5/1/69 validate study findings (Table 2). While this framework on scoping study methodology. We searched CINAHL, provided an excellent methodological foundation, pub- MEDLINE, PubMed, ERIC, PsycInfo, and Web of lished scoping studies continue to lack sufficient metho- Science databases using the search terms ‘scoping,’ dological description or detail about the data analysis ‘scoping study,’‘scoping review,’ and ‘scoping methodol- process, making it challenging for readers to understand ogy’ for papers published in English between January how study findings were determined [1]. Arksey and 1990 and May 2010. Reference lists of pertinent papers O’Malley [6] encouraged other authors to refine their fra- were also searched. This search yielded seven citations mework in order to enhance the methodology. that reflected on scoping study methodology, which In this paper, we apply our experiences using the Arksey were reviewed by one author (DL). After independently and O’Malley framework to build on the existing metho- considering our own experiences utilizing the Arskey dological framework. Specifically, we propose recommen- and O’Malley [6] framework, we met on seven occasions dations for each stage of the framework, followed by to discuss the challenges and develop recommendations considerations for the advancement, application, and rele- for each stage of the methodological framework. vance of scoping studies in health research. Continual refinement of the framework stages may provide greater Recommendations to enhance scoping study clarity about scoping study methodology, encourage methodology researchers and clinicians to engage in this process, and We outline the challenges and recommendations asso- help to enhance the methodological rigor with which ciated with each stage of the methodological framework authors undertake and report scoping studies [1]. (Table 3). Discussion Framework stage one: Identifying the research question We each completed a scoping study in separate areas of Scoping study research questions are broad in nature as rehabilitation using the Arksey and O’Malley framework the focus is on summarizing breadth of evidence. Arksey [6]. Goals of these studies included: identifying research and O’Malley [6] acknowledge the need to maintain a priorities within HIV and rehabilitation [7], applying broad scope to research questions, however we found motor learning strategies within pediatric physical and our research questions lacked the direction, clarity, and occupational therapy intervention approaches [8], and focus needed to inform subsequent stages of the exploring the use of theory within studies of knowledge research process, such as identifying studies and making translation [9]. The amount of literature reviewed in our decisions about study inclusion. To clarify this stage, we studies ranged from 31 (DL) to 146 (KO) publications. recommend that researchers combine a broad research Upon discovering that we had similar challenges imple- question with a clearly articulated scope of inquiry. This menting the scoping study methodology, we decided to includes defining the concept, target population, and use our experiences to further develop the existing health outcomes of interest to clarify the focus of the framework. We conducted an informal literature search scoping study and establish an effective search strategy. Table 2 Overview of the Arksey and O’Malley methodological framework for conducting a scoping study Arksey and O’Malley Framework Description Stage 1: Identifying the research question Identifying the research question provides the roadmap for subsequent stages. Relevant aspects of the question must be clearly defined as they have ramifications for search strategies. Research questions are broad in nature as they seek to provide breadth of coverage. 2: Identifying relevant studies This stage involves identifying the relevant studies and developing a decision plan for where to search, which terms to use, which sources are to be searched, time span, and language. Comprehensiveness and breadth is important in the search. Sources include electronic databases, reference lists, hand searching of key journals, and organizations and conferences. Breadth is important; however, practicalities of the search are as well. Time, budget and personnel resources are potential limiting factors and decisions need to be made upfront about how these will impact the search. 3: Study selection Study selection involves post hoc inclusion and exclusion criteria. These criteria are based on the specifics of the research question and on new familiarity with the subject matter through reading the studies. 4: Charting the data A data-charting form is developed and used to extract data from each study. A ‘narrative review’ or ‘descriptive analytical’ method is used to extract contextual or process oriented information from each study. 5: Collating, summarizing, and An analytic framework or thematic construction is used to provide an overview of the breadth of the literature reporting results but not a synthesis. A numerical analysis of the extent and nature of studies using tables and charts is presented. A thematic analysis is then presented. Clarity and consistency are required when reporting results. 6: Consultation (optional) Provides opportunities for consumer and stakeholder involvement to suggest additional references and provide insights beyond those in the literature. Levac et al. Implementation Science 2010, 5:69 Page 4 of 9 http://www.implementationscience.com/content/5/1/69 Table 3 Summary of challenges and recommendations for scoping studies Framework Stage Challenges Recommendations for clarification or additional steps #1 Identifying the 1. Scoping study questions are broad. 1. Clearly articulate the research question that will guide the research question 2. Establishing scoping study purpose is not associated scope of inquiry. Consider the concept, target population, and with a framework stage. health outcomes of interest to clarify the focus of the scoping 3. The four purposes of scoping studies lack clarity. study and establish an effective search strategy. 2. Mutually consider the purpose of the scoping study with the research question. Envision the intended outcome (e.g., framework, list of recommendations) to help determine the purpose of the study. 3. Consider rationale for conducting the scoping study to help clarify the purpose. #2 Identifying 1. Balancing breadth and comprehensiveness of the 1a. Research question and purpose should guide decision- relevant studies scoping study with feasibility of resources can be making around the scope of the study. challenging. 1b. Assemble a suitable team with content and methodological expertise that will ensure successful completion of the study. 1c. When limiting scope is unavoidable, justify decisions and acknowledge the potential limitations to the study. #3 Study selection 1. The linearity of this stage is misleading. 1. This stage should be considered an iterative process 2. The process of decision making for study selection is involving searching the literature, refining the search strategy, unclear. and reviewing articles for study inclusion. 2a. At the beginning of the process, the team should meet to discuss decisions surrounding study inclusion and exclusion. At least two reviewers should independently review abstracts for inclusion. 2b. Reviewers should meet at the beginning, midpoint and final stages of the abstract review process to discuss challenges and uncertainties related to study selection and to go back and refine the search strategy if needed. 2c. Two researchers should independently review full articles for inclusion. 2d. When disagreements on study inclusion occur, a third reviewer can determine final inclusion. #4 Charting the data 1. The nature and extent of data to extract from included 1a. The research team should collectively develop the data- studies is unclear. charting form and determine which variables to extract in 2. The ‘descriptive analytical method’ of charting data is order to answer the research question. poorly defined. 1b. Charting should be considered an iterative process in which researchers continually extract data and update the data- charting form. 1c. Two authors should independently extract data from the first five to ten included studies using the data-charting form and meet to determine whether their approach to data extraction is consistent with the research question and purpose. 2. Process-oriented data may require extra planning for analysis. A qualitative content analysis approach is suggested. #5 Collating, 1. Little detail provided and multiple steps are summarized Researchers should break this stage into three distinct steps: summarizing, and as one framework stage. 1a. Analysis (including descriptive numerical summary analysis reporting the results and qualitative thematic analysis); 1b. Reporting the results and producing the outcome that refers to the overall purpose or research question; 1c. Consider the meaning of the findings as they relate to the overall study purpose; discuss implications for future research, practice and policy. #6 Consultation 1. This stage is optional. 1. Consultation should be an essential component of scoping 2. Lack of clarity exists about when, how and why to study methodology. consult with stakeholders and how to integrate the 2a. Clearly establish a purpose for the consultation. information with study findings. 2b. Preliminary findings can be used as a foundation to inform the consultation. 2c. Clearly articulate the type of stakeholders to consult and how data will be collected, analyzed, reported and integrated within the overall study outcome. 2d. Incorporate opportunities for knowledge transfer and exchange with stakeholders in the field. Levac et al. Implementation Science 2010, 5:69 Page 5 of 9 http://www.implementationscience.com/content/5/1/69 For example, in one author’s (KO) scoping study, the conclusions about the nature and extent of those gaps if research question was broadly ‘what is known about the quality of the evidence is not assessed. The purpose HIV and rehabilitation?’ Defining the concept of ‘rehabi- ‘to identify the key research priorities in HIV and reha- litation’ was essential in order to establish a clear scope bilitation to advance policy and practice for people liv- to the study, guide the search strategy, and establish ing with HIV in Canada’ does not explicitly align with parameters around study selection in subsequent stages one of the four Arskey and O’Malley purposes [7]. How- of the process [7]. ever, it appears authors inherently first summarized the Although Arskey and O’Malley [6] outline four main extent, range, and nature of research (purpose one) and purposes for undertaking a scoping study, they do not identified gaps in the existing literature (purpose four) articulate that purpose be specified within a specific fra- in order to subsequently identify the key research priori- mework stage. We recommend researchers simulta- ties in HIV and rehabilitation (author purpose). This neously consider the purpose of the scoping study when suggests authors might have an overall study purpose articulating the research question. Linking a clear pur- with multiple objectives articulated by Arksey and pose for undertaking a scoping study to a well-defined O’Malley that are required in order to help achieve their research question at the first stage of the framework will overall purpose. help to provide a clear rationale for completing the study and facilitate decision making about study selec- Framework stage two: Identifying relevant studies tion and data extraction later in the methodological pro- A strength of scoping studies includes the breadth and cess. A helpful strategy may be to envision the content depth, or comprehensiveness, of evidence covered in a and format of the intended outcome that may assist given field [1]. However, practical issues related to researchers to clearly determine the purpose at the time, funding, and access to resources often require beginning of a study. In the abovementioned HIV study, researchers to consider the balance between feasibility, authors linked the broadly stated research question with breadth, and comprehensiveness. Brien et al. [5] amorespecificpurpose ‘to identify the key research reported that their search strategy yielded a vast priorities in HIV and rehabilitation to advance policy amount of literature, making it difficult to determine and practice for people living with HIV in Canada’ [7]. how in depth to carry out the information synthesis. The envisioned outcome was a thematic framework that Although Arksey and O’Malley [6] identify these con- represented strengths and opportunities in HIV rehabili- cerns and provide some suggestions to support these tation research, followed by a list of the key research decisions, we also struggled with the trade-off between priorities to pursue in future work. breadth and comprehensiveness and feasibility in our Finally, the purposes put forth by Arksey and O’Mal- scoping studies. As such, we recommend that research- ley [6] require more debate. We concur with Anderson ers ensure decisions surrounding feasibility do not et al. [2] and Davis et al. [1], who state that researchers compromise their ability to answer the research ques- may benefit from further clarification of the purposes tion or achieve the study purpose. Second, we recom- for undertaking a scoping study. The first purpose, as mend that a scoping study team be assembled whose articulated by Arksey and O’Malley [6], is to summarize members provide the methodological and context the extent, range, and nature of research activity; how- expertise needed for decisions regarding breadth and ever, researchers are not required to reflect on their comprehensiveness. When limiting scope is unavoid- underlying motivation for doing so. We recommend able, researchers should justify their decisions and that researchers consider the rationale for why they acknowledge the potential limitations of their study. should summarize the activity in a field and the implica- tions that this will have on research, practice, or policy. Framework stage three: Study selection The second purpose is to assess the need for a full sys- Arksey and O’Malley [6] provide suggestions to manage tematic review. However, it is difficult to determine the time-consuming process of determining which stu- whether a systematic review is advantageous when a dies to include in a scoping study. We experienced this scoping study does not involve methodological quality stage as more iterative and requiring additional steps assessment of included studies. Furthermore, it is than implied in the original framework. While Arksey unclear how this purpose differs from existing methods and O’Malley [6] do not indicate a team approach is of determining feasibility for a systematic review. The imperative, we agree with others and suggest scoping third purpose is to summarize and disseminate research studies involve multidisciplinary teams using a transpar- findings, but we question how this differs from other ent and replicable process [2,10]. In two of our studies narrative or systematic literature reviews. Lastly, the (HC and DL) where decision making was primarily fourth purpose of undertaking a scoping study – to completed by a single author, we faced several chal- identify gaps in the existing literature – may yield false lenges, including uncertainty about which studies to Levac et al. Implementation Science 2010, 5:69 Page 6 of 9 http://www.implementationscience.com/content/5/1/69 include, variables to extract on the data-charting form, the included studies into three areas –HIV disability, and the nature and extent of detail to conduct the data interventions, and roles of rehabilitation professionals in extraction process. This raised questions related to rigor HIV care – to help determine the nature and extent and led to our recommendations for undertaking a sys- of information to extract from each of the types of stu- tematic team approach to conducting a scoping study. dies [7]. Specifically, we recommend that the team meet to dis- Arksey and O’Malley [6] refer to a ‘descriptive analyti- cuss decisions surrounding study inclusion and exclu- cal method’ that involves summarizing process informa- sion at the beginning of the scoping process. Refining tion, such as the use of a theory or model in a the search strategy based on abstracts retrieved from the meaningful format. Our experiences indicated that this search and reviewing full articles for study inclusion is is a highly valuable, though challenging aspect of scop- also a critical step. We recommend that at least two ing studies, as we struggled to chart and summarize researchers each independently review abstracts yielded complex concepts in a meaningful way. Arksey and from the search strategy for study selection. Reviewers O’Malley [6] indicate that synthesis of material is critical should meet at the beginning, midpoint, and final stages as scoping studies are not a short summary of many of the abstract review process to discuss any challenges articles. We agree, and feel that additional direction in or uncertainties related to study selection and to go the framework might help to navigate this crucial but back and refine the search strategy if needed. This can challenging stage. Perhaps synthesizing process informa- help to alleviate potential ambiguity with a broad tion may benefit from utilization of qualitative content research question and to ensure that abstracts selected analysis approaches to make sense of the wealth of are relevant for full article review. Next, two reviewers extracted data [11]. This issue also highlights the overlap should independently review the full articles for inclu- with the next analytical stage. The role and relevance of sion. When disagreements occur, a third reviewer can analyzing process data and using qualitative content be consulted to determine final inclusion. analysis within scoping study methodology requires further discussion. Framework stage four: Charting the data This stage involves extracting data from included stu- Framework stage five: Collating, summarizing, and dies. Based on our experiences, we were uncertain about reporting the results the nature and extent of information to extract from the Stage five is the most extensive in the scoping process, included studies. To clarify this stage, we recommend yet it lacks detail in the Arksey and O’Malley frame- that the research team collectively develop the data- work. Scoping studies have been criticized for rarely charting form to determine which variables to extract providing methodological detail about how results were that will help to answer the research question. Secondly, achieved [1]. We appreciate the importance of breaking we recommend that charting be considered an iterative the analysis phase into meaningful and systematic steps process in which researchers continually update the so that researchers can provide this undertake scoping data-charting form. This is particularly true for process- studies and report on findings in a rigorous manner. As oriented data, such as understanding how a theory or a result, we recommend three distinct steps in frame- model has been used within a study. Uncertainty about work stage five to increase the consistency with which the nature and extent of data that should be extracted researchers undertake and report scoping study metho- may be resolved by researchers beginning the charting dology: analyzing the data, reporting results, and apply- process and becoming familiar with study data, and then ing meaning to the results. As described in the existing meeting again to refine the form. We recommend an framework, analysis (otherwise referred to as collating additional step to charting the data in which two and summarizing) should involve a descriptive numeri- researchers independently extract data from the first five cal summary and a thematic analysis. Arksey and to ten studies using the data-charting form and meet to O’Malley [6] describe the need to provide a descriptive determine whether their approach to data extraction is numerical summary, stating that researchers should consistent with the research question and purpose. describe the characteristics of included studies, such as Researchers may review one study several times within the overall number of studies included, types of study this stage. The number of researchers involved in the design, years of publication, types of interventions, char- data extraction process will likely depend upon the acteristics of the study populations, and countries where number of included studies. For example, in one study, studies were conducted. However, the description of authors had difficulty developing one data-charting form thematic analysis requires additional detail to assist that could apply to all included studies representing a authors in understanding and completing this step. In range study designs, reviews, reports, and commentaries our experience, this analytical stage resembled qualita- [7]. As a preliminary step, authors decided to classify tive data analytical techniques, and researchers may Levac et al. Implementation Science 2010, 5:69 Page 7 of 9 http://www.implementationscience.com/content/5/1/69 consider using qualitative content analytical techniques stakeholders with whom they wish to consult, how they [10] and qualitative software to facilitate this process. will collect the data (e.g., focus groups, interviews, sur- Second, when reporting results, we recommend that veys), and how these data will be analyzed, reported, researchers consider the best approach to stating the and integrated within the overall study outcome. outcome or end product of the study and how the scop- Finally, given that consultation requires researchers to ing study findings will be articulated to readers (e.g., orient stakeholders on the scoping study purpose, through themes, a framework, or a table of strengths research question, preliminary findings, and plans for and gaps in the evidence). This product should be tied dissemination, we recommend that this stage addition- to the purpose of the scoping study as recommended in ally be considered a knowledge transfer mechanism. framework stage one. This may address Brien et al.’s [5] concern about the Finally, in order to advance the legitimacy of scoping usefulness of scoping studies for stakeholders and how study methodology, we must consider the implications to translate knowledge about scoping studies. Given the of findings within the broader context. As a result, we importance of knowledge transfer and exchange in the recommend that researchers consider the meaning of uptake of research evidence [12,13], the consultation their scoping study results and the broader implications stage can be used to specifically translate the prelimin- for research, policy, and practice. For example, for the ary scoping study findings and develop effective dissemi- question ‘how are motor-learning strategies used within nation strategies with stakeholders in the field, offering contemporary physical and occupational therapy inter- additional value to a scoping study. vention approaches for children with neuromotor condi- One scoping study included a consultation phase com- tions?,’ the author (DL) presented themes that described prised of focus groups and interviews with 28 stake- strategy use. Results yielded insights into how research- holders including people living with HIV, researchers, ers should better describe interventions in their publica- educators, clinicians, and policy makers [7]. Authors tions and provided further considerations for clinicians shared preliminary findings from the literature review to make informed decisions about which therapeutic phase of the scoping study with stakeholders and asked approach might best fit their clients’ needs. Considering whether they may be able to identify any additional emer- the overall implications of the results as an explicit fra- ging issues related to HIV and rehabilitation not yet pub- mework stage will help to ensure that scoping study lished in the evidence. The team proceeded to conduct a results have practical implications for future clinical second consultation with 17 new and returning stake- practice, research, and policy. This recommendation holders whereby the team presented a preliminary frame- leads to the final stage of the framework. work of HIV and rehabilitation research and stakeholders refined the framework to further identify six key research Optional stage six: Consultation priorities on HIV and rehabilitation. This series of con- Arksey and O’Malley [6] suggest that consultation is an sultations engaged community members in the develop- optional stage in conducting a scoping study. Although ment of the study outcome and provided opportunities only one of our three scoping studies incorporated this for knowledge transfer about HIV and rehabilitation stage, we argue that it adds methodological rigor and research. This process offered an ideal mechanism to should be considered a required component. Arksey and enhance the validity of the study outcome while translat- O’Malley [6] suggest that the purposes of consulting ing findings with the community. Nevertheless, further with stakeholders are to offer additional sources of development of steps for undertaking knowledge transla- information, perspectives, meaning, and applicability to tion as a part of the scoping study framework is required. the scoping study. However, it is unclear when, how, and why to consult with stakeholders, and how to ana- Additional considerations for scoping studies to support lyze and integrate these data with the findings. We the advancement, application, and relevance of scoping recommend researchers clearly establish a purpose for studies in health research the consultation, which may include sharing preliminary Scoping study terminology findings with stakeholders, validating the findings, or Discrepancies in nomenclature between ‘scoping informing future research. We suggest researchers use reviews,’‘scoping studies,’‘scoping literature reviews,’ preliminary findings from stage five (either in the form and ‘scoping exercises’ lead to confusion. Despite our of a framework, themes, or list of findings) as a founda- collective use of the Arksey and O’Malley framework, tion from which to inform the consultation. This will two authors (DL, HC) titled their studies as ‘scoping enable stakeholders to build on the evidence and offer a reviews’ while the other used ‘scoping study.’ In this higher level of meaning, content expertise, and perspec- paper, we use ‘scoping studies’ for consistency with Ark- tive to the preliminary findings. We also recommend sey and O’Malley’s original framework. Nevertheless, the that researchers clearly articulate the type of potential differences (if any) among the terms merit Levac et al. Implementation Science 2010, 5:69 Page 8 of 9 http://www.implementationscience.com/content/5/1/69 clarification. Lack of a universal definition for scoping Summary studies is also problematic to researchers trying to Scoping studies present an increasingly popular option clearly articulate their reasons for undertaking a scoping for synthesizing health evidence. Brien et al. [5] argue study. Finally, we advocate for labeling the methodology that guidelines are required to facilitate scoping review as the ‘Arksey and O’Malley framework’ to provide con- reporting and transparency. In this paper, we build on sistency for future use. the existing methodologicalframework for scoping stu- Quality assessment dies outlined by Arksey and O’Malley [6] and provide Another consideration for scoping study methodology is recommendations to clarify and enhance each stage, the potential need to assess included studies for metho- which may increase the consistency with which dological quality. Brien et al. [5] state that this lack of researchers undertake and report scoping studies. quality assessment makes the results of scoping studies Recommendations include: clarifying and linking the more challenging to interpret. Grant and Booth [4] purpose and research question; balancing feasibility with imply that a lack of quality assessment limits the uptake breadth and comprehensiveness of the scoping process; of scoping study findings into policy and practice. While using an iterative team approach to selecting studies our research questions did not directly relate to any and extracting data; incorporating a numerical summary quality assessment debate, we recognize the challenges and qualitative thematic analysis; identifying the implica- in assessing quality among the vast range of published tions of the study findings for policy, practice, or and grey literature that may be included in scoping stu- research; and adopting consultation as a required com- dies. This also raises the question of whether and how ponent of scoping study methodology. Ongoing consid- evidence from stakeholder consultation is evaluated in erations include: establishing a common accepted the scoping study process. It remains unclear whether definition and purpose(s) of scoping studies; defining the lack of quality assessment impacts the uptake and methodological rigor for the assessment of scoping relevance of scoping study findings. study quality; debating the need for quality assessment A final consideration for legitimization of scoping of included studies; and formalizing knowledge transla- study methodology includes the development of a criti- tion as a required element of scoping methodology. cal appraisal tool for scoping study quality [5]. Anderson Continued debate and development about scoping study et al. [2] offer criteria for assessing the value and utility methodology will help to maximize the usefulness of of a commissioned scoping study in health policy con- scoping study findings within healthcare research and texts, but these criteria are not necessarily applicable to practice. scoping studies in other areas of health research. Devel- oping a critical appraisal tool would require the ele- Acknowledgements ments of a methodologically rigorous scoping study to DL is supported by a Doctoral Award from the Canadian Child Health be defined. This could include, but would not be limited Clinician Scientist Program, a strategic training initiative of the Canadian Institutes of Health Research (CIHR), and the McMaster Child Health Research to, the minimum level of analysis required and the Institute. HC is supported by a Doctoral Award from the CIHR, the CIHR requirements for reporting results. Overall, the issues Quality of Life Strategic Training Program in Rehabilitation Research and the surrounding quality assessment of included studies and Canadian Occupational Therapy Foundation. KO is supported by a Fellowship from the CIHR, HIV/AIDS Research Program and a Michael subsequent scoping studies require further discussion. DeGroote Postdoctoral Fellowship (McMaster University). The authors acknowledge the helpful feedback of Dr. Cheryl Missiuna on an earlier draft of this manuscript. Limitations This paper responds to Arksey and O’Malley’s[6] Author details request for feedback to their proposed methodological School of Rehabilitation Science, McMaster University, 1400 Main Street West, Room 403, Hamilton, Ontario, Canada. Department of Physical framework. However, the recommendations that we pro- Therapy, University of Toronto, 160-500 University Avenue, Toronto, Ontario, pose are derived from our subjective experiences under- Canada. taking scoping studies of varying sizes in the Authors’ contributions rehabilitation field, and we recognize that they may not DL and HC conceived of this paper. DL undertook the literature review represent the opinions of all scoping study authors. process. DL, HC and KO developed challenges and recommendations. All Other than our individual experiences with our own stu- authors drafted the manuscript. All authors read and approved the final manuscript. dies, we have not yet implemented the full framework recommendations. Hence, readers can determine how Authors’ information strongly to interpret and implement these recommenda- DL is a physical therapist and doctoral candidate in the School of Rehabilitation Science at McMaster University. HC is an occupational tions in their scoping study research. We invite others therapist and doctoral candidate in the School of Rehabilitation Science at to trial our recommendations and continue the process McMaster University. KO is a clinical epidemiologist, physical therapist, and of refining and improving this methodology. postdoctoral fellow in the School of Rehabilitation Science at McMaster Levac et al. Implementation Science 2010, 5:69 Page 9 of 9 http://www.implementationscience.com/content/5/1/69 University. She is also a Lecturer in the Department of Physical Therapy at the University of Toronto. Competing interests The authors declare that they have no competing interests. Received: 11 June 2010 Accepted: 20 September 2010 Published: 20 September 2010 References 1. Davis K, Drey N, Gould D: What are scoping studies? A review of the nursing literature. Int J Nurs Stud 2009, 46:1386-1400. 2. Anderson S, Allen P, Peckham S, Goodwin N: Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Sys 2008, 6:7. 3. Rumrill P, Fitzgerald S, Merchant W: Using scoping literature reviews as a means of understanding and interpreting existing literature. Work 2010, 35:399-404. 4. Grant M, Booth A: A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009, 26:91-108. 5. Brien S, Lorenzetti D, Lewis S, Kennedy J, Ghali W: Overview of a formal scoping review on health system report cards. Implement Sci 2010, 5:2. 6. Arksey H, O’Malley L: Scoping studies: Towards a Methodological Framework. Int J Soc Res Methodol 2005, 8:19-32. 7. O’Brien K, Wilkins A, Zack E, Solomon P: Scoping the field: identifying key research priorities in HIV and rehabilitation. AIDS Behav 2010, 14:448-458. 8. Levac D, Wishart L, Missiuna C, Wright V: The application of motor learning strategies within functionally based interventions for children with neuromotor conditions. Peds Phys Ther 2009, 21:345-355. 9. Colquhoun H, Letts L, Law M, MacDermid J, Missiuna C: A scoping review of the use of theory in studies of knowledge translation. Can J Occup Ther . 10. Ehrich K, Freeman G, Richards S, Robinson I, Shepperd S: How to do a scoping exercise: continuity of care. Res Pol Plan 2002, 20:25-29. 11. Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res 2005, 15:1277-1288. 12. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: time for a map? J Contin Educ Health Prof 2006, 26:13-24. 13. Schuster M, McGlynn E, Brook R: How good is the quality of healthcare in the United States? Milbank Q 2005, 83:843-895. 14. Mays N, Roberts E, Popay J: Synthesising research evidence. In Studying the organization and delivery of health services: research methods. Edited by: Fulop N, Allen P, Clarke A, Black N. London: Routledge; 2001:194. 15. Canadian Institutes of Health Research: Knowledge Translation. [http:// www.cihr-irsc.gc.ca/e/29418.html]. doi:10.1186/1748-5908-5-69 Cite this article as: Levac et al.: Scoping studies: advancing the methodology. Implementation Science 2010 5:69. Submit your next manuscript to BioMed Central and take full advantage of: • Convenient online submission • Thorough peer review • No space constraints or color figure charges • Immediate publication on acceptance • Inclusion in PubMed, CAS, Scopus and Google Scholar • Research which is freely available for redistribution Submit your manuscript at www.biomedcentral.com/submit

Journal

Implementation Science – Springer Journals

Published: Sep 20, 2010

There are no references for this article.