the difference between empirical and literature review

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 2 Chapter 3: What is Empirical Literature & Where can it be Found?

In Module 1, you read about the problem of pseudoscience. Here, we revisit the issue in addressing how to locate and assess scientific or empirical literature . In this chapter you will read about:

• distinguishing between what IS and IS NOT empirical literature
• how and where to locate empirical literature for understanding diverse populations, social work problems, and social phenomena.

Probably the most important take-home lesson from this chapter is that one source is not sufficient to being well-informed on a topic. It is important to locate multiple sources of information and to critically appraise the points of convergence and divergence in the information acquired from different sources. This is especially true in emerging and poorly understood topics, as well as in answering complex questions.

What Is Empirical Literature

Social workers often need to locate valid, reliable information concerning the dimensions of a population group or subgroup, a social work problem, or social phenomenon. They might also seek information about the way specific problems or resources are distributed among the populations encountered in professional practice. Or, social workers might be interested in finding out about the way that certain people experience an event or phenomenon. Empirical literature resources may provide answers to many of these types of social work questions. In addition, resources containing data regarding social indicators may also prove helpful. Social indicators are the “facts and figures” statistics that describe the social, economic, and psychological factors that have an impact on the well-being of a community or other population group.The United Nations (UN) and the World Health Organization (WHO) are examples of organizations that monitor social indicators at a global level: dimensions of population trends (size, composition, growth/loss), health status (physical, mental, behavioral, life expectancy, maternal and infant mortality, fertility/child-bearing, and diseases like HIV/AIDS), housing and quality of sanitation (water supply, waste disposal), education and literacy, and work/income/unemployment/economics, for example.

Three characteristics stand out in empirical literature compared to other types of information available on a topic of interest: systematic observation and methodology, objectivity, and transparency/replicability/reproducibility. Let’s look a little more closely at these three features.

Systematic Observation and Methodology. The hallmark of empiricism is “repeated or reinforced observation of the facts or phenomena” (Holosko, 2006, p. 6). In empirical literature, established research methodologies and procedures are systematically applied to answer the questions of interest.

Objectivity. Gathering “facts,” whatever they may be, drives the search for empirical evidence (Holosko, 2006). Authors of empirical literature are expected to report the facts as observed, whether or not these facts support the investigators’ original hypotheses. Research integrity demands that the information be provided in an objective manner, reducing sources of investigator bias to the greatest possible extent.

Transparency and Replicability/Reproducibility.   Empirical literature is reported in such a manner that other investigators understand precisely what was done and what was found in a particular research study—to the extent that they could replicate the study to determine whether the findings are reproduced when repeated. The outcomes of an original and replication study may differ, but a reader could easily interpret the methods and procedures leading to each study’s findings.

What is NOT Empirical Literature

By now, it is probably obvious to you that literature based on “evidence” that is not developed in a systematic, objective, transparent manner is not empirical literature. On one hand, non-empirical types of professional literature may have great significance to social workers. For example, social work scholars may produce articles that are clearly identified as describing a new intervention or program without evaluative evidence, critiquing a policy or practice, or offering a tentative, untested theory about a phenomenon. These resources are useful in educating ourselves about possible issues or concerns. But, even if they are informed by evidence, they are not empirical literature. Here is a list of several sources of information that do not meet the standard of being called empirical literature:

• your course instructor’s lectures
• political statements
• advertisements
• newspapers & magazines (journalism)
• television news reports & analyses (journalism)
• many websites, Facebook postings, Twitter tweets, and blog postings
• the introductory literature review in an empirical article

You may be surprised to see the last two included in this list. Like the other sources of information listed, these sources also might lead you to look for evidence. But, they are not themselves sources of evidence. They may summarize existing evidence, but in the process of summarizing (like your instructor’s lectures), information is transformed, modified, reduced, condensed, and otherwise manipulated in such a manner that you may not see the entire, objective story. These are called secondary sources, as opposed to the original, primary source of evidence. In relying solely on secondary sources, you sacrifice your own critical appraisal and thinking about the original work—you are “buying” someone else’s interpretation and opinion about the original work, rather than developing your own interpretation and opinion. What if they got it wrong? How would you know if you did not examine the primary source for yourself? Consider the following as an example of “getting it wrong” being perpetuated.

Example: Bullying and School Shootings . One result of the heavily publicized April 1999 school shooting incident at Columbine High School (Colorado), was a heavy emphasis placed on bullying as a causal factor in these incidents (Mears, Moon, & Thielo, 2017), “creating a powerful master narrative about school shootings” (Raitanen, Sandberg, & Oksanen, 2017, p. 3). Naturally, with an identified cause, a great deal of effort was devoted to anti-bullying campaigns and interventions for enhancing resilience among youth who experience bullying.  However important these strategies might be for promoting positive mental health, preventing poor mental health, and possibly preventing suicide among school-aged children and youth, it is a mistaken belief that this can prevent school shootings (Mears, Moon, & Thielo, 2017). Many times the accounts of the perpetrators having been bullied come from potentially inaccurate third-party accounts, rather than the perpetrators themselves; bullying was not involved in all instances of school shooting; a perpetrator’s perception of being bullied/persecuted are not necessarily accurate; many who experience severe bullying do not perpetrate these incidents; bullies are the least targeted shooting victims; perpetrators of the shooting incidents were often bullying others; and, bullying is only one of many important factors associated with perpetrating such an incident (Ioannou, Hammond, & Simpson, 2015; Mears, Moon, & Thielo, 2017; Newman &Fox, 2009; Raitanen, Sandberg, & Oksanen, 2017). While mass media reports deliver bullying as a means of explaining the inexplicable, the reality is not so simple: “The connection between bullying and school shootings is elusive” (Langman, 2014), and “the relationship between bullying and school shooting is, at best, tenuous” (Mears, Moon, & Thielo, 2017, p. 940). The point is, when a narrative becomes this publicly accepted, it is difficult to sort out truth and reality without going back to original sources of information and evidence.

What May or May Not Be Empirical Literature: Literature Reviews

Investigators typically engage in a review of existing literature as they develop their own research studies. The review informs them about where knowledge gaps exist, methods previously employed by other scholars, limitations of prior work, and previous scholars’ recommendations for directing future research. These reviews may appear as a published article, without new study data being reported (see Fields, Anderson, & Dabelko-Schoeny, 2014 for example). Or, the literature review may appear in the introduction to their own empirical study report. These literature reviews are not considered to be empirical evidence sources themselves, although they may be based on empirical evidence sources. One reason is that the authors of a literature review may or may not have engaged in a systematic search process, identifying a full, rich, multi-sided pool of evidence reports.

There is, however, a type of review that applies systematic methods and is, therefore, considered to be more strongly rooted in evidence: the systematic review .

Systematic review of literature. A systematic reviewis a type of literature report where established methods have been systematically applied, objectively, in locating and synthesizing a body of literature. The systematic review report is characterized by a great deal of transparency about the methods used and the decisions made in the review process, and are replicable. Thus, it meets the criteria for empirical literature: systematic observation and methodology, objectivity, and transparency/reproducibility. We will work a great deal more with systematic reviews in the second course, SWK 3402, since they are important tools for understanding interventions. They are somewhat less common, but not unheard of, in helping us understand diverse populations, social work problems, and social phenomena.

Locating Empirical Evidence

Social workers have available a wide array of tools and resources for locating empirical evidence in the literature. These can be organized into four general categories.

Journal Articles. A number of professional journals publish articles where investigators report on the results of their empirical studies. However, it is important to know how to distinguish between empirical and non-empirical manuscripts in these journals. A key indicator, though not the only one, involves a peer review process . Many professional journals require that manuscripts undergo a process of peer review before they are accepted for publication. This means that the authors’ work is shared with scholars who provide feedback to the journal editor as to the quality of the submitted manuscript. The editor then makes a decision based on the reviewers’ feedback:

• Accept as is
• Accept with minor revisions
• Request that a revision be resubmitted (no assurance of acceptance)

When a “revise and resubmit” decision is made, the piece will go back through the review process to determine if it is now acceptable for publication and that all of the reviewers’ concerns have been adequately addressed. Editors may also reject a manuscript because it is a poor fit for the journal, based on its mission and audience, rather than sending it for review consideration.

Indicators of journal relevance. Various journals are not equally relevant to every type of question being asked of the literature. Journals may overlap to a great extent in terms of the topics they might cover; in other words, a topic might appear in multiple different journals, depending on how the topic was being addressed. For example, articles that might help answer a question about the relationship between community poverty and violence exposure might appear in several different journals, some with a focus on poverty, others with a focus on violence, and still others on community development or public health. Journal titles are sometimes a good starting point but may not give a broad enough picture of what they cover in their contents.

In focusing a literature search, it also helps to review a journal’s mission and target audience. For example, at least four different journals focus specifically on poverty:

• Journal of Children & Poverty
• Journal of Poverty
• Journal of Poverty and Social Justice
• Poverty & Public Policy

Let’s look at an example using the Journal of Poverty and Social Justice . Information about this journal is located on the journal’s webpage: http://policy.bristoluniversitypress.co.uk/journals/journal-of-poverty-and-social-justice . In the section headed “About the Journal” you can see that it is an internationally focused research journal, and that it addresses social justice issues in addition to poverty alone. The research articles are peer-reviewed (there appear to be non-empirical discussions published, as well). These descriptions about a journal are almost always available, sometimes listed as “scope” or “mission.” These descriptions also indicate the sponsorship of the journal—sponsorship may be institutional (a particular university or agency, such as Smith College Studies in Social Work ), a professional organization, such as the Council on Social Work Education (CSWE) or the National Association of Social Work (NASW), or a publishing company (e.g., Taylor & Frances, Wiley, or Sage).

Indicators of journal caliber.  Despite engaging in a peer review process, not all journals are equally rigorous. Some journals have very high rejection rates, meaning that many submitted manuscripts are rejected; others have fairly high acceptance rates, meaning that relatively few manuscripts are rejected. This is not necessarily the best indicator of quality, however, since newer journals may not be sufficiently familiar to authors with high quality manuscripts and some journals are very specific in terms of what they publish. Another index that is sometimes used is the journal’s impact factor . Impact factor is a quantitative number indicative of how often articles published in the journal are cited in the reference list of other journal articles—the statistic is calculated as the number of times on average each article published in a particular year were cited divided by the number of articles published (the number that could be cited). For example, the impact factor for the Journal of Poverty and Social Justice in our list above was 0.70 in 2017, and for the Journal of Poverty was 0.30. These are relatively low figures compared to a journal like the New England Journal of Medicine with an impact factor of 59.56! This means that articles published in that journal were, on average, cited more than 59 times in the next year or two.

Impact factors are not necessarily the best indicator of caliber, however, since many strong journals are geared toward practitioners rather than scholars, so they are less likely to be cited by other scholars but may have a large impact on a large readership. This may be the case for a journal like the one titled Social Work, the official journal of the National Association of Social Workers. It is distributed free to all members: over 120,000 practitioners, educators, and students of social work world-wide. The journal has a recent impact factor of.790. The journals with social work relevant content have impact factors in the range of 1.0 to 3.0 according to Scimago Journal & Country Rank (SJR), particularly when they are interdisciplinary journals (for example, Child Development , Journal of Marriage and Family , Child Abuse and Neglect , Child Maltreatmen t, Social Service Review , and British Journal of Social Work ). Once upon a time, a reader could locate different indexes comparing the “quality” of social work-related journals. However, the concept of “quality” is difficult to systematically define. These indexes have mostly been replaced by impact ratings, which are not necessarily the best, most robust indicators on which to rely in assessing journal quality. For example, new journals addressing cutting edge topics have not been around long enough to have been evaluated using this particular tool, and it takes a few years for articles to begin to be cited in other, later publications.

Beware of pseudo-, illegitimate, misleading, deceptive, and suspicious journals . Another side effect of living in the Age of Information is that almost anyone can circulate almost anything and call it whatever they wish. This goes for “journal” publications, as well. With the advent of open-access publishing in recent years (electronic resources available without subscription), we have seen an explosion of what are called predatory or junk journals . These are publications calling themselves journals, often with titles very similar to legitimate publications and often with fake editorial boards. These “publications” lack the integrity of legitimate journals. This caution is reminiscent of the discussions earlier in the course about pseudoscience and “snake oil” sales. The predatory nature of many apparent information dissemination outlets has to do with how scientists and scholars may be fooled into submitting their work, often paying to have their work peer-reviewed and published. There exists a “thriving black-market economy of publishing scams,” and at least two “journal blacklists” exist to help identify and avoid these scam journals (Anderson, 2017).

This issue is important to information consumers, because it creates a challenge in terms of identifying legitimate sources and publications. The challenge is particularly important to address when information from on-line, open-access journals is being considered. Open-access is not necessarily a poor choice—legitimate scientists may pay sizeable fees to legitimate publishers to make their work freely available and accessible as open-access resources. On-line access is also not necessarily a poor choice—legitimate publishers often make articles available on-line to provide timely access to the content, especially when publishing the article in hard copy will be delayed by months or even a year or more. On the other hand, stating that a journal engages in a peer-review process is no guarantee of quality—this claim may or may not be truthful. Pseudo- and junk journals may engage in some quality control practices, but may lack attention to important quality control processes, such as managing conflict of interest, reviewing content for objectivity or quality of the research conducted, or otherwise failing to adhere to industry standards (Laine & Winker, 2017).

One resource designed to assist with the process of deciphering legitimacy is the Directory of Open Access Journals (DOAJ). The DOAJ is not a comprehensive listing of all possible legitimate open-access journals, and does not guarantee quality, but it does help identify legitimate sources of information that are openly accessible and meet basic legitimacy criteria. It also is about open-access journals, not the many journals published in hard copy.

An additional caution: Search for article corrections. Despite all of the careful manuscript review and editing, sometimes an error appears in a published article. Most journals have a practice of publishing corrections in future issues. When you locate an article, it is helpful to also search for updates. Here is an example where data presented in an article’s original tables were erroneous, and a correction appeared in a later issue.

• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2017). A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 12(8): e0181722. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5558917/
• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2018).Correction—A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 13(3): e0193937.  http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193937

Search Tools. In this age of information, it is all too easy to find items—the problem lies in sifting, sorting, and managing the vast numbers of items that can be found. For example, a simple Google® search for the topic “community poverty and violence” resulted in about 15,600,000 results! As a means of simplifying the process of searching for journal articles on a specific topic, a variety of helpful tools have emerged. One type of search tool has previously applied a filtering process for you: abstracting and indexing databases . These resources provide the user with the results of a search to which records have already passed through one or more filters. For example, PsycINFO is managed by the American Psychological Association and is devoted to peer-reviewed literature in behavioral science. It contains almost 4.5 million records and is growing every month. However, it may not be available to users who are not affiliated with a university library. Conducting a basic search for our topic of “community poverty and violence” in PsychINFO returned 1,119 articles. Still a large number, but far more manageable. Additional filters can be applied, such as limiting the range in publication dates, selecting only peer reviewed items, limiting the language of the published piece (English only, for example), and specified types of documents (either chapters, dissertations, or journal articles only, for example). Adding the filters for English, peer-reviewed journal articles published between 2010 and 2017 resulted in 346 documents being identified.

Just as was the case with journals, not all abstracting and indexing databases are equivalent. There may be overlap between them, but none is guaranteed to identify all relevant pieces of literature. Here are some examples to consider, depending on the nature of the questions asked of the literature:

• Academic Search Complete—multidisciplinary index of 9,300 peer-reviewed journals
• AgeLine—multidisciplinary index of aging-related content for over 600 journals
• Campbell Collaboration—systematic reviews in education, crime and justice, social welfare, international development
• Google Scholar—broad search tool for scholarly literature across many disciplines
• MEDLINE/ PubMed—National Library of medicine, access to over 15 million citations
• Oxford Bibliographies—annotated bibliographies, each is discipline specific (e.g., psychology, childhood studies, criminology, social work, sociology)
• PsycINFO/PsycLIT—international literature on material relevant to psychology and related disciplines
• SocINDEX—publications in sociology
• Social Sciences Abstracts—multiple disciplines
• Social Work Abstracts—many areas of social work are covered
• Web of Science—a “meta” search tool that searches other search tools, multiple disciplines

Placing our search for information about “community violence and poverty” into the Social Work Abstracts tool with no additional filters resulted in a manageable 54-item list. Finally, abstracting and indexing databases are another way to determine journal legitimacy: if a journal is indexed in a one of these systems, it is likely a legitimate journal. However, the converse is not necessarily true: if a journal is not indexed does not mean it is an illegitimate or pseudo-journal.

Government Sources. A great deal of information is gathered, analyzed, and disseminated by various governmental branches at the international, national, state, regional, county, and city level. Searching websites that end in.gov is one way to identify this type of information, often presented in articles, news briefs, and statistical reports. These government sources gather information in two ways: they fund external investigations through grants and contracts and they conduct research internally, through their own investigators. Here are some examples to consider, depending on the nature of the topic for which information is sought:

• Agency for Healthcare Research and Quality (AHRQ) at https://www.ahrq.gov/
• Bureau of Justice Statistics (BJS) at https://www.bjs.gov/
• Census Bureau at https://www.census.gov
• Morbidity and Mortality Weekly Report of the CDC (MMWR-CDC) at https://www.cdc.gov/mmwr/index.html
• Child Welfare Information Gateway at https://www.childwelfare.gov
• Children’s Bureau/Administration for Children & Families at https://www.acf.hhs.gov
• Forum on Child and Family Statistics at https://www.childstats.gov
• National Institutes of Health (NIH) at https://www.nih.gov , including (not limited to):
• National Institute on Aging (NIA at https://www.nia.nih.gov
• National Institute on Alcohol Abuse and Alcoholism (NIAAA) at https://www.niaaa.nih.gov
• National Institute of Child Health and Human Development (NICHD) at https://www.nichd.nih.gov
• National Institute on Drug Abuse (NIDA) at https://www.nida.nih.gov
• National Institute of Environmental Health Sciences at https://www.niehs.nih.gov
• National Institute of Mental Health (NIMH) at https://www.nimh.nih.gov
• National Institute on Minority Health and Health Disparities at https://www.nimhd.nih.gov
• National Institute of Justice (NIJ) at https://www.nij.gov
• Substance Abuse and Mental Health Services Administration (SAMHSA) at https://www.samhsa.gov/
• United States Agency for International Development at https://usaid.gov

Each state and many counties or cities have similar data sources and analysis reports available, such as Ohio Department of Health at https://www.odh.ohio.gov/healthstats/dataandstats.aspx and Franklin County at https://statisticalatlas.com/county/Ohio/Franklin-County/Overview . Data are available from international/global resources (e.g., United Nations and World Health Organization), as well.

Other Sources. The Health and Medicine Division (HMD) of the National Academies—previously the Institute of Medicine (IOM)—is a nonprofit institution that aims to provide government and private sector policy and other decision makers with objective analysis and advice for making informed health decisions. For example, in 2018 they produced reports on topics in substance use and mental health concerning the intersection of opioid use disorder and infectious disease,  the legal implications of emerging neurotechnologies, and a global agenda concerning the identification and prevention of violence (see http://www.nationalacademies.org/hmd/Global/Topics/Substance-Abuse-Mental-Health.aspx ). The exciting aspect of this resource is that it addresses many topics that are current concerns because they are hoping to help inform emerging policy. The caution to consider with this resource is the evidence is often still emerging, as well.

Numerous “think tank” organizations exist, each with a specific mission. For example, the Rand Corporation is a nonprofit organization offering research and analysis to address global issues since 1948. The institution’s mission is to help improve policy and decision making “to help individuals, families, and communities throughout the world be safer and more secure, healthier and more prosperous,” addressing issues of energy, education, health care, justice, the environment, international affairs, and national security (https://www.rand.org/about/history.html). And, for example, the Robert Woods Johnson Foundation is a philanthropic organization supporting research and research dissemination concerning health issues facing the United States. The foundation works to build a culture of health across systems of care (not only medical care) and communities (https://www.rwjf.org).

While many of these have a great deal of helpful evidence to share, they also may have a strong political bias. Objectivity is often lacking in what information these organizations provide: they provide evidence to support certain points of view. That is their purpose—to provide ideas on specific problems, many of which have a political component. Think tanks “are constantly researching solutions to a variety of the world’s problems, and arguing, advocating, and lobbying for policy changes at local, state, and federal levels” (quoted from https://thebestschools.org/features/most-influential-think-tanks/ ). Helpful information about what this one source identified as the 50 most influential U.S. think tanks includes identifying each think tank’s political orientation. For example, The Heritage Foundation is identified as conservative, whereas Human Rights Watch is identified as liberal.

While not the same as think tanks, many mission-driven organizations also sponsor or report on research, as well. For example, the National Association for Children of Alcoholics (NACOA) in the United States is a registered nonprofit organization. Its mission, along with other partnering organizations, private-sector groups, and federal agencies, is to promote policy and program development in research, prevention and treatment to provide information to, for, and about children of alcoholics (of all ages). Based on this mission, the organization supports knowledge development and information gathering on the topic and disseminates information that serves the needs of this population. While this is a worthwhile mission, there is no guarantee that the information meets the criteria for evidence with which we have been working. Evidence reported by think tank and mission-driven sources must be utilized with a great deal of caution and critical analysis!

In many instances an empirical report has not appeared in the published literature, but in the form of a technical or final report to the agency or program providing the funding for the research that was conducted. One such example is presented by a team of investigators funded by the National Institute of Justice to evaluate a program for training professionals to collect strong forensic evidence in instances of sexual assault (Patterson, Resko, Pierce-Weeks, & Campbell, 2014): https://www.ncjrs.gov/pdffiles1/nij/grants/247081.pdf . Investigators may serve in the capacity of consultant to agencies, programs, or institutions, and provide empirical evidence to inform activities and planning. One such example is presented by Maguire-Jack (2014) as a report to a state’s child maltreatment prevention board: https://preventionboard.wi.gov/Documents/InvestmentInPreventionPrograming_Final.pdf .

When Direct Answers to Questions Cannot Be Found. Sometimes social workers are interested in finding answers to complex questions or questions related to an emerging, not-yet-understood topic. This does not mean giving up on empirical literature. Instead, it requires a bit of creativity in approaching the literature. A Venn diagram might help explain this process. Consider a scenario where a social worker wishes to locate literature to answer a question concerning issues of intersectionality. Intersectionality is a social justice term applied to situations where multiple categorizations or classifications come together to create overlapping, interconnected, or multiplied disadvantage. For example, women with a substance use disorder and who have been incarcerated face a triple threat in terms of successful treatment for a substance use disorder: intersectionality exists between being a woman, having a substance use disorder, and having been in jail or prison. After searching the literature, little or no empirical evidence might have been located on this specific triple-threat topic. Instead, the social worker will need to seek literature on each of the threats individually, and possibly will find literature on pairs of topics (see Figure 3-1). There exists some literature about women’s outcomes for treatment of a substance use disorder (a), some literature about women during and following incarceration (b), and some literature about substance use disorders and incarceration (c). Despite not having a direct line on the center of the intersecting spheres of literature (d), the social worker can develop at least a partial picture based on the overlapping literatures.

Figure 3-1. Venn diagram of intersecting literature sets.

Take a moment to complete the following activity. For each statement about empirical literature, decide if it is true or false.

Social Work 3401 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

• 66 Ogoja Road, Abakaliki, Ebonyi State 23480 NG.
• Sun - Fri 24Hours Saturday CLOSED
• support [@] writersking.com
• +23480-6075-5653 Hot Line

• Data Collection/Analysis
• Hire Proposal Writers
• Hire Essay Writers
• Hire Paper Writers
• Proofreading Services
• Thesis/Dissertation Writers
• Virtual Supervisor
• Turnitin Checker
• Book Chapter Writer
• Hire Business Writing Services
• Hire Blog Writers
• Writers King TV
• Proposal Sample
• Chapter 1-3 Sample
• Term Paper Sample
• Report Assignment Sample
• Course work Sample
• Payment Options
• Privacy Policy
• Terms of Service/Use

• Business Guide
• Academic Writing Guide
• General Writing Guide
• Research News
• Writing Paper Samples
• February 19, 2022
• Posted by: Chimnecherem Eke
• Category: Academic Writing Guide

Differences between Empirical Review and Literature Review

As a researcher, you might wonder about the difference between an Empirical Review and Literature Review. If you are going to write an essay or article, you must first figure out what your topic is. The subject determines the flow of the writing, the material provided, and every other element.

Content Outline

Get your seat closer; let’s get to what you need to know about Empirical Review and Literature Review and their notable differences.

Students often accost empirical studies and literature reviews while preparing a research paper . It’s critical to know the difference between the two in order to craft a solid piece of writing. Both articles are responsible for presenting the facts, but their strategies differ.

Students are often perplexed when asked what empirical research is. The following is an explanation of the differences between a systematic review and a literature review:

Definition of Empirical review -Empirical Review and Literature Review

An empirical literature review, also known as a systematic literature review, analyzes previous empirical studies in order to provide an answer to a specific research topic.

Rather than drawing information from theories or beliefs, empirical research relies on observations and measurements to arrive at conclusions. To address specific research inquiries, it could involve making a list of people, behaviours, or events that are being researched.

In some cases, reviews of studies that involve experiments are used in empirical reviews to generate findings based on experience that may be seen directly or indirectly. Most of the time, the analysis entails quantifying the data and drawing conclusions.

The goal is to provide data that can be quantified using established scientific methods . Research reviewers thoroughly examine all findings of other authors before drawing any conclusions in an essay or paper.

As a result of carefully planned and monitored observations, the experiment is carried out, and the resulting conclusion is rigorously monitored. In contrast to other types of literature reviews, the focus here is on the most recent results of the experimental studies as it is now being conducted. A hypothesis may also be a forecast of a previously presented theory based on prior material.

Empirical evidence refers to data gathered via testing or observation in this context. These data are collected and analyzed by scientists.

For example, an empirical review could involve reviewing the study of another researcher on a group of listeners exposed to upbeat music or a work on learning and improvisation that examines other studies on work that theorizes about the educational value of improvisational principles and practices, such as Viola Spolin and Keith Johnstone ‘s writings in which they present their beliefs, impressions, ideas, and theories about those.

Defining Literature Review -Empirical Review and Literature Review

The literature review , instead of an empirical review, necessitates reading several related studies. Other theoretical sources are used to compile the facts and information included in this piece. The accumulation of all literary works may lead to new deductions. Information and hypotheses, on the other hand, have already been developed.

To generate cohesive findings, a literature review compiles all necessary data. No new theories can be developed since no experimental work can be done.

There is an important function for the literature review in uncovering defining, and clarifying key ideas that will be utilized throughout the empirical parts of the paper argues.

A well-written review article may shed light on the current state of knowledge, explain apparent inconsistencies, pinpoint areas in need of more study, and even help to forge a consensus where none previously existed. A well-written review may also assist you in your professional life. Reviews aid in recognition and advancement due to their high citation frequency.

Selecting the type of review to conduct -Empirical Review and Literature Review

College students are often required to write several papers as part of their studies. When a student does a literature review, he or she attempts to use the written word to support or refute an idea or hypothesis. He or she may test a theory or try to find an answer to a specific issue based on already-known information.

An empirical review is a piece of writing based on a study that was done purely for the purpose of publishing it. Calibrated instruments are used to conduct the experiment in a scientifically controlled way.

In this age of AI, research has become more interesting as you can use AI for your research study to make reviewing literature easier (Thesis, Dissertation, Research paper, among others). AI tools like SciSpace literature review can help you compare, contrast, and analyze research papers more efficiently.

It would be best to start writing as soon as you finish the experiment. During the study, observations should be recorded methodically. This aids in developing coherence, which is more easily understood even in the future. Additionally, starting the writing process early allows for more time for revision and results in higher-quality work.

Because experiments may take some time to produce the desired results, this is especially important for empirical studies. Leaving the writing to the last minute and beginning it when the deadline is nearing will just add to the stress and complexity of the process. This interferes with the job and lowers the quality. As a result, staying on top of your job helps your paper and your personal life grow.

Almost every research article includes a review of the literature. An empirical study must first be established inside an accepted theory before they can publish their findings. In other words, before we get into our methodology and research questions, we’ll go through what’s been done previously and how the variables we want to investigate fit into the theories and frameworks of our research field.

An empirical literature review, also known as a systematic literature review, analyzes previous empirical studies in order to provide an answer to a specific research topic. Randomized controlled trials are the most common kind of empirical study.

Both of these tasks are similar in that they need to review previous work on the topic. The empirical literature review, on the other hand, seeks to address a particular empirical issue by analyzing data. The theoretical literature review serves primarily to place your research within a broader framework. A theoretical review will be included in systematic empirical reviews to help researchers understand why a specific research topic is worth investigating.

Thank you for your time, we hope you got value reading Differences between Empirical Review and Literature Review?

Frequently Asked Questions

What is an example of an empirical review.

An empirical review is an academic review that focuses on summarizing and analyzing the findings of empirical research studies. Empirical research involves collecting and analysing primary data through experiments, surveys, observations, or any other data collection method(s).

An example of an empirical review would be a critical examination of several recent scientific studies that have investigated the effectiveness of a specific drug in treating a particular medical condition. The empirical review would include the study objectives, a summary of the research methods, key findings, and a critical evaluation of the research design, data collection, and statistical analysis methods employed in those studies.

What is the purpose of a literature review in empirical research studies?

A literature review in empirical research studies provides a comprehensive and systematic overview of existing published research and scholarly works related to the topic under investigation. This serves several important purposes:

Contextualization : It helps researchers place their study within the context of existing knowledge and identify gaps or areas where further research is needed.

Identification of Theoretical Frameworks : A literature review aids in identifying relevant theories and concepts that can inform the research design and data analysis in empirical studies.

Methodological Guidance: It offers insights into prior studies’ methodologies and research methods, helping researchers make informed decisions about their own data collection and analysis techniques.

Hypothesis Development: It can assist in formulating research hypotheses and research questions based on the existing literature.

Validation of Research Question: By reviewing the literature, researchers can ensure that their research questions or hypotheses have not been adequately addressed in prior studies.

Identification of Variables: It helps identify key variables and factors examined in previous research, which can guide the selection of variables in the new study.

In summary, a literature review in empirical research serves as the foundation upon which a new empirical study is built, helping researchers gain a deep understanding of the existing knowledge in their field, refine their research questions, and make informed decisions about the research design and methods they will employ.

Some quotations on code mixing and code switching base on empirical review.

Leave a Reply

Find Us Today

Writers King LTD,  Akachukwu Plaza, 

66 Ogoja Road Abakaliki,  Ebonyi State, 

480101 Nigeria

Phone: 0806-075-5653

• Website: https://writersking.com/
• Email: support {@} writersking.com
• +2348060755653

Quick Links

Writing guide.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism. Run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 5, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Meriam Library

SWRK 330 - Social Work Research Methods

• Literature Reviews and Empirical Research
• Databases and Search Tips
• Article Citations
• Scholarly Journal Evaulation
• Statistical Sources
• Books and eBooks

What is a Literature Review?

Empirical research.

• Annotated Bibliographies

A literature review  summarizes and discusses previous publications  on a topic.

It should also:

explore past research and its strengths and weaknesses.

be used to validate the target and methods you have chosen for your proposed research.

consist of books and scholarly journals that provide research examples of populations or settings similar to your own, as well as community resources to document the need for your proposed research.

The literature review does not present new  primary  scholarship. 

be completed in the correct citation format requested by your professor  (see the  C itations Tab)

Access Purdue  OWL's Social Work Literature Review Guidelines here .  

Empirical Research  is  research  that is based on experimentation or observation, i.e. Evidence. Such  research  is often conducted to answer a specific question or to test a hypothesis (educated guess).

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

These are some key features to look for when identifying empirical research.

NOTE:  Not all of these features will be in every empirical research article, some may be excluded, use this only as a guide.

• Statement of methodology
• Research questions are clear and measurable
• Individuals, group, subjects which are being studied are identified/defined
• Data is presented regarding the findings
• Controls or instruments such as surveys or tests were conducted
• There is a literature review
• There is discussion of the results included
• Citations/references are included

See also Empirical Research Guide

• << Previous: Citations
• Next: Annotated Bibliographies >>
• Last Updated: Feb 6, 2024 8:38 AM
• URL: https://libguides.csuchico.edu/SWRK330

Meriam Library | CSU, Chico

Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions

• February 2009
• Human Resource Development Review 8(1):120-130
• 8(1):120-130

• Florida International University

• National Research University Higher School of Economics

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Jakub Spurný
• Kamila Procházková
• Zamhuri Rachman

• Md Khairul Islam
• Md Rakibul Hasan Khan
• Boitumelo Joy Molato

• Magdalena P. Koen
• Molekodi Jacob Matsipane
• Craig L. Pearce
• Luis E. Orozco
• Judy Kraatz

• Sharan B. Merriam
• Edward F. McQuarrie
• Catherine Marshall

• J.W. Cresswell
• Vicki L. Plano Clark
• M.L. Gutmann
• William E. Hanson
• Stacey L. Hogan
• Ronald M. Cervero
• Arthur L. Wilson
• ADULT EDUC QUART

• D. Elaine Archie-Booker
• Christine A. Langone
• Elizabeth Watson
• Kathryn S. O’Neill

• Gary L. May
• Richard J. Torraco
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Getting Started
• Topic Overviews
• Other Databases
• Tests and Measures

What is an Empirical Study?

Literature review.

• Organize Your Sources
• NEW: LibKey Nomad

An empirical article reports the findings of a study conducted by the authors and uses data gathered from an experiment or observation. An empirical study is verifiable and "based on facts, systematic observation, or experiment, rather than theory or general philosophical principle" ( APA Databases Methodology Field Values ).  In other words, it tells the story of a research conducted, doing it in great detail. The study may utilize quantitative research methods to produce numerical data and seek to find a causal relationship between two or more variables. Conversely, it may use qualitative research methods, which involves collecting non-numerical data to analyze concepts, opinions, or experiences.

Key parts of an empirical article:

• Abstract  - Provides a brief overview of the research.
• Introduction  - The introduction provides a review of previous research on the topic and states the hypothesis. 
• Methods  - The methods area describes how the research was conducted, identifies the design of the study, the participants, and any measurements that were taken during the study.
• Results  - The results section describes the outcome of the study. 
• Discussion (or conclusion)  - The discussion section addresses the researchers' interpretations of their study and any future implications from their findings.
• References  - A list of works that were cited in the study.
• What is a Lit. Review?
• Purpose of a Lit. Review
• Limitations
• Non-Empirical Research
• Useful Links/Additional Info

A review of the published resources related to a specific issue, area of research, or specific theory. It provides a summary, description, and critical evaluation of each resource.

A literature review:

•  Synthesizes and places into context the research and scholarly literature relevant to the topic.
• Maps the different approaches to a given question and reveals patterns.
• Forms the foundation for subsequent research 
• Justifies the significance of the new investigation.
• Contains the most pertinent studies and points to important past and current research and practices.

A Lit. Review provides background and context; it shows how your research will contribute to the field. 

There are generally five parts to a literature review:

• Introduction
• Bibliography

A literature review should: 

• Provide a comprehensive and updated review of the literature
• Explain why this review has taken place
• Articulate a position or hypothesis
• Acknowledge and account for conflicting and corroborating points of view

 Add / Reorder  

A lit. review's purpose is to offer an overview of the significant works published on a topic. It can be written as an introduction to a study in order to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

It could be a separate work (a research article on its own) that:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Some limitations of a literature review include:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. Future developments could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to ensure that all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Non-Empirical Research articles focus more on theories, methods and their implications for research. Non-Empirical Research can include comprehensive reviews and articles that focus on methodology. They rely on empirical research literature as well but does not need to be essentially data-driven.

Write a Literature Review (UCSC)

• Literature Review (Purdue)
• Overview: Lit Reviews (UNC)
• Review of Literature (UW-Madison)
• << Previous: Tests and Measures
• Next: Organize Your Sources >>
• Last Updated: Aug 2, 2024 4:33 PM
• URL: https://libguides.macalester.edu/psyc

Literature Review: 3 Essential Ingredients

The theoretical framework, empirical research and research gap

By: Derek Jansen (MBA) | Reviewer: Eunice Rautenbach (DTech) | July 2023

Writing a comprehensive but concise literature review is no simple task. There’s a lot of ground to cover and it can be challenging to figure out what’s important and what’s not. In this post, we’ll unpack three essential ingredients that need to be woven into your literature review to lay a rock-solid foundation for your study.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Essential Ingredients

• Ingredients vs structure
• The theoretical framework (foundation of theory)
• The empirical research
• The research gap
• Summary & key takeaways

Ingredients vs Structure

As a starting point, it’s important to clarify that the three ingredients we’ll cover in this video are things that need to feature within your literature review, as opposed to a set structure for your chapter . In other words, there are different ways you can weave these three ingredients into your literature review. Regardless of which structure you opt for, each of the three components will make an appearance in some shape or form. If you’re keen to learn more about structural options, we’ve got a dedicated post about that here .

1. The Theoretical Framework

Let’s kick off with the first essential ingredient – that is the theoretical framework , also called the foundation of theory . 

The foundation of theory, as the name suggests, is where you’ll lay down the foundational building blocks for your literature review so that your reader can get a clear idea of the core concepts, theories and assumptions (in relation to your research aims and questions) that will guide your study. Note that this is not the same as a conceptual framework .

Typically you’ll cover a few things within the theoretical framework:

Firstly, you’ll need to clearly define the key constructs and variables that will feature within your study. In many cases, any given term can have multiple different definitions or interpretations – for example, different people will define the concept of “integrity” in different ways. This variation in interpretation can, of course, wreak havoc on how your study is understood. So, this section is where you’ll pin down what exactly you mean when you refer to X, Y or Z in your study, as well as why you chose that specific definition. It’s also a good idea to state any assumptions that are inherent in these definitions and why these are acceptable, given the purpose of your study.

Related to this, the second thing you’ll need to cover in your theoretical framework is the relationships between these variables and/or constructs . For example, how does one variable potentially affect another variable – does A have an impact on B, B on A, and so on? In other words, you want to connect the dots between the different “things” of interest that you’ll be exploring in your study. Note that you only need to focus on the key items of interest here (i.e. those most central to your research aims and questions) – not every possible construct or variable.

Lastly, and very importantly, you need to discuss the existing theories that are relevant to your research aims and research questions . For example, if you’re investigating the uptake/adoption of a certain application or software, you might discuss Davis’ Technology Acceptance Model and unpack what it has to say about the factors that influence technology adoption. More importantly, though, you need to explain how this impacts your expectations about what you will find in your own study . In other words, your theoretical framework should reveal some insights about what answers you might expect to find to your research questions .

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of the limited-time offer (60% off the standard price).

Need a helping hand?

2. The Empirical Research

Onto the second essential ingredient, which is  empirical research . This section is where you’ll present a critical discussion of the existing empirical research that is relevant to your research aims and questions.

But what exactly is empirical research?

Simply put, empirical research includes any study that involves actual data collection and analysis , whether that’s qualitative data, quantitative data, or a mix of both . This contrasts against purely theoretical literature (the previous ingredient), which draws its conclusions based exclusively on logic and reason , as opposed to an analysis of real-world data.

In other words, theoretical literature provides a prediction or expectation of what one might find based on reason and logic, whereas empirical research tests the accuracy of those predictions using actual real-world data . This reflects the broader process of knowledge creation – in other words, first developing a theory and then testing it out in the field.

Long story short, the second essential ingredient of a high-quality literature review is a critical discussion of the existing empirical research . Here, it’s important to go beyond description . You’ll need to present a critical analysis that addresses some (if not all) of the following questions:

• What have different studies found in relation to your research questions ?
• What contexts have (and haven’t been covered)? For example, certain countries, cities, cultures, etc.
• Are the findings across the studies similar or is there a lot of variation ? If so, why might this be the case?
• What sorts of research methodologies have been used and how could these help me develop my own methodology?
• What were the noteworthy limitations of these studies?

Simply put, your task here is to present a synthesis of what’s been done (and found) within the empirical research, so that you can clearly assess the current state of knowledge and identify potential research gaps , which leads us to our third essential ingredient.

The Research Gap

The third essential ingredient of a high-quality literature review is a discussion of the research gap (or gaps).

But what exactly is a research gap?

Simply put, a research gap is any unaddressed or inadequately explored area within the existing body of academic knowledge. In other words, a research gap emerges whenever there’s still some uncertainty regarding a certain topic or question.

For example, it might be the case that there are mixed findings regarding the relationship between two variables (e.g., job performance and work-from-home policies). Similarly, there might be a lack of research regarding the impact of a specific new technology on people’s mental health. On the other end of the spectrum, there might be a wealth of research regarding a certain topic within one country (say the US), but very little research on that same topic in a different social context (say, China).

These are just random examples, but as you can see, research gaps can emerge from many different places. What’s important to understand is that the research gap (or gaps) needs to emerge from your previous discussion of the theoretical and empirical literature . In other words, your discussion in those sections needs to start laying the foundation for the research gap.

For example, when discussing empirical research, you might mention that most studies have focused on a certain context , yet very few (or none) have focused on another context, and there’s reason to believe that findings may differ. Or you might highlight how there’s a fair deal of mixed findings and disagreement regarding a certain matter. In other words, you want to start laying a little breadcrumb trail in those sections so that your discussion of the research gap is firmly rooted in the rest of the literature review.

But why does all of this matter?

Well, the research gap should serve as the core justification for your study . Through your literature review, you’ll show what gaps exist in the current body of knowledge, and then your study will then attempt to fill (or contribute towards filling) one of those gaps. In other words, you’re first explaining what the problem is (some sort of gap) and then proposing how you’ll solve it.

Key Takeaways

To recap, the three ingredients that need to be mixed into your literature review are:

• The foundation of theory or theoretical framework
• The empirical or evidence-based research

As we mentioned earlier, these are components of a literature review and not (necessarily) a structure for your literature review chapter. Of course, you can structure your chapter in a way that reflects these three components (in fact, in some cases that works very well), but it’s certainly not the only option. The right structure will vary from study to study , depending on various factors.

If you’d like to get hands-on help developing your literature review, be sure to check out our private coaching service , where we hold your hand through the entire research journey, step by step. 

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

very good , as the first writer of the thesis i will need ur advise . please give me a piece of idea on topic -impact of national standardized exam on students learning engagement . Thank you .

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

PSYC 200 Lab in Experimental Methods (Atlanta)

• Find Research Articles

Empirical vs. Review Articles

How to recognize empirical journal articles, scholarly vs. non-scholarly sources.

• Cite Sources
• Find Tests & Measures
• Find Research Methods Materials
• Post-Library Lab Activity on Finding Tests and Measures
• Find Books, E-books, and Films

Psychology Librarian

Know the difference between empirical and review articles.

Empirical article An empirical (research) article reports methods and findings of an original research study conducted by the authors of the article.  

Literature Review article A review article or "literature review" discusses past research studies on a given topic.

Definition of an empirical study:  An empirical research article reports the results of a study that uses data derived from actual observation or experimentation. Empirical research articles are examples of primary research.

Parts of a standard empirical research article:  (articles will not necessary use the exact terms listed below.)

• Abstract  ... A paragraph length description of what the study includes.
• Introduction ...Includes a statement of the hypotheses for the research and a review of other research on the topic.
• Who are participants
• Design of the study
• What the participants did
• What measures were used
• Results ...Describes the outcomes of the measures of the study.
• Discussion ...Contains the interpretations and implications of the study.
• References ...Contains citation information on the material cited in the report. (also called bibliography or works cited)

Characteristics of an Empirical Article:

• Empirical articles will include charts, graphs, or statistical analysis.
• Empirical research articles are usually substantial, maybe from 8-30 pages long.
• There is always a bibliography found at the end of the article.

Type of publications that publish empirical studies:

• Empirical research articles are published in scholarly or academic journals
• These journals are also called “peer-reviewed,” or “refereed” publications.

Examples of such publications include:

• Computers in Human Behavior
• Journal of Educational Psychology

Examples of databases that contain empirical research:  (selected list only)

• Web of Science

This page is adapted from the Sociology Research Guide: Identify Empirical Articles page at Cal State Fullerton Pollak Library.

Know the difference between scholarly and non-scholarly articles.

"Scholarly" journal = "Peer-Reviewed" journal = "Refereed" journal

When researching your topic, you may come across many different types of sources and articles. When evaluating these sources, it is important to think about: 

• Who is the author? 
• Who is the audience or why was this written? 
• Where was this published? 
• Is this relevant to your research? 
• When was this written? Has it been updated? 
• Are there any citations? Who do they cite?  

Helpful Links and Guides

Here are helpful links and guides to check out for more information on scholarly sources: 

• This database contains data on different types of serials and can be used to determine whether a periodical is peer-reviewed or not:  Ulrich's Periodicals Directory  
• The UC Berkeley Library published this useful guide on evaluating resources, including the differences between scholarly and popular sources, as well as how to find primary sources:  UC Berkeley's Evaluating Resources LibGuide
• << Previous: Quick Poll
• Next: Cite Sources >>
• Last Updated: Feb 14, 2024 3:32 PM
• URL: https://guides.libraries.emory.edu/main/psyc200

Identifying and Locating Empirical Articles

• Types of Scholarly Articles

Empirical Articles vs Review Articles

• Locating Empirical Articles

Empirical articles are written to share the results of original research. Their authors will share their findings, including results, data, and ideas for future research. This will allow other researchers to learn more and conduct further studies.

Review articles are written to compare and discuss the results of multiple articles. They may be structured similarly to original research articles, but they are synthesizing what others have written about, rather than reporting on their own research.

Watch the video below (~1:30 min) for more information.

• << Previous: Types of Scholarly Articles
• Next: Locating Empirical Articles >>
• Last Updated: May 28, 2024 2:55 PM
• URL: https://libguides.ucmerced.edu/identifying-locating-empirical-articles

Secondary Education: Empirical and Review Articles

• Articles & Databases
• Annotated Bibliographies & Literature Reviews
• Scholarly Books & Ebooks
• Empirical and Review Articles
• Lesson Planning Sources

Article Exercise

An  empirical article  is a research article that reports the results of a study that uses  data derived from actual observation  or  experimentation .

A scholarly  literature review  article  summarizes and critically evaluates  academic articles and  other scholarly works  on a topic.  It  does not contain original research .

Click on the links below to determine if the sample articles present empirical studies or review the literature. 

Zbiek, R. M., & Larson, M. R. (2015). Teaching strategies to improve algebra learning.  Mathematics Teacher, 108 (9), 696-699.

Is it peer-reviewed?

• Ulrichs Periodicals Directory This link opens in a new window Directory of approximately 250,000 regularly and irregularly published journals, magazines, newspapers, and full-text electronic resources from more than 200 countries. Search or browse by title, keyword, subject, ISSN or publisher; list and sorting options; and highlights of new, changed, and ceased titles.
• << Previous: Statistics
• Next: Lesson Planning Sources >>
• Last Updated: Jul 26, 2024 10:47 AM
• URL: https://libraryguides.fullerton.edu/EDSC

This site is maintained by Pollak Library .

To report problems or comments with this site, please contact [email protected] . © California State University, Fullerton. All Rights Reserved.

Web Accessibility

CSUF is committed to ensuring equal accessibility to our users. Let us know about any accessibility problems you encounter using this website. We'll do our best to improve things and get you the information you need.

• Adobe Reader
• Microsoft Viewers
• Report An ATI Issue
• Accessible @ CSUF

Colorado College

Research guides.

Tutt Library Research Guides

• Find Articles
• Search Tips
• Empirical v. Theoretical
• Credibility
• Position Papers
• Curricular Education Databases
• Statistics & Data
• Streaming Films & DVDs
• Print Books
• Professional Development & Careers
• Citation Guide This link opens in a new window
• ED 101 Information

Empirical or Theoretical?

Empirical: Based on data gathered by original experiments or observations.

Theoretical: Analyzes and makes connections between empirical studies to define or advance a theoretical position.

Videos on Finding Empirical Articles

Where Can I Find Empirically-Based Education Articles?

The most direct route is to search PsycInfo, linked above.

This will take you to the Advanced Search, where you can type in your key words at the top. Then scroll down through all the limiting options to the Methodology menu. Select Empirical Study. 

In other databases without the Methodology limiter, such as Education Source , try keywords like empirical , study , and research .

How Can I Tell if an Article is Empirical?

Check for these components:

• Peer-reviewed
• Charts, graphs, tables, and/or statistical analyses
• More than 5 pages
• Sections with names like: Abstract, Introduction, Literature Review, Method, Data, Analysis, Results, Discussion, References

Look for visual cues of data collection and analysis:

• << Previous: Search Tips
• Next: Credibility >>
• Last Updated: Mar 27, 2024 10:27 AM
• URL: https://coloradocollege.libguides.com/education

MSU Libraries

• Need help? Ask a Librarian

Research Methods Resources - Kinesiology: Empirical vs Review Articles

• Finding Journal Articles
• Searching Indexes and Databases
• How to Locate the Full Text of an Article, once you have found the Citation
• Not sure if your Journal Article is considered Academic/Scholarly or Peer-reviewed?
• Obtaining Materials not Owned by MSU
• Connecting to Library Resources from Off Campus
• Empirical vs Review Articles
• Grants & Fundraising
• General Information
• Literature Review
• Citation Mapping
• SAGE Research Methods

Empirical Journal Articles

Empirical Article - (Original Research) Based on experience and observation, rather than systematic logic. (according to MedicineNet.com)

The articles contain original research (such as scientific experiments, surveys and research studies) A list of references or sources is provided at the end of each article An editorial board, composed of experts in the field, reviews articles to decide whether they should be accepted; this is also known as "refereed," "peer-reviewed," "professional," "scholarly", or "academic". Uses a specialized vocabulary for that field.

Below are two websites that explain Empirical articles and research:

Empirical Research: How to Recognize and Locate (Penn State University) - Empirical Research PDF

Review Journal Articles

Review Article

An article that summarizes the progress or current state in some particular subject, area, or topic.

How to write a "Review Article?" (National Library of Medicine) - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/

Subject Guide

• << Previous: Connecting to Library Resources from Off Campus
• Next: Grants & Fundraising >>
• Last Updated: May 10, 2022 5:57 PM
• URL: https://libguides.lib.msu.edu/c.php?g=96527

"How Do I?" @JWULibrary

Sample Question

• JWU-Providence Library

Q. What's the difference between a research article and a review article?

• 35 about the library
• 29 articles & journals
• 1 Borrowing
• 8 citing sources
• 17 company & industry
• 11 computers
• 1 copyright compliance
• 5 countries & travel
• 2 course registration
• 50 databases
• 3 education
• 2 Interlibrary loan
• 5 job search
• 6 libguides
• 9 market research
• 25 my library account
• 12 requests
• 26 research basics
• 21 research topics
• 2 study rooms
• 16 technology
• 7 textbooks
• 42 university
• 3 video tutorial
• 1 writing_help

Answered By: Sarah Naomi Campbell Last Updated: Sep 07, 2018     Views: 215733

Watch this short video to learn about types of scholarly articles, including research articles and literature reviews!

Not in the mood for a video? Read on!

What's the difference between a research article and a review article?

Research articles , sometimes referred to as empirical  or primary sources , report on original research. They will typically include sections such as an introduction, methods, results, and discussion.

Here is a more detailed explanation of research articles .

Review articles , sometimes called literature reviews  or secondary sources , synthesize or analyze research already conducted in primary sources. They generally summarize the current state of research on a given topic.

Here is a more detailed explanation of review articles .

The video above was created by the Virginia Commonwealth University Libraries .

The defintions, and the linked detailed explanations, are paraphrased from the Publication Manual of the American Psychological Association , 6th ed .

The linked explanations are provided by the Mohawk Valley Community College Libraries .

Links & Files

• How do I find empirical articles in the library databases?
• Share on Facebook

Was this helpful? Yes 63 No 19

Comments (0)

Related topics.

• about the library
• articles & journals
• citing sources
• company & industry
• copyright compliance
• countries & travel
• course registration
• Interlibrary loan
• market research
• my library account
• research basics
• research topics
• study rooms
• video tutorial
• writing_help

Downcity Library:

111 Dorrance Street Providence, Rhode Island 02903

401-598-1121

Harborside Library:

321 Harborside Boulevard Providence, RI 02905

401-598-1466

• Location and Directions
• Off-Campus Access
• Staff Directory
• Student Employment
• Pay Bills and Fines
• Chat with a Librarian
• Course Reserves
• Interlibrary Loan (ILL)
• Study Rooms
• Research Appointment
• Culinary Museum

• Undergraduate
• Short Courses
• Online Courses
• Masters/Postgraduate
• Postdoctoral
• Study in Africa
• Study in Asia
• Study in Australia
• Study in Europe
• Study in USA
• Fellowships
• Internships
• Volunteering
• OD Live Series
• Young Person of the Month
• Success Stories
• General Tips
• ODIC 2023 Application
• ODIC Judges
• ODIC 2023 Winners
• ODIC 2022 Winners
• ODIC 2021 Winners
• ODIC 2019 Winners
• ODIC 2018 Winners
• ODIC 2017 Winners
• ODIC 2016 Winners
• Australia and Oceania

Empirical Study vs. Literature Review

When writing an essay or an article, it is imperative to identify its theme. The flow of the written content, the information that is presented and all other aspects are based on the topic. When writing the research paper, students often accost empirical studies and literature review. To write a legit paper, it is essential to understand the difference between the two concepts. Both papers account for showing the facts, however, they vary in the approach. If there are difficulties with writing essays, a student can always turn to  write my essays services. They will help you write a paper on almost any subject and on any topic. A question what is empirical research often confuses those who learn. Systematic review vs literature review description is shown below:

1. Legit meaning of empirical research

Empirical research derives conclusions based on experience, which can be directly visualized or indirectly observed with the help of experiments. Normally, the analysis is performed by quantifying the results and the inference is drawn. The idea is to present the findings that are quantifiably observable through calibrated scientific applications. Paper review writers investigate every result before writing any conclusion in a paper or an essay. The experiment is carried out under a planned or controlled observation and the inference is strictly monitored. In contrast to the existing literature, emphasis is on the current findings of the experiment under operation. The previous literature is used to make a hypothesis, which can also be a prediction of a previously proposed theory.

2. How to work on the writing

The results are recorded, and the conclusions are drawn from the same. It is a frequent occurrence that science students are not adept for writing scientific papers. There are multiple services available online that help students with their assignments. The learners can easily avail the services online and get the paper written in accordance with their specific requirements. Services like Edusson or EssayServiceScanner are genuine ones. Students need to login and specify their requirements so that an expert assist with developing the paper. However, before paying a writer, it is better to go through the reviews about the platform. Ensure that it is not a scam. It is also recommended to review the article before you make any submissions.

3. Literature review

In contrast to the empirical study, it is essential to review different researches while working on the literature review. Such an article contains facts and information derived from other theoretical sources. New inferences may be drawn from the compilation of all literature pieces. However, the information and hypothesis are already existent. The literature review is a compilation of all the relevant data in order to produce coherent results. No experimental work may be conducted and the results come out of already existing theories in a systematic manner.

4. Deciding the type of article

College students often have to write different papers. A literature review is when a student attempts to prove or disprove a conceived point or an analysis through literature. By means of existing facts, he or she can challenge a hypothesis or attempt to answer a particular question. The article based on an experiment that was conducted solely to write the paper is an empirical review . The experiment is carried out by means of calibrated instruments in a controlled manner.

It is better to begin writing while conducting the experiment. Observations should be written down systematically during the research. This helps bring out coherence, which is easier to comprehend even later. Furthermore, beginning the writing process earlier provides one with enough time for the review and also produces better work. This is particularly necessary for the empirical papers as the experiments may take time to deliver the desired results. Leaving the writing behind and starting to write it when the deadline is approaching may make the process stressful enough and may lead to confusion. This also hampers with the work leading to decreased levels of quality. Hence, working in a timely way helps in developing the paper and your life on the whole.

For more information, visit OD Blog .

Opportunity Desk is the one stop place for all global opportunities. Connect with us on Facebook, Twitter and Instagram - @opportunitydesk

Related Posts

Key internet technologies for top managers, top countries for hiring remote software talent: a global guide, mehreen raza from pakistan is od young person of the month – august 2024..

Type above and press Enter to search. Press Esc to cancel.

The difference between empirical and discussion chapters (and how to write them)

Jun 7, 2019

It is a common misconception that the empirical chapters are the place for your analysis. Often this confuses the reader. In fact, you need to split the empirical facts and discussion of those facts into two distinct sections. 

In this post I want to explain why, more often that not, you need have  separate  empirical and discussion chapters and why you shouldn’t combine them. Then, I’ll talk you through practical steps to employ when you’re writing each up. 

What is the difference between an empirical and discussion chapter?

Whilst they are closely related, they occupy two very different spaces within a PhD thesis.

We can understand a PhD as having   four distinct sections :

1. Introduction – this is where you introduce and outline the entire study.

2. Background – this where you lay the groundwork for your thesis (in your literature review, theory framework and methods).

3. Core – this is where you present your findings.

4. Synthesis  – this is where you relate the core to the background.

The empirical chapter(s) is/are where you present the facts of your study. They occupy the core of the thesis. 

The discussion chapter though is where you interpret and discuss your findings in relation to the thesis and wider discipline. That is why is occupies the synthesis stage of the research.

Your job when writing your discussion then is to interrogate and critically engage with the findings and relate them to the research aims, objectives, research questions and gap. Most of your cutting-edge analysis and engagement with your findings will take place here.

The job of a discussion chapter is therefore to critically examine your findings with reference to the discussion in the background chapters of the thesis (introduction, literature review, theoretical framework and methods) and to make judgments as to what has been learnt in your work. In essence, the job of a discussion chapter is to tell the readers what your findings (may) mean.

The reason for this distinction between empirics and discussion is to make life easier for your examiner. They are looking at whether you are capable of both presenting observations in line with your methodology, and interpreting their significance in the context of the thesis as a whole.

However, as with everything in the PhD, there will be exceptions to this. Some theses, particularly those from within the liberal arts and social sciences, may not need a separate discussion section because the nature of their study may mean that empirics and discussion are intertwined.  It is also worth pointing out that although not every thesis will have a discussion chapter, all theses will contain discussions of some sort, however short.

How do I write an empirical chapter?

It’s important to note that the form your empirical chapter will take – or indeed if you have one at all – will depend heavily on the nature of your thesis and the discipline in which you’re working. Someone working in a more theoretical space – for example, philosophers or the theoretical mathematicians – might rely less on first-hand empirical data and more on proofs. Those working on quantitative studies will have more data-driven, empiric-heavy empirical chapters.

Broadly speaking then, the emphasis in the empirical section or chapter is on factual recount and summary. You’ll be categorising your findings into particular themes and using a variety of visual elements (tables, figures, charts, and so on) to present your results. You need to show the reader what your data ‘looks like’.

You need to do it well, too. If you present data in a messy way, your examiner might think that your thinking is messy.

By the time you have finished your empirical chapter, your reader should be able to answer six questions:

1. What are the results of your investigations?

2. How do the findings relate to previous studies?

3. Was there anything surprising or that didn’t work out as planned?

4. Are there any themes or categories that emerge from the data?

5. Have you explained to the reader why you have reached particular conclusions?

6. Have you explained the results?

You are providing sufficient detail that others can draw their own inferences and construct their own explanations. Think of it as presenting the case for a jury.

That means that an empirical discussion should:

1. Tell the reader how the data was collected, with reference to the methods chapter/section

2. Tell them how they can access it if they wanted to replicate your study

3. Discuss what the results look like (using visual aids, such as tables, diagrams, graphs and so on)

4. Provide rich summaries of the findings

5. Discuss the gaps in the findings and analysis

6. Analyse the results

7. Discuss the implications of your findings

8. Discuss the limitations of the findings

Your PhD thesis. All on one page. 

Use our free PhD structure template to quickly visualise every element of your thesis. 

How do I write a discussion chapter?

Many students struggle to write up their discussion chapters . The reason is that they lack the confidence needed to make the kinds of knowledge claims required. The discussion chapter is where you start to develop your scholarly authority, and where you start to make truth claims about your interpretation of what’s going on. By implication, that means it is where you start to agree or disagree with existing literature and theoretical ideas. 

Another reason why students struggle is that they fail to realise the significance of their findings or, put differently, they don’t think their findings are significant enough in their own right. By the time we come to write our discussion, we are so conditioned by the findings that we may not realise that they are significant and do, in fact, make a contribution. There’s often an expectation gap here; students expect their contribution to be big, whereas, more often than not, the contribution a PhD actually makes is small and specific. 

As with every stage of your thesis, you must relate your discussion section/chapter to the background. Specifically, you need to relate it to the empirical chapter, aims and objectives, research questions, the gap in the literature and, if relevant, your theoretical framework. There will, therefore, be a lot of signposting to other parts of the thesis; doing so is necessary if you want to show the examiner that you can relate your findings to the broader context of your thesis and discipline.

One of the biggest obstacles is synthesising your empirical data and being able to critically discuss it in relation to this broader context. The authors of   How To Write A Better Thesis   offer up an effective technique you can use if you’re struggling to do this. They refer to it as a ‘mud-map’:

1. You can start by writing a long list of everything you have found.

2. See if you can sort and organise this list. Categorise each finding based on whether it is speculative or based in empirical fact. This is important because your discussion will need to be somewhat (but not too) speculative.

3. Try to categorise your different findings into themes.

4. Now try to find linkages between these themes.

5. Organise these themes into different section headings for the discussion chapter, and try to come up with sub-headings. 

When it comes to writing your discussion chapter, you can start by writing a few sentences that summarise the most important results. 

One danger when writing discussion sections is that they can be too wordy, offer too much interpretation and lack a clear structure.  To avoid this, you should make sure that every element of your discussion section addresses one of the following questions: 

• What are the relationships between observations? (The mud-map you developed earlier will help here.)
• Are there any trends and generalisations amongst the results? Are there any exceptions to these? 
• What are the causes of, or mechanisms behind, the underlying patterns you have uncovered?
• Do your results agree or disagree with previous work? 
• How do your findings relate to the theoretical framework you developed, if applicable? 
• How do the findings relate to the hypotheses you developed, if applicable? 
• What other explanations could there be for your results? This issue is more pertinent if you are engaging in theory creation/inductive reasoning.
• What do we now know as a result of your research that we didn’t know before? 
• What is the significance of these findings? 
• Why should we care about the findings?

When your discussion chapter is finished, your reader should be able to answer the following questions:

1. How do the findings relate to the theory and methods discussed previously?   

2. Why you have reached particular conclusions?   

3. How do your findings relate to the gaps in the literature you identified earlier?   

4. What implications do the findings have for the discipline and for existing understanding?

5. How do the findings relate to your research questions/aims and objectives?   

A particularly important theme that I want you to always bear in mind is that your interpretation and discussion of the findings needs to be done in such a way that it relates back to the aims, objectives, research questions, gap and any theory. Running through your thesis will be a central argument – your thesis statement – and it is in the empirical and, particularly, the discussion chapter, that you will present all of the evidence and logical argument necessary to support that argument.

Unsurprisingly then, these core sections of the thesis are the most important, as they are where you make your contribution. Of course, you have outlined what your contribution is in the introduction, so what you are arguing is no surprise. But, it is in the core of the thesis where you drill down into the detail and critically engage with that contribution, using your data to rigorously support your line of argument.

Hello, Doctor…

Sounds good, doesn’t it?  Be able to call yourself Doctor sooner with our five-star rated How to Write A PhD email-course. Learn everything your supervisor should have taught you about planning and completing a PhD.

Now half price. Join hundreds of other students and become a better thesis writer, or your money back. 

Share this:

10 comments.

Really great advice. I love simple, clear guidance. Especially when writing something that’s pretty complicated!!

Thanks for the kind words. I’m glad you found it useful.

It’s really guiding me a lot.

This is really great. I have had serious challenges putting up a convincing empirical chapter.

Glad you enjoyed it!

Very insightful guidelines for PhD candidates. So appreciative for such work.

Thanks for the kind words. I’m glad you’re finding the resources useful.

It’s really useful guidance. It helps a lot in simplifying the complex and messy ideas when starting the writing process.

Thanks for the kind words.

Very instructive.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Search The PhD Knowledge Base

Most popular articles from the phd knowlege base.

The PhD Knowledge Base Categories

• Your PhD and Covid
• Mastering your theory and literature review chapters
• How to structure and write every chapter of the PhD
• How to stay motivated and productive
• Techniques to improve your writing and fluency
• Advice on maintaining good mental health
• Resources designed for non-native English speakers
• PhD Writing Template
• Explore our back-catalogue of motivational advice

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals

You are here

• Online First
• Identifying evidence-practice gaps for shoulder injury risk factors in competitive swimmers: uniting literature and expert opinion
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-1071-4958 Alec Kenneth McKenzie 1 , 2 ,
• http://orcid.org/0000-0003-2908-4271 Andrea Hams 2 , 3 ,
• http://orcid.org/0000-0002-7591-1162 Jonathon Headrick 2 ,
• http://orcid.org/0000-0003-4764-2361 Alex Donaldson 4 ,
• http://orcid.org/0000-0003-1269-4612 Rick Dann 2 ,
• http://orcid.org/0000-0002-5855-4298 Joseph Coyne 5 , 6 ,
• http://orcid.org/0000-0002-4014-7731 Steven John Duhig 2
• 1 Queensland Academy of Sport , Brisbane , Queensland , Australia
• 2 School of Health Sciences and Social Work , Griffith University , Gold Coast , Queensland , Australia
• 3 Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE) , Griffith University , Gold Coast , Queensland , Australia
• 4 Centre for Sport and Social Impact , LaTrobe University , Melbourne , Victoria , Australia
• 5 Bond University , Robina , Queensland , Australia
• 6 University of Southern Queensland , Toowoomba , Queensland , Australia
• Correspondence to Alec Kenneth McKenzie; alec.mckenzie2{at}griffithuni.edu.au

Objectives To identify evidence-practice gaps regarding shoulder injury risk factors in competitive swimmers.

Methods We gathered insights from 27 swimming experts including elite swimmers, coaches, high-performance staff and applied researchers using Concept Mapping. Participants brainstormed, sorted and rated (from 1 (least) to 10 (most) important and modifiable) their ideas of shoulder injury risk factors in competitive swimmers. Proposed risk factors rated above the grand mean for importance (6.2±0.4) or modifiability (6.5±0.5) ratings were considered highly important/modifiable. Expert opinions were then juxtaposed with systematic review findings to identify overlaps or convergences.

Results Brainstorming generated 126 proposed shoulder injury risk factors for competitive swimmers, subsequently refined to 61 unique proposed risk factors by removing duplicates and combining similar responses. The participants sorted the 61 risk factors into seven distinct clusters. Experts perceived 36/61 proposed risk factors as highly important, of which 6 were supported by literature, 6 showed no association with injury, 2 had conflicting evidence and the remaining 22 have not yet been investigated, suggesting an evidence-practice gap. Three proposed risk factors ‘inconsistent training load’, ‘poor stroke technique’ and ‘low posterior shoulder strength-endurance’ exhibited high perceived importance, high perceived modifiability and supporting evidence.

Conclusion An evidence-practice gap was identified for 28 proposed risk factors perceived as highly important by swimming experts despite either (1) no relevant empirical research (n=22), or (2) no association with injury (n=6) from synthesised evidence. Greater collaboration between researchers and practitioners is needed to effectively address shoulder injury risk factors in competitive swimmers.

• Sporting injuries
• Risk factor

Data availability statement

Data are available upon reasonable request. All collected data is presented in-text. Please email corresponding author for any further information.

https://doi.org/10.1136/bjsports-2023-108023

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

X @a_k_mckenzie, @andrea_hams, @Jon_Headrick, @AlexDonaldson13, @josephcoyne, @duhigs

Contributors This study was part of AKM's (guarantor) research programme supervised by SJD, AH and JH. AD provided expert guidance on concept mapping methodologies. JC provided practical input as an elite swimming strength and conditioning practitioner. RD aided with concept mapping analysis, data synthesis and manuscript drafting.

Funding This project was funded by a $A3000 grant award by Swimming Australia.

Competing interests None declared.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF: