what type of evidence is a case study

Evidence Based Practice: Study Designs & Evidence Levels

• Databases to Search
• EBP Resources
• Study Designs & Evidence Levels
• How Do I...

Introduction

This section reviews some research definitions and provides commonly used evidence tables.

Levels of Evidence Johns Hopkins Nursing Evidence Based Practice

Dang, D., & Dearholt, S. (2017). Johns Hopkins nursing evidence-based practice: model and guidelines. 3rd ed. Indianapolis, IN: Sigma Theta Tau International. www.hopkinsmedicine.org/evidence-based-practice/ijhn_2017_ebp.html

Identifying the Study Design

The type of study can generally be figured out by looking at three issues:

Q1. What was the aim of the study?

• To simply describe a population (PO questions)  = descriptive
• To quantify the relationship between factors (PICO questions)  =  analytic.

Q2. If analytic, was the intervention randomly allocated?

• Yes?  =  RCT 
• No? = Observational study  

For an observational study, the main type will then depend on the timing of the measurement of outcome, so our third question is:

Q3. When were the outcomes determined?

• Some time after the exposure or intervention? = Cohort study ('prospective study')
• At the same time as the exposure or intervention? = Cross sectional study or survey
• Before the exposure was determined? = Case-control study ('retrospective study' based on recall of the exposure)

Centre for Evidence-Based Medicine (CEBM)

Definitions of Study Types

Case report / Case series:  A report on a series of patients with an outcome of interest. No control group is involved.

Case control study:  A study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Cohort study:  Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Randomized controlled clinical trial:  Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Systematic review:  A summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Meta-analysis:  A systematic review that uses quantitative methods to synthesize and summarize the results.

Meta-synthesis: A systematic approach to the analysis of data across qualitative studies. -- EJ Erwin, MJ Brotherson, JA Summers. Understanding Qualitative Meta-synthesis. Issues and Opportunities in Early Childhood Intervention Research, 33(3) 186-200 .

Cross sectional study:  The observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously.

Prospective, blind comparison to a gold standard:  Studies that show the efficacy of a diagnostic test are also called prospective, blind comparison to a gold standard study. This is a controlled trial that looks at patients with varying degrees of an illness and administers both diagnostic tests — the test under investigation and the “gold standard” test — to all of the patients in the study group. The sensitivity and specificity of the new test are compared to that of the gold standard to determine potential usefulness.

Qualitative research:  answers a wide variety of questions related to human responses to actual or potential health problems.The purpose of qualitative research is to describe, explore and explain the health-related phenomena being studied.

Retrospective cohort:  follows the same direction of inquiry as a cohort study.  Subjects begin with the presence or absence of an exposure or risk factor and are followed until the outcome of interest is observed.  However, this study design uses information that has been collected in the past and kept in files or databases.  Patients are identified for exposure or non-exposures and the data is followed forward to an effect or outcome of interest.

(Adapted from CEBM's Glossary and Duke Libraries' Intro to Evidence-Based Practice )

American Association of Critical Care Nursing-- Levels of Evidence

Level A   Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention or treatment

Level B  Well designed controlled studies, both randomized and nonrandomized, with results that consistently support a specific action, intervention, or treatment

Level C   Qualitative studies, descriptive or correlational studies, integrative reviews, systematic reviews, or randomized controlled trials with inconsistent results

Level D Peer-reviewed professional organizational standards, with clinical studies to support recommendations

Level E Theory-based evidence from expert opinion or multiple case reports

Level M  Manufacturers’ recommendations only  

Armola RR, Bourgault AM, Halm MA, Board RM, Bucher L, Harrington L, Heafey CA, Lee R, Shellner PK, Medina J. (2009) AACN levels of evidence: what's new ?  J.Crit Care Nurse. Aug;29(4):70-3.

Flow Chart of Study Designs

Figure: Flow chart of different types of studies (Q1, 2, and 3 refer to the three questions below  in "Identifying the Study Design" box.) Centre for Evidence-Based Medicine (CEBM)

What is a "Confidence Interval (CI)"?

A confidence interval (CI) can be used to show within which interval the population's mean score will probably fall. Most researchers use a CI of 95%. By using a CI of 95%, researchers accept there is a 5% chance they have made the wrong decision in treatment. Therefore, if 0 falls within the agreed CI, it can be concluded that there is no significant difference between the two treatments. When 0 lies outside the CI, researchers will conclude that there is a statistically significant difference.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics.  Journal Of Wound Care ,  22 (5), 248-251.

What is a "p-value?"

Categorical (nominal) tests This category of tests can be used when the dependent, or outcome, variable is categorical (nominal), such as the dif­ference between two wound treatments and the healing of the wound (healed versus non­healed). One of the most used tests in this category is the chi­squared test (χ2). The chi­squared statistic is calculated by comparing the differences between the observed and the expected frequencies. The expected frequencies are the frequencies that would be found if there was no relationship between the two variables. 

Based on the calculated χ2 statistic, a probability (p ­value) is given, which indicates the probability that the two means are not different from each other. Researchers are often satisfied if the probability is 5% or less, which means that the researchers would conclude that for p < 0.05, there is a significant difference. A p ­value ≥ 0.05 suggests that there is no significant difference between the means.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics. Journal Of Wound Care, 22(5), 248-251.

• << Previous: EBP Resources
• Next: How Do I... >>
• Last Updated: Sep 9, 2024 1:01 PM
• URL: https://mcw.libguides.com/evidencebasedpractice

MCW Libraries 8701 Watertown Plank Road Milwaukee, WI 53226 (414) 955-8300

Contact Us Locations & Hours Send Us Your Comments

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Prevent plagiarism. Run a free check.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved September 12, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

OHSU Evidence-Based Practice Course for Interprofessional Clinical Teams

• Course Info
• Course Application
• Search Strategy Consultation
• GRADE Table Consultation
• EBP Guideline
• Informatics Consult
• Final Team Presentations and Guidelines
• Hierarchy of Evidence and Study Design

Session 2: Pre-Session Work

Hierarchy of evidence, is it a good fit for my pico, types of study designs.

Please watch the 3 videos below for more information on study design. This should take about 10 minutes. 

Overview of Research Studies - The 5 C's

Randomized Controlled Trials (RCTs)

Systematic review & Meta-analysis

Randomized Controlled Trial  is a prospective, analytical, experimental study using primary data generated in the clinical environment. Individuals similar at the beginning are randomly allocated to two or more groups (treatment and control) then followed to determine the outcome of the intervention. 

Cohort Study (prospective) is a study of a group of individuals, some of whom are exposed to a variable of interest (e.g., drug or environmental exposure), in which participants are followed up over time to determine who develops the outcome of interest and whether the outcome is associated with the exposure.

Cohort Study (retrospective) is when data is gathered for a cohort that was formed sometime in the past. Exposures and outcomes have already occurred at the start of the study. You are studying the risk factor and see if you can associate a disease to it. Individuals split by exposure.

Case Control Study  is a study in which patients who already have a specific condition or outcome are compared with people who do not. Researchers look back in time (retrospective) to identify possible exposures. They often rely on medical records and patient recall for data collection. Individuals split by disease.

Survey Study is an epidemiologic study that produces survey results, and will consist of simultaneous assessments of the health outcome, primary risk exposure and potential confounders and effect modifiers. Two types of survey research are cross-sectional and longitudinal studies.

Cross-Sectional Study is the observation of a defined population at a single point in time or during a specific time interval to examine associations between the outcomes and exposure to interventions. Exposure and outcome are determined simultaneously. Often rely on data originally collected for other purposes.

Longitudinal Study follow subjects over time with continuous or repeated monitoring of risk factors or health outcomes, or both. Researchers conduct several observations of the same subjects over a period of time, sometimes lasting many years.

Before and After Study is a study in   in which observations are made before (pre) and after (post) the implementation of an intervention, both in a group that receives the intervention and in a control group that does not. 

Case Series  and  Case Reports  are descriptive study/studies that consist of collections of reports on the treatment of individual patients or a report on a single patient. 

Systematic Review  usually focuses on a specific clinical question and conducts an extensive literature search to identify studies with sound methodology. The studies are reviewed, assessed, and the results summarized according to pre-determined criteria of the review question.

Meta-Analysis  takes a systematic review one step further by combining all the results using accepted statistical methodology.

• << Previous: Session 2
• Next: Session 3 >>
• Last Updated: Sep 6, 2024 12:17 PM
• URL: https://libguides.ohsu.edu/EBP4ClinicalTeams

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

• Evidence-Based Medicine
• Finding the Evidence
• eJournals for EBM

Levels of Evidence

• JAMA Users' Guides
• Tutorials (Learning EBM)
• Web Resources

Resources That Rate The Evidence

• ACP Smart Medicine
• Agency for Healthcare Research and Quality
• Clinical Evidence
• Cochrane Library
• Health Services/Technology Assessment Texts (HSTAT)
• PDQ® Cancer Information Summaries from NCI
• Trip Database

Critically Appraised Individual Articles

• Evidence-Based Complementary and Alternative Medicine
• Evidence-Based Dentistry
• Evidence-Based Nursing
• Journal of Evidence-Based Dental Practice

Grades of Recommendation

Critically-appraised individual articles and synopses include:

Filtered evidence:

• Level I: Evidence from a systematic review of all relevant randomized controlled trials.
• Level II: Evidence from a meta-analysis of all relevant randomized controlled trials.
• Level III: Evidence from evidence summaries developed from systematic reviews
• Level IV: Evidence from guidelines developed from systematic reviews
• Level V: Evidence from meta-syntheses of a group of descriptive or qualitative studies
• Level VI: Evidence from evidence summaries of individual studies
• Level VII: Evidence from one properly designed randomized controlled trial

Unfiltered evidence:

• Level VIII: Evidence from nonrandomized controlled clinical trials, nonrandomized clinical trials, cohort studies, case series, case reports, and individual qualitative studies.
• Level IX: Evidence from opinion of authorities and/or reports of expert committee

Two things to remember:

1. Studies in which randomization occurs represent a higher level of evidence than those in which subject selection is not random.

2. Controlled studies carry a higher level of evidence than those in which control groups are not used.

Strength of Recommendation Taxonomy (SORT)

• SORT The American Academy of Family Physicians uses the Strength of Recommendation Taxonomy (SORT) to label key recommendations in clinical review articles. In general, only key recommendations are given a Strength-of-Recommendation grade. Grades are assigned on the basis of the quality and consistency of available evidence.
• << Previous: eJournals for EBM
• Next: JAMA Users' Guides >>
• Last Updated: Aug 16, 2024 4:11 PM
• URL: https://guides.library.stonybrook.edu/evidence-based-medicine
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Systematic Reviews

• Levels of Evidence
• Evidence Pyramid
• Joanna Briggs Institute

The evidence pyramid is often used to illustrate the development of evidence. At the base of the pyramid is animal research and laboratory studies – this is where ideas are first developed. As you progress up the pyramid the amount of information available decreases in volume, but increases in relevance to the clinical setting.

Meta Analysis  – systematic review that uses quantitative methods to synthesize and summarize the results.

Systematic Review  – summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate st atistical techniques to combine these valid studies.

Randomized Controlled Trial – Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Cohort Study – Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Case Control Study – study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Case Series   – report on a series of patients with an outcome of interest. No control group is involved.

• Levels of Evidence from The Centre for Evidence-Based Medicine
• The JBI Model of Evidence Based Healthcare
• How to Use the Evidence: Assessment and Application of Scientific Evidence From the National Health and Medical Research Council (NHMRC) of Australia. Book must be downloaded; not available to read online.

When searching for evidence to answer clinical questions, aim to identify the highest level of available evidence. Evidence hierarchies can help you strategically identify which resources to use for finding evidence, as well as which search results are most likely to be "best".                                             

Image source: Evidence-Based Practice: Study Design from Duke University Medical Center Library & Archives. This work is licensed under a Creativ e Commons Attribution-ShareAlike 4.0 International License .

The hierarchy of evidence (also known as the evidence-based pyramid) is depicted as a triangular representation of the levels of evidence with the strongest evidence at the top which progresses down through evidence with decreasing strength. At the top of the pyramid are research syntheses, such as Meta-Analyses and Systematic Reviews, the strongest forms of evidence. Below research syntheses are primary research studies progressing from experimental studies, such as Randomized Controlled Trials, to observational studies, such as Cohort Studies, Case-Control Studies, Cross-Sectional Studies, Case Series, and Case Reports. Non-Human Animal Studies and Laboratory Studies occupy the lowest level of evidence at the base of the pyramid.

• << Previous: What is a Systematic Review?
• Next: Locating Systematic Reviews >>
• Getting Started
• What is a Systematic Review?
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: Aug 14, 2024 11:07 AM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Levels of evidence in research

• 5 minute read
• 120.9K views

Table of Contents

Level of evidence hierarchy

When carrying out a project you might have noticed that while searching for information, there seems to be different levels of credibility given to different types of scientific results. For example, it is not the same to use a systematic review or an expert opinion as a basis for an argument. It’s almost common sense that the first will demonstrate more accurate results than the latter, which ultimately derives from a personal opinion.

In the medical and health care area, for example, it is very important that professionals not only have access to information but also have instruments to determine which evidence is stronger and more trustworthy, building up the confidence to diagnose and treat their patients.

5 levels of evidence

With the increasing need from physicians – as well as scientists of different fields of study-, to know from which kind of research they can expect the best clinical evidence, experts decided to rank this evidence to help them identify the best sources of information to answer their questions. The criteria for ranking evidence is based on the design, methodology, validity and applicability of the different types of studies. The outcome is called “levels of evidence” or “levels of evidence hierarchy”. By organizing a well-defined hierarchy of evidence, academia experts were aiming to help scientists feel confident in using findings from high-ranked evidence in their own work or practice. For Physicians, whose daily activity depends on available clinical evidence to support decision-making, this really helps them to know which evidence to trust the most.

So, by now you know that research can be graded according to the evidential strength determined by different study designs. But how many grades are there? Which evidence should be high-ranked and low-ranked?

There are five levels of evidence in the hierarchy of evidence – being 1 (or in some cases A) for strong and high-quality evidence and 5 (or E) for evidence with effectiveness not established, as you can see in the pyramidal scheme below:

Level 1: (higher quality of evidence) – High-quality randomized trial or prospective study; testing of previously developed diagnostic criteria on consecutive patients; sensible costs and alternatives; values obtained from many studies with multiway sensitivity analyses; systematic review of Level I RCTs and Level I studies.

Level 2: Lesser quality RCT; prospective comparative study; retrospective study; untreated controls from an RCT; lesser quality prospective study; development of diagnostic criteria on consecutive patients; sensible costs and alternatives; values obtained from limited stud- ies; with multiway sensitivity analyses; systematic review of Level II studies or Level I studies with inconsistent results.

Level 3: Case-control study (therapeutic and prognostic studies); retrospective comparative study; study of nonconsecutive patients without consistently applied reference “gold” standard; analyses based on limited alternatives and costs and poor estimates; systematic review of Level III studies.

Level 4: Case series; case-control study (diagnostic studies); poor reference standard; analyses with no sensitivity analyses.

Level 5: (lower quality of evidence) – Expert opinion.

By looking at the pyramid, you can roughly distinguish what type of research gives you the highest quality of evidence and which gives you the lowest. Basically, level 1 and level 2 are filtered information – that means an author has gathered evidence from well-designed studies, with credible results, and has produced findings and conclusions appraised by renowned experts, who consider them valid and strong enough to serve researchers and scientists. Levels 3, 4 and 5 include evidence coming from unfiltered information. Because this evidence hasn’t been appraised by experts, it might be questionable, but not necessarily false or wrong.

Examples of levels of evidence

As you move up the pyramid, you will surely find higher-quality evidence. However, you will notice there is also less research available. So, if there are no resources for you available at the top, you may have to start moving down in order to find the answers you are looking for.

• Systematic Reviews: -Exhaustive summaries of all the existent literature about a certain topic. When drafting a systematic review, authors are expected to deliver a critical assessment and evaluation of all this literature rather than a simple list. Researchers that produce systematic reviews have their own criteria to locate, assemble and evaluate a body of literature.
• Meta-Analysis: Uses quantitative methods to synthesize a combination of results from independent studies. Normally, they function as an overview of clinical trials. Read more: Systematic review vs meta-analysis .
• Critically Appraised Topic: Evaluation of several research studies.
• Critically Appraised Article: Evaluation of individual research studies.
• Randomized Controlled Trial: a clinical trial in which participants or subjects (people that agree to participate in the trial) are randomly divided into groups. Placebo (control) is given to one of the groups whereas the other is treated with medication. This kind of research is key to learning about a treatment’s effectiveness.
• Cohort studies: A longitudinal study design, in which one or more samples called cohorts (individuals sharing a defining characteristic, like a disease) are exposed to an event and monitored prospectively and evaluated in predefined time intervals. They are commonly used to correlate diseases with risk factors and health outcomes.
• Case-Control Study: Selects patients with an outcome of interest (cases) and looks for an exposure factor of interest.
• Background Information/Expert Opinion: Information you can find in encyclopedias, textbooks and handbooks. This kind of evidence just serves as a good foundation for further research – or clinical practice – for it is usually too generalized.

Of course, it is recommended to use level A and/or 1 evidence for more accurate results but that doesn’t mean that all other study designs are unhelpful or useless. It all depends on your research question. Focusing once more on the healthcare and medical field, see how different study designs fit into particular questions, that are not necessarily located at the tip of the pyramid:

• Questions concerning therapy: “Which is the most efficient treatment for my patient?” >> RCT | Cohort studies | Case-Control | Case Studies
• Questions concerning diagnosis: “Which diagnose method should I use?” >> Prospective blind comparison
• Questions concerning prognosis: “How will the patient’s disease will develop over time?” >> Cohort Studies | Case Studies
• Questions concerning etiology: “What are the causes for this disease?” >> RCT | Cohort Studies | Case Studies
• Questions concerning costs: “What is the most cost-effective but safe option for my patient?” >> Economic evaluation
• Questions concerning meaning/quality of life: “What’s the quality of life of my patient going to be like?” >> Qualitative study

Find more about Levels of evidence in research on Pinterest:

17 March 2021 – Elsevier’s Mini Program Launched on WeChat Brings Quality Editing Straight to your Smartphone

Professor anselmo paiva: using computer vision to tackle medical issues with a little help from elsevier author services, you may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Welcome to the new OASIS website! We have academic skills, library skills, math and statistics support, and writing resources all together in one new home.

• Walden University
• Faculty Portal

Evidence-Based Research: Evidence Types

Introduction.

Not all evidence is the same, and appraising the quality of the evidence is part of evidence-based practice research. The hierarchy of evidence is typically represented as a pyramid shape, with the smaller, weaker and more abundant research studies near the base of the pyramid, and systematic reviews and meta-analyses at the top with higher validity but a more limited range of topics.

Several versions of the evidence pyramid have evolved with different interpretations, but they are all comprised of the types of evidence discussed on this page. Walden's Nursing 6052 Essentials of Evidence-Based Practice class currently uses a simplified adaptation of the Johns Hopkins model .

Evidence Levels:

Level I:  Experimental, randomized controlled trial (RCT), systematic review RTCs with or without meta-analysis

Level II:  Quasi-experimental studies, systematic review of a combination of RCTs and quasi-experimental studies, or quasi-experimental studies only, with or without meta-analysis

Level III:  Nonexperimental, systematic review of RCTs, quasi-experimental with/without meta-analysis, qualitative, qualitative systematic review with/without meta-synthesis  (see Daly 2007 for a sample qualitative hierarchy) 

Level IV : Respected authorities’ opinions, nationally recognized expert committee or consensus panel reports based on scientific evidence

Level V:  Literature reviews, quality improvement, program evaluation, financial evaluation, case reports, nationally recognized expert(s) opinion based on experiential evidence

Systematic review

What is a systematic review.

A systematic review is a type of publication that addresses a clinical question by analyzing research that fits certain explicitly-specified criteria. The criteria for inclusion is usually based on research from clinical trials and observational studies. Assessments are done based on stringent guidelines, and the reviews are regularly updated. These are usually considered one of the highest levels of evidence and usually address diagnosis and treatment questions.

Benefits of Systematic Reviews

Systematic reviews refine and reduce large amounts of data and information into one document, effectively summarizing the evidence to support clinical decisions. Since they are typically undertaken by a entire team of experts, they can take months or even years to complete, and must be regularly updated. The teams are usually comprised of content experts, an experienced searcher, a bio-statistician, and a methodologist. The team develops a rigorous protocol to thoroughly locate, identify, extract, and analyze all of the evidence available that addresses their specific clinical question.

As systematic reviews become more frequently published, concern over quality led to the PRISMA Statement to establish a minimum set of items for reporting in systematic reviews and meta-analyses.

Many systematic reviews also contain a meta-analysis.

What is a Meta-Analysis?

Meta-analysis is a particular type of systematic review that focuses on selecting and reviewing quantitative research. Researchers conducting a meta-analysis combine the results of several independent studies and reviews to produce a synthesis where possible. These publications aim to assist in making decisions about a particular therapy.

Benefits of Meta-Analysis

A meta-analysis synthesizes large amounts of data using a statistical examination. This type of analysis provides for some control between studies and generalized application to the population.

To learn how to find systematic reviews in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Systematic Reviews

Further reading

• Cochrane Handbook for Systematic Reviews of Interventions *updated 2022

Guidelines & summaries

Practice guidelines.

A practice guideline is a systematically-developed statement addressing common patient health care decisions in specific clinical settings and circumstances.  They should be valid, reliable, reproducible, clinically applicable, clear and flexible. Documentation must be included and referenced. Practice guidelines may come from organizations, associations, government entities, and hospitals/health systems.

ECRI Guidelines Trust

Best Evidence Topics

Best evidence topics are sometimes referred to as Best BETs. These topics are developed and supported for situations or setting when the high levels of evidence don't fit or are unavailable. They originated from emergency medicine providers' need to conduct rapid evidence-based clinical decisions.

Critically-Appraised Topics

Critically-appraised topics are a standardized one- to two-page summary of the evidence supporting a clinical question. They include a critique of the literature and statement of relevant results. They can be found online in many repositories.

To learn how to find critically-appraised topics in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Critically-Appraised Topics

Critically-Appraised Articles

Critically-appraised articles are individual articles by authors that evaluate and synopsize individual research studies. ACP Journal Club is the most well known grouping of titles that include critically appraised articles.

To learn how to find critically-appraised articles in the Walden Library, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Critically-Appraised Articles

Randomized controlled trial

A randomized controlled trial (RCT) is a clinical trial in which participants are randomly assigned to either the treatment group or control group. This random allocation of participants helps to reduce any possible selection bias and makes the RCT a high level of evidence. Having a control group, which receives no treatment or a placebo treatment, to compare the treatment group against allows researchers to observe the potential efficacy of the treatment when other factors remain the same. Randomized controlled trials are quantitative studies and are often the only studies included in systematic reviews.

To learn how to find randomize controlled trials, please see our CINAHL & MEDLINE help pages:

• CINAHL Search Help: Randomized Controlled Trials
• MEDLINE Search Help: Randomized Controlled Trials

Cohort study

A cohort study is an observational longitudinal study that analyzes risk factors and outcomes by following a group (cohort) that share a common characteristic or experience over a period of time.

Cohort studies can be retrospective, looking back over time at data that has already been collected, or can be prospective, following a group forward into the future and collecting data along the way.

While cohort studies are considered a lower level of evidence than randomized controlled trials, they may be the only way to study certain factors ethically. For example, researchers may follow a cohort of people who are tobacco smokers and compare them to a cohort of non-smokers looking for outcomes. That would be an ethical study. It would be highly unethical, however, to design a randomized controlled trial in which one group of participants are forced to smoke in order to compare outcomes.

To learn how to find cohort studies, please see our CINAHL and MEDLINE help pages:

• CINAHL Search Help: Cohort Studies
• MEDLINE Search Help: Cohort Studies

Case-controlled studies

Case-controlled studies are a type of observational study that looks at patients who have the same disease or outcome. The cases are those who have the disease or outcome while the controls do not. This type of study evaluates the relationship between diseases and exposures by retrospectively looking back to investigate what could potentially cause the disease or outcome.

To learn how to find case-controlled studies, please see our CINAHL and MEDLINE help pages:

• CINAHL Search Help: Case Studies
• MEDLINE Search Help: Case Studies

Background information & expert opinion

Background information and expert opinion can be found in textbooks or medical books that provide basic information on a topic. They can be helpful to make sure you understand a topic and are familiar with terms associated with it.

To learn about accessing background information, please see the Levels of Evidence Pyramid page:

• Levels of Evidence Pyramid: Background Information & Expert Opinion
• Previous Page: Levels of Evidence Pyramid
• Next Page: CINAHL Search Help
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Cost of Attendance
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Vet Sci

Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating the Internal Validity of Primary Research

Jan m. sargeant.

1 Department of Population Medicine, Ontario Veterinary College, University of Guelph, Guelph, ON, Canada

Marnie L. Brennan

2 Centre for Evidence-Based Veterinary Medicine, School of Veterinary Medicine and Science, University of Nottingham, Sutton Bonington Campus, Loughborough, United Kingdom

Annette M. O'Connor

3 Department of Large Animal Clinical Sciences, College of Veterinary Medicine, Michigan State University, East Lansing, MI, United States

Clinical decisions in human and veterinary medicine should be based on the best available evidence. The results of primary research are an important component of that evidence base. Regardless of whether assessing studies for clinical case management, developing clinical practice guidelines, or performing systematic reviews, evidence from primary research should be evaluated for internal validity i.e., whether the results are free from bias (reflect the truth). Three broad approaches to evaluating internal validity are available: evaluating the potential for bias in a body of literature based on the study designs employed (levels of evidence), evaluating whether key study design features associated with the potential for bias were employed (quality assessment), and applying a judgement as to whether design elements of a study were likely to result in biased results given the specific context of the study (risk of bias assessment). The level of evidence framework for assessing internal validity assumes that internal validity can be determined based on the study design alone, and thus makes the strongest assumptions. Risk of bias assessments involve an evaluation of the potential for bias in the context of a specific study, and thus involve the least assumptions about internal validity. Quality assessment sits somewhere between the assumptions of these two. Because risk of bias assessment involves the least assumptions, this approach should be used to assess internal validity where possible. However, risk of bias instruments are not available for all study designs, some clinical questions may be addressed using multiple study designs, and some instruments that include an evaluation of internal validity also include additional components (e.g., evaluation of comprehensiveness of reporting, assessments of feasibility or an evaluation of external validity). Therefore, it may be necessary to embed questions related to risk of bias within existing quality assessment instruments. In this article, we overview the approaches to evaluating internal validity, highlight the current complexities, and propose ideas for approaching assessments of internal validity.

Introduction

Every day in clinical practice, veterinary professionals need to make decisions ranging from a decision as to whether (or not) to use an intervention or to apply a diagnostic test, to decisions about the overall management of complex clinical conditions. Increasingly, it is expected that clinical decisions are evidence-based. Evidence-based veterinary medicine incorporates clinician experience, client preferences, animal needs, and scientific evidence when making clinical decisions ( 1 ). In this approach, scientific evidence is obtained from relevant research. When research-based evidence does not exist, other sources of evidence, such as expert opinion may need to be used. Traditional narrative reviews provide an overview of a topic, and thus may be an attractive way of quickly acquiring knowledge for making clinical decisions. However, narrative reviews generally do not provide information on the identification and selection of the primary research being summarized (if any), the methodological quality of the studies, or the magnitude of the expected effect ( 2 , 3 ).

Formal methods have been developed to systematically identify, select, and synthesize the available evidence to assist veterinary professionals in evidence-based decision-making. These include critically appraised topics (CATs) ( 4 ), systematic review and meta-analysis (SR-MA) ( 5 – 7 ), and clinical practice guidelines ( 8 ) (see Box 1 for a short overview of these methods). These evidence synthesis approaches have different purposes which results in different processes and endpoints, but each includes an assessment of the internal validity of the research used. Critical appraisal of an individual study also includes an evaluation of internal validity, in addition to an evaluation of feasibility and generalizability ( 10 ). The evaluation of internal validity is the focus of this article. Understanding the different ways internal validity can be assessed, and the assumptions associated with these approaches, is necessary for researchers evaluating internal validity, and for veterinary professionals to assess studies for integration of evidence into practice.

Overview of synthesis methods used in veterinary practice and research.

Systematic review, meta-analysis, and network meta-analysis: Systematic review is a structured methodology for identifying, selecting and evaluating all relevant research to address a structured question, which may relate to descriptive characteristics such as prevalence, etiology, efficacy of interventions, or diagnostic test accuracy ( 5 ). Meta-analysis is the statistical combination of results from multiple studies. For addressing questions on intervention efficacy, meta-analysis provides an overall effect size for pairwise comparisons between two intervention groups. Network meta-analysis allows an estimation of the comparative efficacy across all available intervention options ( 6 ), which may provide more relevant information for veterinary professionals when there are multiple intervention options available. However, systematic reviews with pairwise meta-analysis or network meta-analysis require that a body of research exists that can be synthesized to address a clinical question and can also be resource and time intensive to conduct. Therefore, there are many clinical questions for which formally synthesized research summaries do not exist.

Critically appraised topics: Critically appraised topics (CATs) use the same principles as systematic reviews to address clinical questions but employ a more rapid approach, particularly in relation to the screening and summation of the evidence. They were designed to be employed by clinicians as a way of rapidly gathering and interpreting evidence on clinical questions relating to specific cases ( 4 ). Therefore, there is a greater risk that research addressing the question may be missed. However, in the absence of a well conducted systematic review or meta-analysis, CATs can provide a faster evaluation of research addressing a clinical question and can be undertaken by veterinary professionals who may have fewer resources and potentially less methodological or statistical expertise, particularly if they are freely available and accessible.

Clinical practice guidelines: Veterinary professionals often are involved in the management of complex clinical conditions, where an array of questions need to be addressed, including those related to etiology, prognosis, diagnostic test accuracy, and intervention efficacy. Clinical practice guidelines are intended to assist healthcare professionals in assessing more than one aspect of case approach, including appropriate prevention, diagnosis, treatment, or clinical management of diseases, disorders, and other health conditions ( 9 ). Although there are differences in the methods among authors and institutions, the key elements of guideline development include the establishment of a multidisciplinary working group to develop the guidelines, the involvement of appropriate stakeholders, identification of the topic area, systematic searches for research evidence, assessment of the internal validity of studies comprising the evidence base, a process for drafting recommendations, and ongoing review and updating of the guidelines as new evidence becomes available ( 8 ).

Internal validity refers to the extent to which the study results reflect the true state of nature (i.e., whether the effect size estimated in a study is free from systematic error, also called bias) ( 11 ). Although there are a large number of named biases ( 12 ), for studies that assess interventions or risk factors, the biases can be categorized into three broad types of bias: selection bias, information bias, and confounding ( 13 ). Selection bias impacts the effect size if, compared to the source population, the exposure or intervention groups differ in the distribution of factors associated with the outcome at the time the study population is selected, or if differential loss to follow up between groups occurs during the study. In case-control studies, selection bias occurs if cases or controls are selected based on criteria that are related to the exposure of interest. Information bias occurs when there are errors in measuring the exposure or intervention, or the outcome, or both. Finally, confounding is a mixing of effects that occurs when a variable (the confounder) that is independently associated with both the exposure and the outcome is not properly controlled. When confounding is not controlled, the estimate of the relationship between the exposure and the outcome will be biased.

There are several terms used to describe the approaches to assessing internal validity of primary research studies, including evidence hierarchies and levels of evidence, quality assessment, and risk of bias assessment. The use of these terms may be confusing, and it is not uncommon for some of these terms to be used interchangeably ( 14 , 15 ). Also, authors may mislabel the approaches and some evaluation tools (instruments) available for assessing internal validity may include additional components, such as those related to comprehensiveness (quality) of reporting, feasibility of applying an intervention, or external validity. Finally, some instruments may use the approach as a label for the instrument [e.g., Cochrane's risk of bias tool ( 16 ), which is an instrument that employs a risk of bias approach] and other instruments may not include the approach in the instrument name [e.g., the Jadad scale ( 17 ), which employs a quality assessment approach]. In an evaluation of the comprehensiveness of reporting in animal health systematic reviews (SRs), Sargeant et al., ( 18 )found that a range of instruments involving all three approaches had been used for assessing the internal validity of primary research studies. Although a large number of instruments are available, the approaches within each instrument used to assess internal validity can be grouped into three broad categories: based on study design, based on the presence or absence of design features, or based on a judgement about bias in the context of the study. These categories generally correspond to levels of evidence, quality assessment, and risk of bias, respectively. Therefore, our objective was to review these approaches to assessing internal validity as distinct entities and to describe the assumptions associated with each approach. Although we provide examples of specific instruments that include an evaluation of internal validity, our focus is on the approaches, rather than the tools. We discuss advances in the use of these approaches to assessing internal validity in human healthcare and propose a process for veterinary medicine for selecting the approach with the least assumptions as appropriate to the clinical question, the purpose of the assessment, and the research found that addresses the question of interest. The target audience for this article is individuals who assess internal validity of studies, individuals who develop instruments that include items related to the assessment of internal validity, and those who use evidence synthesis products created by others, such as systematic reviews or clinical practice guidelines.

Evaluating Internal Validity by Study Design: Levels of Evidence

Levels of evidence is an approach to evaluating the internal validity of a body of evidence, based on the potential for bias which is inherent to the employed study designs that were used to address the clinical question. The concept behind levels of evidence is that there is a hierarchy of study designs, with different study designs having different potential for bias. The way evidence hierarchies are used is based on either the name of the design or the description of the design. Readers of a study look for this information, then determine the design and assign a level of evidence. No further differentiation of methodological features or judgment is conducted.

Evidence hierarchies were initially introduced in 1979 by the Canadian Task Force on the Periodic Health Examination ( 19 ), with further development into an evidence pyramid by David Sackett in 1989 ( 20 ). A pyramid shaped figure commonly is used to illustrate the hierarchy of study designs for evaluating the efficacy of an intervention under realistic-use conditions (owned animals, as opposed to experimental settings), with the potential for bias decreasing from the base to the top of the pyramid ( Figure 1 ). Thus, study designs on the top of the pyramid represent those with inherently lower risk of bias compared to study designs lower on the hierarchy. The pyramid shape acknowledges that the quantity of research tends to decrease in the higher levels of evidence (for instance, there will be a larger volume of randomized controlled trials (RCTs) compared to SR-MA). Suggested modifications to the evidence pyramid for veterinary intervention studies include dividing RCTs into those conducted under realistic-use conditions vs. those conducted in nonrealistic-use conditions (e.g., research facility) ( 21 ), the inclusion of challenge trials (where disease outcomes are deliberately induced) below RCTs in the pyramid ( 21 , 22 ), and increasing the interpretability of the concept for students by displaying the hierarchy as a staircase rather than a pyramid ( 23 ).

Illustration of an evidence pyramid hierarchy for addressing intervention studies in veterinary medicine. SR, systematic review; MA, meta-analysis; RCT, randomized controlled trial.

The concept of evaluating the potential for bias in an individual study based on the study design can be extended to an evaluation of the potential for bias in a body of literature. This approach for evaluating the internal validity of a body of literature is referred to as “levels of evidence”. The approach is applied by identifying research (or other evidence) that pertains to the clinical question, determining the study design used for each of the studies, and then assigning each study to a level of evidence based on that design. For instance, a framework for levels of evidence in veterinary clinical nutrition has been proposed by Roudebush et al. ( 24 ). In this framework, level 1 evidence corresponds to at least 1 appropriately designed RCT in the target species with natural disease development, level 2 evidence would correspond to RCTs in laboratory settings with natural disease development, level 3 evidence would be obtained from non-randomized trials, deliberate disease induction trials, analytical observational studies or case series, and level 4 evidence would correspond to expert opinion, descriptive studies, studies in other species, or pathophysiological justification. Therefore, if the clinical question involves interventions, and the evidence found to address the question consists of 2 RCTs, 3 case-control studies, and 3 case series, the evidence would be designated as “level 1 evidence” because study designs with the highest evidentiary level in the available research consisted of RCTs. If all available evidence was from expert opinion, the body of research would comprise “level 4” evidence. This evidence would represent the best available evidence to inform decision-making at the time the assessment was made, although the overall level assigned would change as higher evidentiary level information becomes available.

The levels of evidence approach may be perceived as a quick and easy approach to assessing internal validity because it requires only a knowledge of the study design employed and not the individual features of a study that may or may not be associated with the potential for bias. However, that ease of use is based on very strong assumptions: 1) that study design maps directly to bias, 2) that authors always correctly label study designs, and 3) that authors execute and report study designs appropriately. The approach also pertains to a body of evidence, implying that there are multiple comparable studies available to address the question of interest.

An important critique of levels of evidence is that the approach focuses on the study design, rather than the actual design features that were used or the context of the study. Thus, although this framework illustrates the inherent potential for bias of the different study designs, it does not provide a consideration of the methodological rigor with which any specific individual study was conducted ( 25 ). For instance, although a well-conducted cohort study may be less biased than a poorly executed RCT, this nuance is not captured by a levels of evidence approach. Additionally, levels of evidence are based on the potential for confounding and selection biases, but there is no mechanism to evaluate the potential for information bias because this is linked to the outcome and the levels of evidence approach is based on features at the study, rather than outcome, level. For instance, RCTs provide a higher level of evidence compared to observational studies because random allocation to intervention groups minimizes the potential for confounding, and case-control studies provide a lower level of evidence than cohort studies because they are more prone to selection bias. However, a RCT that used a subjectively measured outcome would be assigned a higher level of evidence than a cohort study with an objective outcome, although the observational study may have a lower risk of information bias. Finally, studies may be mislabeled in terms of their study design; there is empirical evidence that this occurs in the veterinary literature ( 26 – 28 ). For example, studies labeled as case series in veterinary medicine frequently include a component corresponding to a cohort study design ( 27 ); these studies may be assigned an inappropriately low level of evidence if individuals classifying these studies rely on authors terminology rather than the complete design description to determine the design employed.

An additional consideration is that for questions related to aspects of clinical care other than selection of interventions, the framework and positioning of study designs included in Figure 1 may not be appropriate. Levels of evidence schema are available for other clinical questions, such as prognosis, diagnostic test accuracy, disease screening, and etiology ( 29 , 30 ).

Evaluating Internal Validity Based on Inclusion of Study Features Associated With Bias: Quality Assessment

As the name implies, quality assessment represents an evaluation of the quality of a primary research article. However, the term “quality” is difficult to specifically define in the context of evidence-based medicine, in that it does not appear to have been used consistently in the literature. The Merriam-Webster dictionary defines quality as “how good or bad something is” or “a high level of value or excellence” ( https://www.merriam-webster.com/dictionary/quality ). Quality generally is understood to be a multi-dimensional concept. While clear definitions are difficult to find in the research literature, the lay literature includes numerous treaties on the dimensions of quality. One example is the eight dimensions of quality delineated by David Gavin, which include performance, features, reliability, conformance, durability, serviceability, aesthetics, and perceived quality ( https://en.wikipedia.org/wiki/Eight_dimensions_of_quality ).

The findings from a review ( 31 ) identified that available instruments labeled as quality assessment tools varied in clarity and often involved more than just assessing internal validity. In addition to including an assessment of internal validity, quality assessment instruments also generally contain elements related to quality of reporting or an assessment of the inclusion of study features not directly related to bias, such as whether ethical approval was sought or whether the study participants were similar to those animals in the care of the individual doing the critique ( 14 , 31 – 33 ).

Quality assessment as an approach to evaluating internal validity involves an evaluation of the presence or absence of design features, i.e., a methodological checklist ( 14 , 15 ). For example, the Jadad scale ( 17 ) involves completing a checklist of whether the study was described as randomized, whether the study was described as double blind, and whether there was a description of withdrawals and dropouts, with points assigned for each category. Therefore, the Jadad scale uses a quality assessment approach to evaluating internal validity. In terms of assumptions, the quality assessment approach also makes strong assumptions, although these are less than those used in levels of evidence assessments. Instead of mapping bias to the study design, quality assessment maps bias to a design feature i.e., if a trial was randomized, it is assumed to be “good quality” and if the trial was not randomized the assumption is that it is “poor quality”. The same process is followed for additional study aspects, such a blinding or losses to follow-up, and an overall assessment of quality is then based on how the study 'performs' against these questions.

Quality assessment also considers more than just confounding and selection bias as components of internal validity. The inclusion of blinding as a design feature of interest illustrates this. Blinding as a design feature is intended to reduce the potential for differential care as a source of confounding bias (blinding of caregivers) or may be intended to reduce the potential for information bias (blinding of outcome assessors). Conducting a quality assessment is more complicated and time-consuming than evaluating levels of evidence because the presence or absence of the specific design features needs to be identified and validated within the study report. However, the approach requires only that the person evaluating internal validity can identify whether (or not) a design feature was used. Therefore, this approach requires more technical expertise that the levels of evidence approach, but less than the risk of bias approach.

Evaluating Internal Validity by Making Contextualized Judgements on Potential Occurrence of Bias: Risk of Bias Assessment

Risk of bias assessments have been developed specifically for evaluating the potential for elements of the design or conduct employed within a study to lead to a biased effect size ( 34 , 35 ). The components of risk of bias assessments are selected based on empirical evidence of their association with estimates of effect sizes ( 24 , 32 ). The way risk of bias assessments work is that individuals evaluating a study for internal validity answer a series of signaling questions about the presence or absence of design features followed by a judgment about the potential for the use of the design feature to lead to a biased estimate in the context of the specific study. A conclusion is then reached about potential for bias based on all evaluated design features in the context of the study. Thus, a risk of bias assessment makes fewer assumptions about the link between study design and design features compared to quality assessment. For instance, a quality assessment for an RCT would include an evaluation as to whether blinding of outcome assessors occurred, whereas a risk of bias assessment would involve an evaluation not only as to whether blinding was used, but also a judgement as to whether a lack of blinding of outcome assessors would be likely to lead to a biased estimate given the context of the study and the outcome measures used. Thus, a RCT that did not include blinding of outcome assessors might be rated as poor on a quality assessment but might not be a concern in a risk of bias assessment if the outcomes were measured objectively, precluding the likelihood that the estimate would be biased by a knowledge of the intervention group when classifying the outcomes. Because of the necessity of making a judgement about the potential that bias is associated with design features in the context of a specific topic area, this approach requires the highest level of knowledge of study design and bias. The risk of bias approach also generally is conducted at the outcome level, rather than at the study level. For instance, an unblinded RCT of interventions to treat lameness might be considered to have a high risk of bias if the outcome was assessed by owners (a subjective outcome) but not if the outcome was assessed by force plate measurement (an objective outcome). For a level of evidence assessment, the assessment of internal validity would be high quality because the trial was an RCT. For quality assessment, the study may be considered poor quality because it was unblinded, but the overall judgement would be dependent on a number of other study design flaws identified. Finally, in a risk of bias assessment, the study would likely be low risk of bias for the objective outcome and high risk of bias for the subjective outcome if blinding was not used.

Some components of a risk of bias assessment are the same as those included in a quality assessment approach (e.g., an assessment of randomization, allocation concealment, and blinding could be included in both). However, the way the assessment is done differs, with quality assessments generally involving present/absent judgements as opposed to assessments as to whether the risk of bias is likely or not. Hartling et al. ( 14 ) applied two instruments using a quality assessment approach and one instrument using a risk of bias approach to a sample of 163 trials and found that there was low correlation between quality assessment and risk of bias approaches when comparing the assessment of internal validity.

Although the critical elements for risk of bias are well described for RCTs in human healthcare and to a large extent in veterinary RCTs, these elements are not as well described for non-randomized trials and observational studies where allocation to groups is not under the control of the investigator. There are some risk of bias tools available for assessing risk of bias in non-randomized studies, such as ROBINS-I ( 36 ). However, ROBINS-I has been criticized for being challenging to use and for having low reliability, particularly amongst less experienced raters ( 37 , 38 ). A review and critique of approaches to risk of bias assessment for observational studies is available ( 39 ). It is anticipated that risk of bias tools for observational study designs, including studies related to questions of prognosis and causation, will continue to evolve as new instruments are developed and validated.

Historical Contexts and Comparisons of Internal Validity Assessment Approaches

Currently, the available approaches to assessing internal validity tend to be used for different applications. Levels of evidence have previously been used for creating evidence-based recommendations or clinical practices guidelines ( 30 , 40 , 41 ), where it is anticipated that multiple study designs may have been used to address the clinical question(s) of interest. Both quality assessment and risk of bias assessment approaches have been used as a component of systematic reviews with meta-analysis or network meta-analysis, as the intended product of these reviews is to summarize a single parameter (such as incidence or prevalence) or a summary effect size (such as a risk ratio, odds ratio, or hazard ratio) where it is desired that the estimate is unbiased. Often, that estimate is derived from studies with the same study design or a narrow range of study designs from high levels in the evidence hierarchy for the research question type. Therefore, the focus is on a specific parameter estimate based on multiple studies, rather than a descriptive summary of the evidentiary strength of those studies.

However, the different approaches are not necessarily mutually exclusive, but are nested within each other based on assumptions, and the methodology and use of the different approaches has evolved over time. As previously described, a criticism of the use of levels of evidence is that the potential for bias is based on the study design that was employed, rather than the methodological rigor of a specific study ( 42 ). For this reason, many frameworks for levels of evidence included wording such as “appropriately designed” ( 24 ) or “well designed” ( 41 )for the study designs, although the criteria for determining whether a study was designed and executed with rigor generally is not described. A lack of transparency for the criteria for evaluating internal validity of studies within an evidence level is problematic for individuals wishing to use the results. An example of the evolution toward more transparent considerations of internal validity of individual studies within a levels of evidence framework is seen in the progression of the Australian National Health and Medical Research Council (NHMRC) system for evaluating evidence in the development of clinical practice guidelines. The designation of levels of evidence in this framework originally was based on levels of evidence, with descriptors such as “properly-designed” or “well-designed” included for each type of study design ( 40 ). A concern with this approach was that the framework was not designed to address the strength of evidence from individual studies within each evidence level ( 43 ). Therefore, the framework was modified to include the use of risk of bias evaluations of individual studies within each evidence level. The combined use of levels of evidence and risk of bias assessment of studies within each level of evidence now forms the “evidence base” component of the NHMRC's FORM framework for the development of evidence-based clinical guidelines ( 44 ).

Another example of the evolution of approaches to assessing internal validity is from the Cochrane Back review group, who conduct systematic reviews of neck and back pain. The initial methods guidelines, published in 1997, recommended that a quality assessment be performed on each included study, with each item in the quality assessment tool scored based on whether the authors reported their use ( 45 ). Updated methods guidelines were published in 2003 ( 46 ). The framework for levels of evidence in this guidance was restricted to a consideration of randomized controlled trials and non-randomized controlled clinical trials, as these were considered the study designs that potentially were appropriate to address research questions in this content area. In the updated guidelines, the recommendations for the assessment of internal validity moved to a risk of bias approach, where judgements were made on whether the characteristics of each study were likely to lead to biased study results. In the 2003 methods guidelines, levels of evidence were recommended as an approach to qualitative analysis rather than the use of “vote counting” (summing the number of studies where a positive or negative outcome was reported). The guidelines were again updated in 2009 ( 47 ). In this version, the assessment of the internal validity of individual studies explicitly employed a risk of bias approach. It was further recommended that the use of evidence levels as a component of a qualitative synthesis be replaced with a formal rating of the quality of the evidence for each of the included outcomes. It was recommended that review authors use the GRADE approach for this component. The GRADE approach explicitly includes a consideration of the risk of bias across all studies included in the review, as well as an assessment of the consistency of results across studies, the directness of the evidence to the review question, the precision in the effect size estimate, and the potential for publication bias ( 48 ).

The examples from the human medical literature illustrate that assessment of internal validity need not be static, and that modifications to our approach to assessing internal validity can strengthen the evidence base for clinical decision making. When developing or using tools which include an evaluation of internal validity, the assessment of internal validity should use the approach with the least assumptions about bias. This implies that the risk of bias approach, where context specific judgements are made related to the potential for bias, is the preferred approach for assessing internal validity. The risk of bias approach is well developed for RCTs. Therefore, when RCTs are included in the evidence available to address a clinical question, a risk of bias assessment approach should be used. When evaluating internal validity as a component of a SR-MA, the Cochrane ROB2.0 tool ( 16 ) could be used for this purpose. Modifications to this tool have been proposed for evaluating trials in livestock trials ( 49 – 51 ). For critical appraisal instruments for RCTs, where additional components such as feasibility and external validity are a desired component, the questions or items within the instrument that are specific to assessing internal validity still could follow a risk of bias approach by specifically requiring a judgement on the potential for bias. Similarly, the use of questions or items requiring a judgement on the potential for bias also could be used for evaluation of RCTs included in clinical practice guidelines when RCTs are present in the evidence base.

However, there are circumstances where these recommendations may not be appropriate or sufficient, such as for observational studies where risk of bias assessment instruments do not formally exist, or where a variety of study designs have been identified that answer the clinical question (particularly non-intervention type questions). When observational studies are used as evidence, individuals assessing internal validity may wish to evaluate risks of bias for each study ad hoc by considering the specific risks of bias related to selection bias, information bias, and confounding in the context of the topic area. However, this approach requires considerable methodological expertise. Alternatively, a quality assessment approach could be used to evaluate internal validity for observational studies, recognizing that more assumptions related to the potential for bias are involved. As instruments for evaluating the risk of bias for observational studies are developed and validated, these could replace ad hoc or quality assessment approaches.

For situations where the evidence base includes multiple study types, such as clinical practice guidelines, the use of levels of evidence may be useful for framing the potential for bias inherent in the studies identified to address the clinical questions. However, within each evidence level, there still is a need to evaluate the internal validity of each study. The proposed approach for situations where RCTs and observational studies are included in the evidence base was described in the preceding paragraphs. For lower levels of evidence, such as case series, textbooks and narrative reviews, and expert opinion, levels of evidence could be used to emphasize that these types of evidence have high potential for bias based on their design.

Broader Considerations

It should be noted that although this article has focused on approaches to evaluating internal validity of studies, this is only one component of the assessment of evidence. Critical appraisal, CATs, SR-MA, and clinical practice guidelines explicitly incorporate other aspects of decision-making, including a consideration of the magnitude and precision of an intervention effect or the potential clinical impact, the consistency of the research results across studies, the applicability (external validity and feasibility) of the research results, and the directness of the evidence to a clinical situation (for instance, whether the study populations are similar to those in a practice setting). However, a discussion of these components for decision-making is beyond the scope of the current study. The interested reader is referred to further details on the components used in evaluating evidence for CATs ( 4 ), for SR-MA using the GRADE approach ( 52 ), for network meta-analysis ( 53 ) and for clinical practice guidelines ( 8 , 44 ).

Author Contributions

JS drafted the manuscript. All authors contributed equally to the conceptualization of this work. All authors read and approved the final contents.

Partial funding support was obtained from the University of Guelph Research Leadership Chair (Sargeant).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Maltese coastline never sleeps: the effects of artificial light at night (alan) on the local infralittoral assemblages—a case study.

1. Introduction

2. materials and methods, 2.1. study area, 2.2. reef (dark), 2.3. harbour (already-present alan), 2.4. field sampling, 2.5. sampling design and video analysis, 2.6. observation protocols, 2.7. statistical analysis, 3.1. descriptive metrics, 3.2. impact of light intensity and habitat on community composition, 3.3. effects of light intensity on biodiversity metrics, 3.4. permanova of shannon diversity index, 3.5. simper analysis, 3.6. best analysis, 3.7. principal coordinates ordination (pco) analysis, 4. discussion, 4.1. alan as a threat to biodiversity: evolution and habituation, 4.2. pulse and press dynamics, 4.3. external factors, 4.4. species-specific light effects, 4.5. recommendations for alan mitigation, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Myers, N.; Mittermeier, R.A.; Mittermeier, C.G.; Da Fonseca, G.A.; Kent, J. Biodiversity hotspots for conservation priorities. Nature 2000 , 403 , 853–858. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Gaston, K.J.; Davies, T.W.; Nedelec, S.L.; Holt, L.A. Impacts of artificial light at night on biological timings. Annu. Rev. Ecol. Evol. Syst. 2017 , 48 , 49–68. [ Google Scholar ] [ CrossRef ]
• Pulgar, J.; Zeballos, D.; Vargas, J.; Aldana, M.; Manriquez, P.H.; Manriquez, K.; Quijón, P.A.; Widdicombe, S.; Anguita, C.; Quintanilla, D.; et al. Endogenous cycles, activity patterns, and energy expenditure of an intertidal fish is modified by artificial light pollution at night (ALAN). Environ. Pollut. 2019 , 244 , 361–366. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dominoni, D.M.; Nelson, R.J. Artificial light at night as an environmental pollutant: An integrative approach across taxa, biological functions, and scientific disciplines. J. Exp. Zool. Part A Ecol. Integr. Physiol. 2018 , 329 , 387–393. [ Google Scholar ] [ CrossRef ]
• Bolton, D.; Mayer-Pinto, M.; Clark, G.F.; Dafforn, K.A.; Brassil, W.A.; Becker, A.; Johnston, E.L. Coastal urban lighting has ecological consequences for multiple trophic levels under the sea. Sci. Total Environ. 2017 , 576 , 1–9. [ Google Scholar ] [ CrossRef ]
• Minnaar, C.; Boyles, J.G.; Minnaar, I.A.; Sole, C.L.; McKechnie, A.E. Stacking the odds: Light pollution may shift the balance in an ancient predator–prey arms race. J. Appl. Ecol. 2015 , 52 , 522–531. [ Google Scholar ] [ CrossRef ]
• Duffy, J.P.; Bennie, J.; Durán, A.P.; Gaston, K.J. Mammalian ranges are experiencing erosion of natural darkness. Sci. Rep. 2015 , 5 , 12042. [ Google Scholar ] [ CrossRef ]
• Gaston, K.J.; Bennie, J.; Davies, T.W.; Hopkins, J. The ecological impacts of nighttime light pollution: A mechanistic appraisal. Biol. Rev. 2013 , 88 , 912–927. [ Google Scholar ] [ CrossRef ]
• Sanna, G.; Domenici, P.; Maggi, E. Artificial light at night alters the locomotor behavior of the Mediterranean sea urchin Paracentrotus lividus . Mar. Pollut. Bull. 2024 , 206 , 116782. [ Google Scholar ] [ CrossRef ]
• Dimitriadis, C.; Fournari–Konstantinidou, I.; Sourbès, L.; Koutsoubas, D.; Mazaris, A.D. Reduction of sea turtle population recruitment caused by nightlight: Evidence from the Mediterranean region. Ocean Coast. Manag. 2018 , 153 , 108–115. [ Google Scholar ] [ CrossRef ]
• Baz, E.S.; Hussein, A.A.; Vreeker, E.M.; Soliman, M.F.; Tadros, M.M.; El-Shenawy, N.S.; Koene, J.M. Consequences of artificial light at night on behavior, reproduction, and development of Lymnaea stagnalis . Environ. Pollut. 2022 , 307 , 119507. [ Google Scholar ] [ CrossRef ]
• Díaz, S.M.; Settele, J.; Brondízio, E.; Ngo, H.; Guèze, M.; Agard, J.; Arneth, A.; Balvanera, P.; Brauman, K.; Butchart, S. The Global Assessment Report on Biodiversity and Ecosystem Services: Summary for Policy Makers. 2019. Available online: https://ri.conicet.gov.ar/handle/11336/116171 (accessed on 1 September 2023).
• Gaston, K.J.; Visser, M.E.; Hölker, F. The biological impacts of artificial light at night: The research challenge. Philos. Trans. R. Soc. B Biol. Sci. 2015 , 370 , 20140133. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Angeler, D.G.; Chaffin, B.C.; Sundstrom, S.M.; Garmestani, A.; Pope, K.L.; Uden, D.R.; Twidwell, D.; Allen, C.R. Coerced regimes: Management challenges in the Anthropocene. Ecol. Soc. 2020 , 25 , 1–4. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Marangoni, L.F.; Davies, T.; Smyth, T.; Rodríguez, A.; Hamann, M.; Duarte, C.; Levy, O. Impacts of Artificial Light at Night (ALAN) in marine ecosystems–A review. Glob. Change Biol. 2024 , 28 , 5346–5367. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Peregrym, M.; Konya, E.P.; Savchenko, M. How are the Mediterranean islands polluted by artificial light at night? Ocean Coast. Manag. 2020 , 198 , 105365. [ Google Scholar ] [ CrossRef ]
• Caruana, J.; Vella, R.; Spiteri, D.; Nolle, M.; Fenech, S.; Aquilina, N.J. A photometric mapping of the night sky brightness of the Maltese islands. J. Environ. Manag. 2020 , 261 , 110196. [ Google Scholar ] [ CrossRef ]
• Raine, H.; Borg, J.J.; Raine, A.; Bairner, S.; Cardona, M.B. Light Pollution and Its Effect on Yelkouan Shearwaters in Malta; Causes and Solutions. BirdLife Malta . 2007, pp. 1–35. Available online: https://dsr.nuclio.pt/wp-content/uploads/2014/01/light_pollution_effect_yelkouan-Shearwaters-in-Malta.pdf (accessed on 1 September 2024).
• Galdies, C. The Climate of Malta: Statistics, Trends and Analysis ; National Statistics Office: Valletta, Malta, 2011.
• Environment and Resources Authority. Monitoring Factsheet: Seabed Habitats. In Malta ; 2015; pp. 1–10. Available online: https://era.org.mt/wp-content/uploads/2019/05/MonitoringFactsheet_D1_D4_D6_Seabed-Habitats.pdf (accessed on 1 September 2023).
• Fitzpatrick, C.; McLean, D.; Harvey, E.S. Using artificial illumination to survey nocturnal reef fish. Fish. Res. 2013 , 146 , 41–50. [ Google Scholar ] [ CrossRef ]
• Lucena, M.B.; Mendes, T.C.; Barbosa, M.C.; Cordeiro, C.A.; Eggertsen, L.M.; Ferreira, C.E. Does the color of light matter? Testing different light colors in nocturnal underwater visual censuses. Mar. Environ. Res. 2021 , 166 , 105261. [ Google Scholar ] [ CrossRef ]
• Wraith, J.; Lynch, T.; Minchinton, T.E.; Broad, A.; Davis, A.R. Bait type affects fish assemblages and feeding guilds observed at baited remote underwater video stations. Mar. Ecol. Prog. Ser. 2013 , 477 , 189–199. [ Google Scholar ] [ CrossRef ]
• Schobernd, Z.; Bacheler, N.; Conn, P. Examining the utility of alternative video monitoring metrics for indexing reef fish abundance. Can. J. Fish. Aquat. Sci. 2013 , 71 , 464–471. [ Google Scholar ] [ CrossRef ]
• Mallet, D.; Pelletier, D. Underwater video techniques for observing coastal marine biodiversity; A review of sixty years of publications (1952–2012). Fish. Res. 2014 , 154 , 44–62. [ Google Scholar ] [ CrossRef ]
• Wartenberg, R.; Booth, A. Video transects are the most appropriate underwater visual census method for surveying high-latitude coral reef fishes in the southwest Indian Ocean. Mar. Biodivers. 2015 , 45 , 633–646. [ Google Scholar ] [ CrossRef ]
• Beyst, B.; Hostens, K.; Mees, J. Factors influencing fish and macrocrustacean communities in the surf zone of sandy beaches in Belgium: Temporal variation. J. Sea Res. 2001 , 46 , 281–294. [ Google Scholar ] [ CrossRef ]
• Milardi, M.; Gavioli, A.; Lanzoni, M.; Fano, E.A.; Castaldelli, G. Meteorological factors influence marine and resident fish movements in a brackish lagoon. Aquat. Ecol. 2019 , 53 , 251–263. [ Google Scholar ] [ CrossRef ]
• Kingsford, M.; Finn, M. The influence of the phase of the moon and physical processes on the input of presettlement fishes to coral reefs. J. Fish Biol. 1997 , 51 , 176–205. [ Google Scholar ] [ CrossRef ]
• Hernández-León, S.; Franchy, G.; Moyano, M.; Menéndez, I.; Schmoker, C.; Putzeys, S. Carbon sequestration and zooplankton lunar cycles: Could we be missing a major component of the biological pump? Limnol. Oceanogr. 2010 , 55 , 2503–2512. [ Google Scholar ] [ CrossRef ]
• Anderson, M.J. Permutational Multivariate Analysis of Variance. Dept. Stat. Univ. Auckland 2005 , 26 , 32–46. [ Google Scholar ] [ CrossRef ]
• Clarke, K.R.; Somerfield, P.J.; Gorley, R.N. Dispersion-based weighting of species counts in assemblage analyses. Mar. Ecol. Prog. Ser. 2006 , 320 , 11–27. [ Google Scholar ] [ CrossRef ]
• Clarke, K.R.; Gorley, R.N. PRIMER: Getting Started with v6 ; PRIMER-E Ltd.: Plymouth, UK, 2005; pp. 931–932. [ Google Scholar ]
• Paul, W.L.; Anderson, M.J. Causal modeling with multivariate species data. J. Exp. Mar. Biol. Ecol. 2013 , 448 , 72–84. [ Google Scholar ] [ CrossRef ]
• Hölker, F.; Wolter, C.; Perkin, E.K.; Tockner, K. Light pollution as a biodiversity threat. Trends Ecol. Evol. 2010 , 25 , 681–682. [ Google Scholar ] [ CrossRef ]
• Rich, C.; Longcore, T. (Eds.) Ecological Consequences of Artificial Night Lighting ; Island Press: Washington, DC, USA, 2013. [ Google Scholar ]
• Lieberman, D.A. Learning: Behavior and Cognition ; Thomson Brooks/Cole Publishing Co.: Devon, UK, 1993. [ Google Scholar ]
• Lynch, D.T.; Magoulick, D.D. Effects of pulse and press drying disturbance on benthic stream communities. Freshw. Sci. 2016 , 35 , 998–1009. [ Google Scholar ] [ CrossRef ]
• Marrone, A.; Mangano, M.C.; Deidun, A.; Berlino, M.; Sarà, G. Effects of habitat fragmentation of a Mediterranean marine reef on the associated fish community: Insights from biological traits analysis. J. Mar. Sci. Eng. 2023 , 11 , 1957. [ Google Scholar ] [ CrossRef ]
• Borland, H.P.; Gilby, B.L.; Henderson, C.J.; Leon, J.X.; Schlacher, T.A.; Connolly, R.M.; Pittman, S.J.; Sheaves, M.; Olds, A.D. The influence of seafloor terrain on fish and fisheries: A global synthesis. Fish Fish. 2021 , 22 , 707–734. [ Google Scholar ] [ CrossRef ]
• Nagelkerken, I.; van der Velde, G. A comparison of fish communities of subtidal seagrass beds and sandy seabeds in 13 marine embayments of a Caribbean island, based on species, families, size distribution, and functional groups. J. Sea Res. 2004 , 52 , 127–147. [ Google Scholar ] [ CrossRef ]
• Clark, B.M.; Bennett, B.A.; Lamberth, S.J. Temporal variations in surf zone fish assemblages from False Bay, South Africa. Mar. Ecol. Prog. Ser. 1996 , 131 , 35–47. [ Google Scholar ] [ CrossRef ]
• Becker, A.; Whitfield, A.K.; Cowley, P.D.; Järnegren, J.; Næsje, T.F. Potential effects of artificial light associated with anthropogenic infrastructure on the abundance and foraging behaviour of estuary-associated fishes. J. Appl. Ecol. 2013 , 50 , 43–50. [ Google Scholar ] [ CrossRef ]
• Marchesan, M.; Spoto, M.; Verginella, L.; Ferrero, E.A. Behavioural effects of artificial light on fish species of commercial interest. Fish. Res. 2005 , 73 , 171–185. [ Google Scholar ] [ CrossRef ]
• Utne-Palm, A. Visual feeding of fish in a turbid environment: Physical and behavioural aspects. Mar. Freshw. Behav. Physiol. 2002 , 35 , 111–128. [ Google Scholar ] [ CrossRef ]
• Hunter, J.R. Effects of light on schooling and feeding of jack mackerel, Trachurus symmetricus . J. Fish. Board Can. 1968 , 25 , 393–407. [ Google Scholar ] [ CrossRef ]
• Sardo, G.; Okpala, C.O.R.; Geraci, M.L.; Fiorentino, F.; Vitale, S. The effects of different artificial light wavelengths on some behavioural features of juvenile pelagic Atlantic horse mackerel, Trachurus trachurus (Actinopterygii: Perciformes: Carangidae). Acta Ichthyol. Piscat. 2020 , 50 , 85–92. [ Google Scholar ] [ CrossRef ]
• Hughes, N.F.; Dill, L.M. Position choice by drift-feeding salmonids: Model and test for Arctic grayling ( Thymallus arcticus ) in subarctic mountain streams, interior Alaska. Can. J. Fish. Aquat. Sci. 1990 , 47 , 2039–2048. [ Google Scholar ] [ CrossRef ]
• Bussotti, S.; Di Franco, A.; Pey, A.; Vieux-Ingrassia, J.V.; Planes, S.; Guidetti, P. Distribution patterns of marine cave fishes and the potential role of the cardinal fish Apogon imberbis (Linnaeus, 1758) for cave ecosystem functioning in the western Mediterranean. Aquat. Living Resour. 2017 , 30 , 15. [ Google Scholar ] [ CrossRef ]
• Rastorgueff, P.A. Social flexibility to balance habitat fragmentation? Insights from the Mediterranean cave-dwelling cardinalfish Apogon imberbis . Mar. Ecol. 2020 , 41 , e12581. [ Google Scholar ] [ CrossRef ]
• Sykes, A.V.; Quintana, D.; Andrade, J.P. The effects of light intensity on growth and survival of cuttlefish (Sepia officinalis) hatchlings and juveniles. Aquac. Res. 2014 , 45 , 2032–2040. [ Google Scholar ] [ CrossRef ]
• Rosa, M.R.; Santana, E.F.C.D.; Farias, G.M.D.; Sumida, P.Y.G.; Francini Filho, R.B. Abundance of the bearded fireworm Hermodice carunculata (Polychaeta: Amphinomidae) increases across a euphotic-mesophotic depth gradient in the remote St. Peter and St. Paul’s archipelago. Ocean Coast. Res. 2022 , 71 , e23009. [ Google Scholar ] [ CrossRef ]
• Crymble, J. Guidelines for Ecologically Responsible Lighting. 2020. Available online: https://birdlifemalta.org/wp-content/uploads/2020/07/Guidelines-for-Ecologically-Responsible-Lighting.pdf (accessed on 1 September 2023).
• Reilly, C.E.; Larson, J.; Amerson, A.M.; Staines, G.J.; Haxel, J.H.; Pattison, P.M. Minimizing Ecological Impacts of Marine Energy Lighting. J. Mar. Sci. Eng. 2022 , 10 , 354. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Grillo, F.; Marrone, A.; Gauci, A.; Deidun, A. Maltese Coastline Never Sleeps: The Effects of Artificial Light at Night (ALAN) on the Local Infralittoral Assemblages—A Case Study. J. Mar. Sci. Eng. 2024 , 12 , 1602. https://doi.org/10.3390/jmse12091602

Grillo F, Marrone A, Gauci A, Deidun A. Maltese Coastline Never Sleeps: The Effects of Artificial Light at Night (ALAN) on the Local Infralittoral Assemblages—A Case Study. Journal of Marine Science and Engineering . 2024; 12(9):1602. https://doi.org/10.3390/jmse12091602

Grillo, Francesca, Alessio Marrone, Adam Gauci, and Alan Deidun. 2024. "Maltese Coastline Never Sleeps: The Effects of Artificial Light at Night (ALAN) on the Local Infralittoral Assemblages—A Case Study" Journal of Marine Science and Engineering 12, no. 9: 1602. https://doi.org/10.3390/jmse12091602

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Case report
• Open access
• Published: 10 September 2024

Long-term TNF-alpha therapy for preserving beta cell function in new onset type 1 diabetes: a case report

• Adya Rao   ORCID: orcid.org/0000-0003-2181-5180 1 ,
• Lauren M Quinn 1 , 2 &
• Parth Narendran 1 , 2  

Clinical Diabetes and Endocrinology volume  10 , Article number:  26 ( 2024 ) Cite this article

24 Accesses

1 Altmetric

Metrics details

Type 1 diabetes mellitus (T1D) is an autoimmune disease caused by destruction of pancreatic islet beta-cells. There is significant residual beta-cell function, measured through circulating C-peptide, present at the time of T1D diagnosis but this subsequently decreases with time. Higher residual beta-cell function at diagnosis associates with better glycaemic control and less glucose variability, and later in the disease course with less hypoglycaemia, lower glucose variability and fewer microvascular complications. There is therefore value in preserving residual beta cell function in new onset T1D Immunotherapeutic agents can protect residual beta-cell function in type 1 diabetes. However, clinical trials of such agents, whilst demonstrating C-peptide preservation in short term studies, have yet to be taken forward into routine clinical care due to concerns around safety and long-term efficacy. Here we report the case of a gentleman with newly diagnosed T1D whose glycaemic control and insulin requirement improved whilst on a five year infusion programme of infliximab, a monoclonal antibody against TNF-alpha, for colitis.

Case presentation

A 52-year-old White Caucasian man was diagnosed with T1D in August 2018. Glucose was 25.6 mmol/L, HbA1c was 98mmol/mol and GAD antibodies were strongly positive. HbA1c marginally improved to 91mmol/mol following initiation of insulin detemir 5 units at night and 1:10 g of insulin aspart (November 2018). In June 2019, he developed rectal bleeding and abdominal pain. Following colonoscopy, he was diagnosed with “indeterminate colitis” and commenced on 6-weekly infusions of 400-450 mg infliximab. Thus far, he has received 32 doses and achieved colitis remission. Following infliximab initiation there was increased frequency of mild-moderate hypoglycaemia and he was gradually weaned off and discontinued detemir in June 2020. Since then, HbA1c improved from 57mmol/mol in August 2019 to 52mmol/mol in April 2022, remaining stable at 51mmol/mol. His most recent HbA1c is 54mmol/mol in February 2024. His c-peptide was 550pmol/L in October 2022 and 442pmol/L in February 2024, suggesting well-preserved beta-cell function almost 6 years post-diagnosis.

Conclusions

Our patient’s improvement in glycaemic control can be explained by immunomodulation and C peptide preservation from infliximab. With the growing focus on type 1 diabetes disease modulation and working towards an ‘insulin free T1D’, our findings strengthen the evidence base for the repurposing of and long-term treatment with anti-TNF-α agents to preserve beta-cell function in new onset T1D.

Type 1 diabetes (T1D) results from the autoimmune destruction of pancreatic-islet insulin-secreting beta cells [ 1 ]. Whilst characterised by significant beta cell loss, some residual beta cell function remains at diagnosis and this subsequently falls off with continued autoimmune attrition [ 2 , 3 , 4 ]. Beta cell function is measured as circulating C peptide, a fragment of the insulin precursor molecule. C peptide levels in adults newly diagnosed with T1D can be over 1000pmol/L, falling to less than 200pmol/L within three years of diagnosis [ 5 ]. Residual beta cell function at diagnosis associates with appreciable clinical benefits. The benefits include lower glucose levels and less glucose variability near the time of diagnosis. Later in the course of disease, residual beta cell function associates with less hypoglycaemia, lower glucose variability and fewer microvascular complications such as retinopathy, neuropathy, and nephropathy [ 6 ]. There is therefore a strong clinical argument for preserving residual beta cell function in people newly diagnosed with T1D [ 5 ].

Immune modulating therapies can preserve beta cell function if administered sufficiently early after T1D diagnosis [ 7 ]. A mature immune response against the pancreatic islet involves different cell types and immune pathways and modulating this potentially attenuates beta cell destruction [ 3 ]. Tumour necrosis factor alpha (TNF-α) is a cell-cell signalling molecule, a cytokine, involved in immune cell survival and proliferation. Inhibiting TNFa action prevents T1D in animal models [ 8 ]. Two studies in patients with newly diagnosed T1D have also been promising. A 2009 pilot randomised controlled trial (RCT) of etanercept in children with new onset T1D demonstrated improved residual beta cell function (measured as C-peptide) and glucose control (measured as glycated haemoglobin HbA1c) [ 9 ]. Etanercept acts as a soluble TNF receptor, binding TNF-α and inhibiting its action [ 10 ]. More recently golimumab administration in children and young adults with newly diagnosed T1D resulted in increased endogenous insulin production and decreased exogenous insulin use [ 11 ]. Golimumab is a humanised monoclonal antibody to TNFa thus blocking the biological activity this cytokine.

Whilst promising, these studies of anti-TNF therapies are of short duration and therefore provide limited information on long-term outcomes. Here we describe a patient who developed inflammatory bowel disease (IBD) requiring treatment with infliximab, a chimeric monoclonal antibody against TNF-α [ 12 ] around the time of developing T1D. He continued on infliximab long-term, thus providing insight into the long-term acceptability and efficacy of anti-TNF therapy in T1D.

A 52-year-old Caucasian man presented in August 2018 to primary care with polydipsia, polyuria and approximately 3 kg of weight loss. He had a history of autoimmune disease, including uveitis and ankylosing spondylitis, and atopy (eczema and asthma). At presentation, his glucose was 25.6mmol/L, HbA1c was 98mmol/mol and he had strongly positive GAD antibodies of 1321.6IU/ml. He was initially treated with metformin 500 mg BD and gliclazide 40 mg OD, but due to persistently elevated blood sugars (11–15 mmol/L) he was referred to the diabetes specialist clinic in October 2018, where his HbA1c was found to have increased to 118mmol/mol. Here, a diagnosis of T1D was made and he was commenced on detemir 5 units at night and insulin aspart titrated to 1 unit for every 10 g of carbohydrates. His HbA1c marginally improved to 91 mmol/mol in November 2018.

In May 2019, nine months after presentation with T1D, he developed intermittent bloody diarrhoea and abdominal pain. A colonoscopy showed patchy, moderate acute and chronic inflammation in the rectum, and patchy acute inflammation and focal ulceration in the colon, with no evidence of active inflammation, dysplasia or neoplasia in the ileum. His faecal calprotectin was 1939ug/g and he had a negative Quantiferon TB assay. A diagnosis of indeterminate colitis was made and he was treated initially with IV hydrocortisone and then a reducing dose of oral prednisolone. He required 3 doses of rescue therapy with infliximab and was commenced on mesalazine 2.4 g BD on discharge. Shortly after his first acute colitis flare, he was commenced on azathioprine 75 mg OD and 6-weekly infliximab infusions with his first outpatient infusion administered in July 2019. He responded very well to his colitis treatment and achieved clinical remission in October 2019 (CRP 2 and faecal calprotectin 103ug/g).

Over the course of the next year, during which he was on regular infliximab infusions, he experienced increased frequency of mild-moderate hypoglycaemia. As such, his insulin requirement decreased, with detemir gradually reduced until it was no longer required, while insulin aspart was continued at 1:10 g until present.

Azathioprine was eventually stopped in August 2022 because the patient developed a superficial BCC [ 13 ] and mesalazine was stopped in January 2023 due to an itchy rash on the torso, face and buttock. The frequency of infliximab infusions was decreased from 6-weekly to 8-weekly in September 2021. To date he has received 33 infliximab infusions of 400-450 mg (excluding those for his first colitis flare as an inpatient). He continues to receive infliximab infusions 8-weekly, with his last infusion being on 11/01/2024. He stated that these infusions are “easy to have” as he “is only in there for an hour” and he has “had no side effects from them” thus far.

Figure 1 shows the timeline of his infliximab infusions and his glucose control. His C peptide was 550pmol/L in October 2022 and 442pmol/L in February 2024.

Timeline of glucose control and infliximab infusions. HbA1c at diagnosis in August 2018 was 98mmol/mol, after which he was treated with metformin and gliclazide. Despite treatment, HbA1c rose to 118mmol/mol in October 2018, after which he was commenced on detemir and insulin aspart. HbA1c dropped slightly to 91mmol/mol in November 2018 on this treatment. The next available HbA1c is 57mmol/mol in August 2019, having received 3 infliximab infusions for his acute colitis flare in July 2019. His most recent HbA1c is 54mmol/mol in February 2024

Discussion and conclusions

We describe a case of adjunctive treatment of new-onset T1D with anti-TNF therapy which resulted in lower insulin requirements, better glucose control and preserved C peptide over a 5-year period. Our case supports the findings of the two short-term RCTs previously alluded to [ 9 , 11 ] as well as that of another case report where treatment with infliximab resulted in resurgence of insulin secretion and lower insulin requirements over a one-year period [ 14 ]. These studies are summarised in an additional file (Supplementary Table 1 , Additional File 1 ). Our case however goes further to demonstrate that these benefits can persist long-term, and that the treatment can be acceptable without undue side-effects.

The main strength of our study is the long-term follow up of the case and available documentation of the timeline of insulin and anti-TNF therapies. It is weakened by the lack of sequential C peptides, including a C peptide measure at diagnosis with T1DM.

Our patient’s C-peptide was 442pmol/L in February 2024, 5.5 years post-T1D diagnosis which is unusually high for his stage of disease [ 15 ]. This suggests that infliximab’s benefit was mediated through beta cell preservation. Our findings demonstrate the protective effect of repeated infliximab infusions over an extended period (over 5 years) on glycaemic control and insulin requirement in newly diagnosed T1D. On an individual level, these benefits translate to a reduction in the practical burdens associated with insulin such as continuous monitoring, multiple injections and risk of hypoglycaemia. A reduction in insulin requirements also facilitates maintenance of healthy weight and associated reduction in cardiovascular risk. Furthermore, higher C peptide levels early in T1D associates with lower rates of long-term vascular complications [ 16 ].

The fall in insulin requirements after initiation of infliximab therapy is interesting and may be mediated through either recovery of beta cell function or increased insulin sensitivity. There is some evidence that TNF mediates resistance to the actions of insulin [ 17 ], and therefore anti-TNF therapy would potentially increase insulin sensitivity and reduce insulin requirements [ 14 ]. Insulin sensitivity was not formally measured in our case, and it is therefore difficult to estimate its contribution to the reduced insulin requirements.

On a broader scale, our findings support the case for immunotherapy for beta cell preservation in newly diagnosed T1D. Several RCTs have demonstrated the efficacy of different immunotherapeutic agents for preserving beta cell function in new onset T1D [ 18 ], but these have yet to be taken forward to clinical practice. These studies have all been less than two-year duration and long-term follow up data is difficult to obtain in a RCT setting. Therefore, case reports such as these help understand the role of these therapies in clinical care.

Immunotherapy to preserve beta cells, if administered before the requirement for insulin and when there is appreciable beta cell function, can delay symptomatic T1D. Such therapies are now becoming licensed – tepelizumab, an anti-CD3 antibody, was licensed in the United States in November 2022 for individuals aged over 8 years with stage 2 type 1 diabetes [ 3 , 19 ], and it is currently undergoing NICE consultation in the UK [ 20 ]. Our case report suggests than in addition to immunoprevention, immunotherapy at diagnosis with symptomatic T1D (Stage 3) [ 3 ] may also be worth pursuing.

Data availability

The dataset(s) supporting the conclusions of this article is(are) included within the article.

Holt RIG, DeVries JH, Hess-Fischl A, Hirsch IB, Kirkman MS, Klupa T, et al. The management of type 1 diabetes in adults. A Consensus Report by the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2021;44(11):2589–625.

Article   CAS   PubMed   Google Scholar  

Palmer JP. C-peptide in the natural history of type 1 diabetes. Diabetes Metab Res Rev. 2009;25(4):325–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dayan CM, Korah M, Tatovic D, Bundy BN, Herold KC. Changing the landscape for type 1 diabetes: the first step to prevention. Lancet. 2019;394(10205):1286–96.

Shields BM, McDonald TJ, Oram R, Hill A, Hudson M, Leete P, et al. C-Peptide decline in type 1 diabetes has two phases: an initial exponential fall and a subsequent stable phase. Diabetes Care. 2018;41(7):1486–92.

VanBuecken DE, Greenbaum CJ. Residual C-peptide in type 1 diabetes: what do we really know? Pediatr Diabetes. 2014;15(2):84–90.

Carr ALJ, Oram RA, Marren SM, McDonald TJ, Narendran P, Andrews RC. Measurement of peak C-Peptide at diagnosis informs Glycemic Control but not hypoglycemia in adults with type 1 diabetes. J Endocr Soc. 2021;5(10):bvab127.

Allen LA, Dayan CM. Immunotherapy for type 1 diabetes. Br Med Bull. 2021;140(1):76–90.

Yang XD, Tisch R, Singer SM, Cao ZA, Liblau RS, Schreiber RD, et al. Effect of tumor necrosis factor alpha on insulin-dependent diabetes mellitus in NOD mice. I. The early development of autoimmunity and the diabetogenic process. J Exp Med. 1994;180(3):995–1004.

Mastrandrea L, Yu J, Behrens T, Buchlis J, Albini C, Fourtner S, et al. Etanercept Treatment in Children with New-Onset type 1 diabetes: pilot randomized, placebo-controlled, double-blind study. Diabetes Care. 2009;32(7):1244–9.

Pan A, Gerriets V. Etanercept. StatPearls. Treasure Island (FL)2024.

Quattrin T, Haller MJ, Steck AK, Felner EI, Li Y, Xia Y, et al. Golimumab and Beta-cell function in Youth with New-Onset type 1 diabetes. N Engl J Med. 2020;383(21):2007–17.

Fatima R, Bittar K, Aziz M, Infliximab. StatPearls. Treasure Island (FL)2024.

Rollan MPC, Schwartz R. R A. Current knowledge of immunosuppression as a risk factor for skin cancer development. Crit Reviews Oncol Hematol. 2022;177(September 2022).

Timper K, Hruz P, Beglinger C, Donath MY. Infliximab in the treatment of Crohn disease and type 1 diabetes. Diabetes Care. 2013;36(7):e90–1.

Article   PubMed   PubMed Central   Google Scholar  

Lynam A, McDonald T, Hill A, Dennis J, Oram R, Pearson E, et al. Development and validation of multivariable clinical diagnostic models to identify type 1 diabetes requiring rapid insulin therapy in adults aged 18–50 years. BMJ Open. 2019;9(9):e031586.

Nathan DM, Group DER. The diabetes control and complications trial/epidemiology of diabetes interventions and complications study at 30 years: overview. Diabetes Care. 2014;37(1):9–16.

Hotamisligil GS, Arner P, Caro JF, Atkinson RL, Spiegelman BM. Increased adipose tissue expression of tumor necrosis factor-alpha in human obesity and insulin resistance. J Clin Investig. 1995;95(5):2409–15.

Rapini N, Schiaffini R, Fierabracci A. Immunotherapy strategies for the Prevention and treatment of distinct stages of type 1 diabetes: an overview. Int J Mol Sci. 2020;21(6).

Iacobucci G. Type 1 diabetes: US approves drug that can delay onset. BMJ. 2022;379:o2798.

Article   PubMed   Google Scholar  

NICE. Teplizumab for delaying the onset of type 1 diabetes in people 8 years and over at risk of developing the condition 2022. https://www.nice.org.uk/guidance/awaiting-development/gid-ta10981 . Accessed 22 February 2024.

Download references

Acknowledgements

Author information, authors and affiliations.

Department of Diabetes, Queen Elizabeth Hospital, University Hospitals of Birmingham, Birmingham, UK

Adya Rao, Lauren M Quinn & Parth Narendran

Institute of Immunology and Immunotherapy, University of Birmingham, Birmingham, UK

Lauren M Quinn & Parth Narendran

You can also search for this author in PubMed   Google Scholar

Contributions

AR, PN and LQ conceptualised the study. AR drafted the initial manuscript which was revised by LQ and PN. All authors have reviewed and approved the submitted version of the manuscript.

Corresponding author

Correspondence to Adya Rao .

Ethics declarations

Ethics approval and consent to participate.

Informed consent was obtained from the patient whose case we report, in accordance with the declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors have no competing interests to declare.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rao, A., Quinn, L.M. & Narendran, P. Long-term TNF-alpha therapy for preserving beta cell function in new onset type 1 diabetes: a case report. Clin Diabetes Endocrinol 10 , 26 (2024). https://doi.org/10.1186/s40842-024-00185-6

Download citation

Received : 14 March 2024

Accepted : 10 May 2024

Published : 10 September 2024

DOI : https://doi.org/10.1186/s40842-024-00185-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Immunotherapy
• Immunoprevention
• Type 1 diabetes
• T1D; Infliximab
• Tumour necrosis factor alpha

Clinical Diabetes and Endocrinology

ISSN: 2055-8260

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.