Copyright © 2021 pubrica. All rights reserved 1

How To Evaluate Bias In Meta-Analysis

Within Meta-Epidemiological Studies?

Dr. Nancy Agnes, Head, Technical Operations, Pubrica, sales@pubrica.com

URL structural:

https://pubrica.com/academy/meta-

analysis/how-many-patients-does-case-series-

should-have-in-comparison-to-case-reports-

case-reporting/

Keywords:Meta-analysis, quantitative research,

data extracted, data collection, meta-analysis.

Meta title:How to evaluate bias in meta-analysis

within meta-epidemiological studies?

Meta description:

A meta-analysis may give a precise estimate of

average bias, rather than an estimate of the

intervention’s effect. Meta-analysis has the

potential to be a powerful tool in

evaluating health care treatments and

interventions, there are many potential pitfalls

and problems that are yet to be resolved.

I. INTRODUCTION

Meta-analysis is a type of statistical approach which

synthesizes results from different studies and the final

result serves as a much stronger evidence than the one

collected from an individual study. It gives an estimate

of the success of a newly introduced treatment/

intervention or the risk factors associated with a disease/

line of treatment(Hayden et al., 2021). Thus, it can serve

as the best source for evidence-based clinical studies.

The studies used in meta-analysis can combine results

from systematic review, randomised controlled trials

(RCT) etc. Meta epidemiological studies is a new type

of method which helps in closing the gap between trials

and practice and is a much improved version of

systematic review(Page, 2020). They adopt either

systematic review or meta-analysis approach and aims

to understand the impact of certain factors on the

outcome. Thus, they try to confirm or nullify the

hypothesis in question. The object of analysis is a study

and not a patient or an individual. Results of meta-

epidemiological study might be directly related to

exposure but can also be a result of an alternative effect

that might have impacted the overall study outcome.

These alternative effects can be a random error, a bias

that can produce incorrect results(Steenland et al.,

2020). Due to these effects, sometimes an association is

falsely accounted for in the outcome when it is not

present and on the other hand, sometimes an association

is overlooked even in its presence.

II. BIAS IN META-ANALYSIS WITHIN META-

EPIDEMIOLOGICAL STUDIES

In some meta epidemiological studies, the effect of

interventions in RCT’s (Randomised Controlled Trials)

can be misunderstood leading to underestimation or

overestimation of the intervention (Christensen and

Berthelsen, 2020). There can be several reasons which

have been elaborated bellow-

Bias arising due to randomisation- The procedure of

sequence generation or allocation concealment might

vary the effects of the introduced interventions. These

two factors also affects the in between heterogeneity.

Bias arising due to opting for unintended interventions-

This type of bias arises when the participant opts for an

intervention different from which they have been

randomly allotted for.

Bias arising due to lack of proper outcome data- The

exaggeration of the intervention effect can arise when

the data of outcomes are either not completely/ falsely

reported. There are some examples when there is

overestimation and underestimation of the intervention

effect even when the outcome has been properly

recorded This is caused due to attrition, but the average

bias reported due to attrition could not be combined as

the definition of attrition differs across studies.

Bias arising due to improper result selection- There has

been reports of bias when the outcomes are not properly

generated due to discrepancies between results and

methods.

Bias arising due to incorrectly measuring outcomes-

Due to lack of proper outcome accessors, bias arises in

properly measuring the outcomes. This results in

improper estimation of intervention effects.

In most meta-epidemiological studies, a written protocol

for selecting the studies need to be framed before

conducting the meta analysis. It is important to include

all the related studies as missing out on one can

https://pubrica.com/services/research-services/meta-analysis/https://pubrica.com/services/research-services/meta-analysis/https://pubrica.com/academy/journal-selection/top-journals-to-publish-your-pubic-health-epidemiology-manuscripts-based-on-acceptance-and-ranking-factors/https://pubrica.com/services/research-services/meta-analysis/

Copyright © 2021 pubrica. All rights reserved 2

introduce bias and makes the study less effective (Pan et

al., 2020). The protocol must focus on the selection

criteria (eligibility criteria, type of studies to be

included, etc.) of the studies to reduce section bias. Fig

1 depicts a flowchart of selecting studies.

Fig 1: Flowchart for selection of studies

Alongside these, the other important points to be

included in the protocol are objectives of the study,

hypothesis to be tested etc(Steenland et al., 2020).

According to some authors, it can be quite tricky to

combine different study designs of meta-

epidemiological studies in a meta-analysis and thus

have stated “a meta-analysis may give a precise estimate

of average bias, rather than an estimate of the

intervention’s effect” and that “heterogeneity between

study results may reflect differential biases rather than

true differences in an intervention’s effect”.In order to

understand the amount of bias that might have impacted

the study outcome, it has been unanimously agreed

upon that all the non-randomized and observational

studies included in the meta-analysis should be

assessed(Puljak et al., 2020). But there has been no

proper agreement on the guidelines of assessing the risk

of bias in different meta-analyses(Mathur and

VanderWeele, 2021).

Meta epidemiological studies helps in overcoming the

challenges of systematic reviews. Out of all, it focuses

to get rid of publication bias. Publication bias is also an

important type of bias that stresses upon the fact that the

data used inmeta-epidemiological studies should also be

drawn upon from unpublished study sources(Lin, 2020).

It is sometimes observed that few studies are not

accepted for publishing as they report negative results.

Thus, missing out on these can enhance the risk of bias

and can give a false impression about the effectiveness

of the interpretation(Tan et al., 2021).

III. CONCLUSION

The bias which arises during different steps of the meta-

analysis must be addressed as this might report

contradictory results. It must be noted that false reports

can impact medical research which can be fatal in few

aspects. The problem with meta-epidemiological study

lies in the fact that when the number of studies reduces,

the statistical power also reduces.

References

1. Hayden, J.A., Ellis, J., Ogilvie, R., Boulos, L., Stanojevic, S., 2021. Meta-epidemiological study of publication

integrity, and quality of conduct and reporting of

randomized trials included in a systematic review of low

back pain. J. Clin. Epidemiol. 134, 65–78.

https://doi.org/10.1016/j.jclinepi.2021.01.020

2. Lin, L., 2020. Hybrid test for publication bias in meta-analysis. Stat. Methods Med. Res. 29, 2881–2899.

https://doi.org/10.1177/0962280220910172

3. Mathur, M.B., VanderWeele, T.J., 2021. Estimating publication bias in meta-analyses of peer-reviewed studies:

A meta-meta-analysis across disciplines and journal tiers.

Res. Synth. Methods 12, 176–191.

https://doi.org/10.1002/jrsm.1464

4. Page, M.J., 2020. Controversy and Debate on Meta-epidemiology. Paper 4: Confounding and other concerns in

meta-epidemiological studies of bias. J. Clin. Epidemiol.

https://doi.org/10.1016/j.jclinepi.2020.03.022

5. Puljak, L., Makaric, Z.L., Buljan, I., Pieper, D., 2020. What is a meta-epidemiological study? Analysis of published

literature indicated heterogeneous study designs and

definitions. J. Comp. Eff. Res. 9, 497–508.

https://doi.org/10.2217/cer-2019-0201

6. Steenland, K., Schubauer-Berigan, M.K., Vermeulen, R., Lunn, R.M., Straif, K., Zahm, S., Stewart, P., Arroyave,

W.D., Mehta, S.S., Pearce, N., 2020. Risk of bias

assessments and evidence syntheses for observational

epidemiologic studies of environmental and occupational

exposures: Strengths and limitations. Environ. Health

Perspect. https://doi.org/10.1289/EHP6980

7. Tan, R.Q.X., Li, W.T.V., Shum, W.Z., Chu, S.C., Li, H.L., Shea, Y.F., Chung, T.W.H., 2021. A systematic review and

meta-analysis protocol examining the clinical characteristics

and epidemiological features of olfactory dysfunction (OD)

in coronavirus disease 2019 (COVID-19). Syst. Rev. 10, 1–

6. https://doi.org/10.1186/s13643-021-01624-6

8. Christensen, R., & Berthelsen, D. B. (2020). Controversy

https://pubrica.com/academy/research/epidemiology-designs-for-clinical-trials/https://pubrica.com/academy/research/epidemiology-designs-for-clinical-trials/https://pubrica.com/services/research-services/meta-analysis/https://pubrica.com/services/research-services/meta-analysis/https://pubrica.com/services/research-services/meta-analysis/

Copyright © 2021 pubrica. All rights reserved 3

and Debate on Meta-epidemiology. Paper 3: Causal

inference from meta-epidemiology: a reasonable goal, or

wishful thinking?. Journal of clinical epidemiology, 123,

131.

9. Pan, Y. C., Chiu, Y. C., & Lin, Y. H. (2020). Systematic review and meta-analysis of epidemiology of internet

addiction. Neuroscience & Biobehavioral Reviews.