Step 5. Statistics
The field of meta-statistics is a rich and fascinating set of tools, literature, and discussions. Herein, we direct the practioner to several extremely useful resources associated with this specific topic. To become competent with an open science worflow for meta-analysis, a detailed knowledge of the statistics is of course preferable but not critical. A viable philosophy for the appropriate application of meta-statistics is best defined by the following set of assumptions. (1) Know thy ‘derived’ data and the limitations of the set of research summarized. This informs the statistics. (2) All statistics are a tool and a means to end - not the end. (3) The formal application of statistics to data from the t-test to the most advanced analysis you can imagine is an exercise in model fitting. A model is a representation of an idea. Period. Statistics are thus one mechanism that provides the evidence-based synthesis researcher with a tool to represent the ideas that stimulated the synthesis. No more, no less. (4) Description of the patterns across a set of studies or datasets is a powerful discovery tool. Do not underestimate the power of… this side of synthesis (or the force). (5) A meta-analysis is not an experiment and does not directly test a hypothesis. It can provide quantitative evidence and probabilities associed with strength of evidence, differences, and consistency and establish a weighted description of the efficacy of different interventions very broadly. (6) Even more profoundly and not uncommonly, meta-analysis as an open science synthesis research tool can highlight research gaps. (7) An ideal starting point is to assume that simple meta-statistics are akin to an ANOVA but for a population of relatively-independent singular data points. This constellation of points can be modeled as fixed or random. More advanced meta-analyses or differences in the type of data simply shift the perspective from ANOVA to reframing as a GLM (same thing) but then slides along to GLMM and this broader family of thinking. Not dissimilar from primary data statistical thinking. The difference is that we do not have one ‘data’ but ‘many representative data’. Very Seussian. There are four primary processes associated with the statistics step of an open science approach to meta-analysis.
Key meta-statistical processes (a) calculate effect size(s)
(b) visualize effect size data (typically a forest plot)
(c) fit a meta-statistical model
(d) explore heterogeneity and bias
Open science product
A forest plot is typically the primary open science research product and assumed to be included in some form within the synthesis publication as a key figure. Ideally, contemporary open science researcher should also publicly and openly disseminate the statistical workflow - i.e. share R code illuminating this element of the workflow and ensuring the meta-statistics like the search and sort processes are reproducible.
Case study
The book Meta-analysis with R provides an extensive set of worked examples including data and R code. The packages ‘meta’ and ‘metafor’ are also bundled with sample datasets. To promote continuity, here are some of the examples from previous steps provided as an indication of the ease of meta-analysis with R. It is not the implementation of the code that limits us in doing meta-analyses but the access to data. The package meta (and metafor) provide most meta-statistical options as functions or arguments within broader functions for all the data differences and data structures that an environmental or natural science reseacher will encounter.
The pattern of steps you will see below in code chunk include the following:
data (read in and tidy up as needed) assign meta-model to object viz examine model heterogeneity bias sensitivity
#library setup####
library(tidyverse)
library(plotrix) #for quick s.e. calculations sometimes needed for data tidy step
library(meta) #nice package for most meta-statistics
#installation as needed
#pkgs <- c("mada", "meta", "metafor", "metasens", "mvmeta", "netmeta", "rmeta", "ellipse") #recommended install for almost every meta-challenge you will encounter, most parsonimous install is 'meta'
#install.packages(pkgs, repos="http://cran.rstudio.com/") #run once per machine not per instance
#R-STEPS REMINDERRRR####
#data
#assign model
#viz
#examine model
#heterogeneity
#bias
#sensitivity
#CASE 1. Effect sizes####
#if reported within primary studies, you could be ready to go already. If not, the package meta provides a wide variety of standard effect size measures provided you have the data.
#data (read, tidy, and structure as needed)
cushions <-read_csv("data/cushion-data.csv") #dataset that captured effect sizes directly from plant interaction studies
data <- cushions %>% group_by(Study) %>% summarise(mean.Rii = mean(Rii), error = std.error(Rii))
#assign model (typically a nice meta. function from one of several packages such as meta, metafor, or netmeta)
m <- metagen(mean.Rii, error, studlab = Study, data = data) #fit generic meta-analysis to an object
#viz (draw a standard forest plot or metaregression plot)
forest(m) #grid-based graphics so a bit of work to resize
#examine model (print meta-statistical model you fit)
m #By default, the DerSimonian-Laird estimate (1986) is used in the random effects model (method.tau="DL"), however this argument change be changed to the statistical norm for your field or specific data. another popular method in ecology is the Restricted maximum-likelihood estimator (method.tau="REML")## 95%-CI %W(fixed) %W(random)
## 1 0.0117 [-0.1031; 0.1266] 9.4 7.2
## 2 NA 0.0 0.0
## 3 0.6613 [ 0.4139; 0.9088] 2.0 6.6
## 4 0.3801 [ 0.2362; 0.5240] 6.0 7.1
## 5 0.3236 [ 0.0996; 0.5476] 2.5 6.7
## 6 0.0455 [-0.1331; 0.2241] 3.9 7.0
## 7 0.3589 [ 0.2737; 0.4441] 17.0 7.3
## 8 -0.5235 [-0.8837; -0.1632] 1.0 5.8
## 9 -0.2980 [-0.3893; -0.2068] 14.8 7.3
## 10 0.2326 [-0.0046; 0.4698] 2.2 6.6
## 11 0.2165 [ 0.0663; 0.3668] 5.5 7.1
## 12 0.1382 [-0.7785; 1.0548] 0.1 2.7
## 13 -0.2330 [-0.4053; -0.0608] 4.2 7.0
## 14 0.6497 [ 0.5730; 0.7264] 21.0 7.3
## 15 0.2473 [ 0.1042; 0.3905] 6.0 7.1
## 16 0.8528 [ 0.6854; 1.0203] 4.4 7.0
##
## Number of studies combined: k = 16
##
## 95%-CI z p-value
## Fixed effect model 0.2555 [0.2203; 0.2906] 14.24 < 0.0001
## Random effects model 0.2138 [0.0263; 0.4013] 2.23 0.0254
##
## Quantifying heterogeneity:
## tau^2 = 0.1230; H = 5.05 [4.38; 5.82]; I^2 = 96.1% [94.8%; 97.1%]
##
## Test of heterogeneity:
## Q d.f. p-value
## 382.76 15 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - DerSimonian-Laird estimator for tau^2#heterogeneity (inspect above model Qstats)
#This is listed in model output. From 'meta-analysis with r' book for the 'meta' package for instance, Q is the weighted sum of squares about the fixed effect estimate. Large values of Q indicate greater heterogeneity between the individual studies in a meta-analysis, and greater values of the between-study heterogeneity. If p < 0.05, there is 'significant' heterogeneity from study-to-study. Nonetheless in this context, you must contrast the fixed versus random-effects models and the sign of the study-level effect sizes measures to conclude whether the treatment is effective and/or consistent.
#bias (explore)
funnel(m) #plot to explore distribution of sign and magnitude of the effect size measures, asymmetry suggests bias in the field of research
metabias(m, method="rank") #if p < 0.05, there is likely plot asymmetry##
## Rank correlation test of funnel plot asymmetry
##
## data: m
## z = -0.44538, p-value = 0.656
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
## ks se.ks
## -9.00000 20.20726radial(m) #also an excellent means to qualitatively explore bias but explores the variance associated with effect sizes
metabias(m, method = "linreg") # nice quick check of bias but inspecting residuals is more powerful##
## Linear regression test of funnel plot asymmetry
##
## data: m
## t = -0.46485, df = 13, p-value = 0.6497
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
## bias se.bias slope
## -1.4736741 3.1702460 0.3475166#the Rosenthal’s fail-safe number is also sometimes reported in the literature and can be calculated using metafor::fsn
#test sensitivity of the effect size measure
#contrast the fixed versus random effect model
#test the importance of the subgroups -- typically this is the MOST important test for bias - sensitivity
#sensitivity (critical step)
#subgroups we could explore include elevation and the measure used to estimate the effect of this potential keystone species
#elevation cofactor/subgroup
data <- cushions %>% group_by(Study, elevation) %>% summarise(mean.Rii = mean(Rii), error = std.error(Rii))
m <- metagen(mean.Rii, error, studlab = Study, byvar = elevation, data = data)
#viz
forest(m) 
#model
m## 95%-CI %W(fixed) %W(random) elevation
## 1 0.2606 [-0.0013; 0.5226] 1.8 3.9 high
## 1 0.0031 [-0.1655; 0.1718] 4.3 4.2 low
## 1 -0.1407 [-0.3153; 0.0340] 4.0 4.2 mid
## 2 0.6449 [ 0.4798; 0.8100] 4.5 4.3 high
## 2 NA 0.0 0.0 low
## 3 0.7407 [ 0.2326; 1.2489] 0.5 2.8 high
## 3 0.4944 [-0.0674; 1.0561] 0.4 2.5 low
## 3 0.7489 [ 0.3201; 1.1778] 0.7 3.1 mid
## 4 0.6024 [ 0.3835; 0.8213] 2.5 4.1 high
## 4 0.2134 [ 0.0373; 0.3894] 3.9 4.2 low
## 5 0.3236 [ 0.0996; 0.5476] 2.4 4.0 mid
## 6 -0.1413 [-0.3924; 0.1099] 1.9 3.9 high
## 6 0.2322 [-0.0085; 0.4729] 2.1 4.0 low
## 7 0.3589 [ 0.2737; 0.4441] 16.7 4.5 mid
## 8 -0.5235 [-0.8837; -0.1632] 0.9 3.4 mid
## 9 -0.3952 [-0.6532; -0.1372] 1.8 3.9 high
## 9 -0.3873 [-0.6079; -0.1667] 2.5 4.1 low
## 9 -0.2680 [-0.3762; -0.1599] 10.4 4.4 mid
## 10 0.2326 [-0.0046; 0.4698] 2.2 4.0 mid
## 11 0.1531 [-0.1101; 0.4163] 1.8 3.9 high
## 11 0.2528 [ 0.0694; 0.4362] 3.6 4.2 low
## 12 0.1382 [-0.7785; 1.0548] 0.1 1.5 mid
## 13 -0.0020 [-0.2790; 0.2751] 1.6 3.8 high
## 13 -0.4454 [-0.6351; -0.2557] 3.4 4.2 low
## 14 0.5423 0.0 0.0 high
## 14 0.6398 0.0 0.0 low
## 14 0.7867 0.0 0.0 mid
## 15 0.4102 [ 0.2684; 0.5519] 6.0 4.3 high
## 15 0.0845 [-0.0031; 0.1721] 15.8 4.5 low
## 16 0.8528 [ 0.6854; 1.0203] 4.3 4.2 mid
##
## Number of studies combined: k = 27
##
## 95%-CI z p-value
## Fixed effect model 0.1592 [0.1244; 0.1941] 8.96 < 0.0001
## Random effects model 0.1602 [0.0258; 0.2947] 2.34 0.0195
##
## Quantifying heterogeneity:
## tau^2 = 0.1035; H = 3.64 [3.19; 4.16]; I^2 = 92.5% [90.2%; 94.2%]
##
## Test of heterogeneity:
## Q d.f. p-value
## 344.76 26 < 0.0001
##
## Results for subgroups (fixed effect model):
## k 95%-CI Q tau^2 I^2
## elevation = high 9 0.3113 [ 0.2376; 0.3850] 74.82 0.1121 89.3%
## elevation = low 9 0.0363 [-0.0218; 0.0945] 54.58 0.0446 85.3%
## elevation = mid 9 0.1837 [ 0.1297; 0.2376] 181.03 0.1745 95.6%
##
## Test for subgroup differences (fixed effect model):
## Q d.f. p-value
## Between groups 34.34 2 < 0.0001
## Within groups 310.43 24 < 0.0001
##
## Results for subgroups (random effects model):
## k 95%-CI Q tau^2 I^2
## elevation = high 9 0.2448 [ 0.0088; 0.4809] 74.82 0.1121 89.3%
## elevation = low 9 0.0247 [-0.1414; 0.1909] 54.58 0.0446 85.3%
## elevation = mid 9 0.1923 [-0.1007; 0.4854] 181.03 0.1745 95.6%
##
## Test for subgroup differences (random effects model):
## Q d.f. p-value
## Between groups 2.56 2 0.2775
##
## Details on meta-analytical method:
## - Inverse variance method
## - DerSimonian-Laird estimator for tau^2#heterogeneity
#inspect heterogeneity now using between AND within groups - very important
#bias is being explored using the byvar argument
#funnel(m) #same as whole model plot
#measure cofactor/subgroup
data <- cushions %>% group_by(Study, measure) %>% summarise(mean.Rii = mean(Rii), error = std.error(Rii))
m <- metagen(mean.Rii, error, studlab = Study, byvar = measure, data = data) #fit generic meta-analysis to an object
forest(m, sortvar=measure) 
#viz
forest(m)
#model
m## 95%-CI %W(fixed) %W(random) measure
## 1 -0.0115 [-0.0237; 0.0006] 84.5 4.9 diversity
## 1 0.0124 [-0.1056; 0.1303] 0.9 4.7 presence
## 2 NA 0.0 0.0 presence
## 3 0.6613 [ 0.4139; 0.9088] 0.2 4.0 abundance
## 4 0.4628 [ 0.2852; 0.6405] 0.4 4.4 abundance
## 4 0.1300 [-0.0072; 0.2672] 0.7 4.6 diversity
## 4 0.3827 [ 0.2261; 0.5394] 0.5 4.5 presence
## 5 0.1139 0.0 0.0 abundace
## 5 0.5906 0.0 0.0 abundance
## 5 0.2949 [ 0.0185; 0.5713] 0.2 3.8 presence
## 6 -0.0780 [-0.2249; 0.0689] 0.6 4.6 diversity
## 6 0.0537 [-0.1365; 0.2439] 0.3 4.3 presence
## 7 0.3589 [ 0.2737; 0.4441] 1.7 4.8 diversity
## 8 -0.5235 [-0.8837; -0.1632] 0.1 3.3 presence
## 9 -0.0994 [-0.1592; -0.0395] 3.5 4.9 diversity
## 9 -0.3092 [-0.4052; -0.2131] 1.4 4.8 presence
## 10 0.4857 [-0.0395; 1.0109] 0.0 2.4 abundance
## 10 0.1060 [-0.0760; 0.2881] 0.4 4.4 diversity
## 11 0.1905 [-0.0380; 0.4190] 0.2 4.1 abundance
## 11 -0.0482 [-0.2504; 0.1539] 0.3 4.3 diversity
## 11 0.2509 [ 0.0432; 0.4586] 0.3 4.2 presence
## 12 0.1382 [-0.7785; 1.0548] 0.0 1.1 presence
## 13 -0.2330 [-0.4053; -0.0608] 0.4 4.4 presence
## 14 0.6497 [ 0.5730; 0.7264] 2.1 4.8 abundance
## 15 0.2775 [ 0.0073; 0.5478] 0.2 3.8 abundance
## 15 0.2171 [ 0.0746; 0.3596] 0.6 4.6 diversity
## 16 0.8528 [ 0.6854; 1.0203] 0.4 4.4 abundance
##
## Number of studies combined: k = 25
##
## 95%-CI z p-value
## Fixed effect model 0.0147 [0.0035; 0.0259] 2.58 0.0098
## Random effects model 0.1739 [0.0627; 0.2852] 3.07 0.0022
##
## Quantifying heterogeneity:
## tau^2 = 0.0651; H = 5.07 [4.53; 5.68]; I^2 = 96.1% [95.1%; 96.9%]
##
## Test of heterogeneity:
## Q d.f. p-value
## 617.82 24 < 0.0001
##
## Results for subgroups (fixed effect model):
## k 95%-CI Q tau^2 I^2
## measure = diversity 8 -0.0055 [-0.0171; 0.0062] 96.35 0.0198 92.7%
## measure = presence 10 -0.0538 [-0.1090; 0.0015] 84.66 0.0601 89.4%
## measure = abundance 7 0.6050 [ 0.5463; 0.6637] 30.86 0.036 80.6%
## measure = abundace 0 NA -- -- --
##
## Test for subgroup differences (fixed effect model):
## Q d.f. p-value
## Between groups 405.95 2 < 0.0001
## Within groups 211.87 22 < 0.0001
##
## Results for subgroups (random effects model):
## k 95%-CI Q tau^2 I^2
## measure = diversity 8 0.0727 [-0.0348; 0.1802] 96.35 0.0198 92.7%
## measure = presence 10 0.0040 [-0.1776; 0.1855] 84.66 0.0601 89.4%
## measure = abundance 7 0.5286 [ 0.3626; 0.6947] 30.86 0.036 80.6%
## measure = abundace 0 NA -- -- --
##
## Test for subgroup differences (random effects model):
## Q d.f. p-value
## Between groups 24.27 2 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - DerSimonian-Laird estimator for tau^2#heterogeneity
#inspect heterogeneity now using between AND within groups - very important
#bias epxlored via the byvar argument
#funnel(m) #same as whole model plot
#and yes, you can do meta-statistics like a GLM with interaction terms and also predictions
#also if data structure and the dataframe are consistent, doing sensitivity analyses can be very rapid using the 'update' global function in R
#there is also a 'metareg' function in the meta package do meta-regression if for instance you test elevation in this example using actual elevation of each study in m and not as a category
#One more quick example to highlight the ease of meta-statistics in R
#CASE 2. Continuous response data
#Revisit dune meta-analysis example
#data
dunes.data <- read_csv("data/dune-growthLRR.csv")
#assign model
m <- metacont(Ne, MEANe, SDe, Nc, MEANc, SDc, studlab = study, data=dunes.data)#metacont is used for continuous data with arguments in this order within call metacont(n.e, mean.e, sd.e, n.c, mean.c, sd.c, studlab = study, data=data)
#viz
forest(m)
#model
m## MD 95%-CI %W(fixed) %W(random)
## 1 1.2900 [ -0.3700; 2.9500] 0.2 2.7
## 1 0.8500 [ -0.5559; 2.2559] 0.3 3.1
## 1 -3.0800 [ -7.8639; 1.7039] 0.0 0.6
## 1 -4.1800 [ -9.3388; 0.9788] 0.0 0.5
## 6 0.6100 [ -0.0510; 1.2710] 1.5 4.4
## 6 1.6100 [ 0.6932; 2.5268] 0.8 4.0
## 6 1.1700 [ 0.5090; 1.8310] 1.5 4.4
## 6 1.5100 [ 0.9836; 2.0364] 2.4 4.6
## 6 0.0500 [ -1.5037; 1.6037] 0.3 2.9
## 6 -0.4900 [ -2.7434; 1.7634] 0.1 2.0
## 6 0.1000 [ -0.7961; 0.9961] 0.8 4.0
## 6 1.7000 [ 1.1231; 2.2769] 2.0 4.6
## 6 0.3200 [ -0.6401; 1.2801] 0.7 3.9
## 6 0.3400 [ -0.9419; 1.6219] 0.4 3.4
## 6 0.7700 [ -1.1837; 2.7237] 0.2 2.3
## 6 1.5900 [ 1.1930; 1.9870] 4.3 4.8
## 6 -0.4100 [ -1.6523; 0.8323] 0.4 3.4
## 6 0.0100 [ -1.6387; 1.6587] 0.2 2.7
## 11 0.1200 [ 0.0220; 0.2180] 70.2 5.0
## 11 0.1900 [ -0.0886; 0.4686] 8.7 4.9
## 14 1.3500 [ 0.4901; 2.2099] 0.9 4.1
## 34 -161.6000 [-210.2628; -112.9372] 0.0 0.0
## 44 7.1600 [ 1.2303; 13.0897] 0.0 0.4
## 44 11.8300 [ 3.4110; 20.2490] 0.0 0.2
## 45 1.8300 [ 0.5187; 3.1413] 0.4 3.3
## 45 4.1000 [ -0.6456; 8.8456] 0.0 0.6
## 45 -3.0800 [ -4.9817; -1.1783] 0.2 2.4
## 45 -4.0400 [ -6.1353; -1.9447] 0.2 2.1
## 45 -0.1600 [ -18.4583; 18.1383] 0.0 0.0
## 45 10.0700 [ -8.4711; 28.6111] 0.0 0.0
## 45 -22.0000 [ -36.7016; -7.2984] 0.0 0.1
## 45 -23.7000 [ -38.5458; -8.8542] 0.0 0.1
## 56 -0.6000 [ -3.1103; 1.9103] 0.1 1.7
## 59 2.3400 [ 1.0971; 3.5829] 0.4 3.4
## 59 16.4200 [ 8.2955; 24.5445] 0.0 0.2
## 59 1.1000 [ 0.3908; 1.8092] 1.3 4.4
## 76 2.4800 [ -0.1938; 5.1538] 0.1 1.6
## 76 2.3900 [ 0.7490; 4.0310] 0.3 2.8
## 76 0.0500 [ -0.9665; 1.0665] 0.7 3.8
##
## Number of studies combined: k = 39
##
## MD 95%-CI z p-value
## Fixed effect model 0.3322 [0.2501; 0.4143] 7.93 < 0.0001
## Random effects model 0.6887 [0.2834; 1.0940] 3.33 0.0009
##
## Quantifying heterogeneity:
## tau^2 = 0.8481; H = 2.70 [2.37; 3.08]; I^2 = 86.3% [82.2%; 89.5%]
##
## Test of heterogeneity:
## Q d.f. p-value
## 277.64 38 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - DerSimonian-Laird estimator for tau^2#heterogeneity
#inspect main model heterogeneity if statistically significant one needs to identify source (i.e. cofactors or subgroups)
#bias
funnel(m)
metabias(m, method="rank") #if p < 0.05, there is likely plot asymmetry##
## Rank correlation test of funnel plot asymmetry
##
## data: m
## z = -2.0686, p-value = 0.03859
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
## ks se.ks
## -171.00000 82.66599radial(m) #also an excellent means to qualitatively explore bias but explores the variance associated with effect sizes
metabias(m, method = "linreg")##
## Linear regression test of funnel plot asymmetry
##
## data: m
## t = 1.1008, df = 37, p-value = 0.2781
## alternative hypothesis: asymmetry in funnel plot
## sample estimates:
## bias se.bias slope
## 0.5592655 0.5080545 0.2550619#sensitivity (bias and heterogeneity)
#in this example, region where the dune study was done is likely an important source of heterogeneity
#data
#no need to wrangle
#assign model
m <- update(m, byvar = Region) #use update
#viz
forest(m)
#model
m## MD 95%-CI %W(fixed) %W(random) Region
## 1 1.2900 [ -0.3700; 2.9500] 0.2 2.7 temperate
## 1 0.8500 [ -0.5559; 2.2559] 0.3 3.1 temperate
## 1 -3.0800 [ -7.8639; 1.7039] 0.0 0.6 temperate
## 1 -4.1800 [ -9.3388; 0.9788] 0.0 0.5 temperate
## 6 0.6100 [ -0.0510; 1.2710] 1.5 4.4 temperate
## 6 1.6100 [ 0.6932; 2.5268] 0.8 4.0 temperate
## 6 1.1700 [ 0.5090; 1.8310] 1.5 4.4 temperate
## 6 1.5100 [ 0.9836; 2.0364] 2.4 4.6 temperate
## 6 0.0500 [ -1.5037; 1.6037] 0.3 2.9 temperate
## 6 -0.4900 [ -2.7434; 1.7634] 0.1 2.0 temperate
## 6 0.1000 [ -0.7961; 0.9961] 0.8 4.0 temperate
## 6 1.7000 [ 1.1231; 2.2769] 2.0 4.6 temperate
## 6 0.3200 [ -0.6401; 1.2801] 0.7 3.9 temperate
## 6 0.3400 [ -0.9419; 1.6219] 0.4 3.4 temperate
## 6 0.7700 [ -1.1837; 2.7237] 0.2 2.3 temperate
## 6 1.5900 [ 1.1930; 1.9870] 4.3 4.8 temperate
## 6 -0.4100 [ -1.6523; 0.8323] 0.4 3.4 temperate
## 6 0.0100 [ -1.6387; 1.6587] 0.2 2.7 temperate
## 11 0.1200 [ 0.0220; 0.2180] 70.2 5.0 temperate
## 11 0.1900 [ -0.0886; 0.4686] 8.7 4.9 temperate
## 14 1.3500 [ 0.4901; 2.2099] 0.9 4.1 temperate
## 34 -161.6000 [-210.2628; -112.9372] 0.0 0.0 temperate
## 44 7.1600 [ 1.2303; 13.0897] 0.0 0.4 temperate
## 44 11.8300 [ 3.4110; 20.2490] 0.0 0.2 temperate
## 45 1.8300 [ 0.5187; 3.1413] 0.4 3.3 tropical
## 45 4.1000 [ -0.6456; 8.8456] 0.0 0.6 tropical
## 45 -3.0800 [ -4.9817; -1.1783] 0.2 2.4 temperate
## 45 -4.0400 [ -6.1353; -1.9447] 0.2 2.1 temperate
## 45 -0.1600 [ -18.4583; 18.1383] 0.0 0.0 tropical
## 45 10.0700 [ -8.4711; 28.6111] 0.0 0.0 tropical
## 45 -22.0000 [ -36.7016; -7.2984] 0.0 0.1 temperate
## 45 -23.7000 [ -38.5458; -8.8542] 0.0 0.1 temperate
## 56 -0.6000 [ -3.1103; 1.9103] 0.1 1.7 subartic
## 59 2.3400 [ 1.0971; 3.5829] 0.4 3.4 temperate
## 59 16.4200 [ 8.2955; 24.5445] 0.0 0.2 temperate
## 59 1.1000 [ 0.3908; 1.8092] 1.3 4.4 temperate
## 76 2.4800 [ -0.1938; 5.1538] 0.1 1.6 temperate
## 76 2.3900 [ 0.7490; 4.0310] 0.3 2.8 temperate
## 76 0.0500 [ -0.9665; 1.0665] 0.7 3.8 temperate
##
## Number of studies combined: k = 39
##
## MD 95%-CI z p-value
## Fixed effect model 0.3322 [0.2501; 0.4143] 7.93 < 0.0001
## Random effects model 0.6887 [0.2834; 1.0940] 3.33 0.0009
##
## Quantifying heterogeneity:
## tau^2 = 0.8481; H = 2.70 [2.37; 3.08]; I^2 = 86.3% [82.2%; 89.5%]
##
## Test of heterogeneity:
## Q d.f. p-value
## 277.64 38 < 0.0001
##
## Results for subgroups (fixed effect model):
## k MD 95%-CI Q tau^2 I^2
## Region = temperate 34 0.3260 [ 0.2437; 0.4083] 268.59 0.8475 87.7%
## Region = tropical 4 2.0181 [ 0.7600; 3.2761] 1.60 0 0.0%
## Region = subartic 1 -0.6000 [-3.1103; 1.9103] 0.00 -- --
##
## Test for subgroup differences (fixed effect model):
## Q d.f. p-value
## Between groups 7.45 2 0.0241
## Within groups 270.19 36 < 0.0001
##
## Results for subgroups (random effects model):
## k MD 95%-CI Q tau^2 I^2
## Region = temperate 34 0.6450 [ 0.2276; 1.0623] 268.59 0.8475 87.7%
## Region = tropical 4 2.0181 [ 0.7600; 3.2761] 1.60 0 0.0%
## Region = subartic 1 -0.6000 [-3.1103; 1.9103] 0.00 -- --
##
## Test for subgroup differences (random effects model):
## Q d.f. p-value
## Between groups 5.26 2 0.0722
##
## Details on meta-analytical method:
## - Inverse variance method
## - DerSimonian-Laird estimator for tau^2#sensitivity
#interpret within versus between heterogeneity and the byvar effect size patternsExercise
Install then load ‘meta’ package. Work through one generic example using data within the package, from this repo, or from the resource site for the “Meta-analysis with R” book that provides many meta-datasets. Then, use your existing synthesis dataframe, generate an rmarkdown file and work through the basic meta-statistical steps listed above.
Conclusions
- There is a global open science workflow for scientific synthesis that culminates with a systematic review or meta-analysis.
- There are several clear steps within the meta-statistics step of this larger worflow.
- The meta-statistical workflow is just as important as the statistics.
- Herein, only the briefest tour of a single package and some function/argument opportunities are shown. There is an immense set of tools available within the contemporary R community for meta-analyses.
- This workflow and parsimonous set of steps to meta-statistics provides a reasonable and balanced approach to interpreting the strength of evidence for a set of studies/datasets and can highlight research gaps. More detailed statistics and tests are available but this approach encompasses 95% of of the synthesis studies currently published within the natural sciences.
Additional resources
Meta-analysis with R is a stunning resource.
The meta vignette is an excellent starting point for an overview of the functions and data in this package.
The metafor resources provided by CRAN are similarly useful.
There are many ‘how-to’ papers published for meta-analysis. Here is nice example.
‘forestplot’ is also a nice, well-supported CRAN alt-viz option.