Meta-analysis and its brief history

Meta-analysis is a quantitative method that combines results from different studies on the same topic in order to draw a general conclusion. Since its introduction in 1976 by a psychologist, Gene Glass, meta-analysis has evolved into an essential and established tool for literature review and research synthesis in the social and medical sciences. Since its first use in the field of ecology and evolution in 1991, meta-analysis has become increasingly popular and by the end of 2000, nearly 200 meta-analytical papers were published.

Reading(s)

• Arnqvist G, Wooster D, 1995. Meta-analysis: synthesizing research findings in ecology and evolution. Trends in Ecology & Evolution 10:236-240. (a good introduction to meta-analysis)
• Hunt M, 1997. How science takes stock: the story of meta-analysis. New York: Russell Sage. (a longer version of the history of meta-analysis)
• Gates S, 2002. Review of methodology of quantitative reviews using meta-analysis in ecology. Journal of Animal Ecology 71:547-557. (a more recent account of meta-analysis)

Meta-analysis and effect size

Effect size is actual difference or relationship of interest and is expressed in a variety of ways. There are usually two classes: unstandardised effect size (e.g. mean differences and slopes from regression) and standardised effect size (e.g., Pearson’s r, Cohen’s d and Hedges’ g). The former can be easily converted to the latter (see the reading list for this section) Meta-analysis uses the latter because it is dimensionless so that the integration from different studies with different units or methodologies is possible. There are several kinds of standardized effect size measures available, but most of these fall into one of two major types, namely the r family and the d family. The r family shows the strength of relationship between two variables while the d family shows the size of difference between two variables. As a benchmark for research planning and evaluation, Cohen proposed ‘conventional’ values for small, medium, and large effects: r = 0.1, 0.3, and 0.5 and d = 0.2, 0.5, and 0.8, respectively (in the way that p values of .05, .01, and .001 are conventional points). In reality, published papers may only provides z, t, F, P and χ², R² values, from which you can calculate r or d along with sample size associated with these statistics (see the provided readings for practical solutions).

Reading(s)

• Lipsey MW, Wilson DB, 2001. Practical meta-analysis. Beverly Hills, CA: Sage. (Appendix of this book is a must-see for calculation of standardised effete size for meta-analysis - download a spreadsheet from mason.gmu.edu/~dwilsonb/ma.html – this is extremely useful)
• Nakagawa S, Cuthill IC, 2007. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews 82:591-605. (general statistical approach in biology, emphasising on effect size rather than statistical significance)

Meta-analysis and publication bias

An estimated mean effect size from meta-analysis will be biased if published studies on a topic suffer from publication bias (only statistical significant results of a particular topic are preferentially published even thought there may be more studies with non-significant results on that topic). There are several ways of dealing with this problem of publication bias such as ‘fail safe number methods’, ‘the trim and fill method’ and methods using correlation between effect size and sample size. Please read the review article on this topic below (the software MetaWin provides two methods for fail safe number method although fail safe methods have shortcomings: see the next section).

Reading(s)

• Moller AP, Jennions MD, 2001. Testing and adjusting for publication bias. Trends in Ecology & Evolution 16:580-586. (a good review of how to deal with publication bias in meta-analysis).

Meta-analysis and software

Meta-analysis is a form of weighted regression analysis and it is recommended to use statistical software capable of doing this. For this project, we will use MetaWin (a Windows based software). For advanced students, packages in R (free statistical software from www.r-project.org) such as “meta” or “rmeta” may be recommended.

Recommended software

• Rosenberg MS, Adams DC, Gurevitch J, 2000. MetaWin: statistical software for meta-analysis. 2 ed. Sunderland, MA: Sinauer. (a good piece of software to start meta-analysis with an excellent overview of meta-analysis and practical guidelines in the accompanying booklet)

Recommended topics for meta-analysis

Of course, students are encouraged to come up with their own meta-analysis ideas but it may not be easy to do so. We recommend the topics below (remember that meta-analysis deals with differences between two groups e.g. Cohen’s d, or relationship between two variables, correlation coefficient r – all recommendations belong to the latter):

1. Male reproductive success and their size (e.g. birds, insects, fish and mammals).
2. Male or female reproductive success and their age (e.g. birds, insects, fish and mammals).
3. Female reproductive success in relation to their secondary sexual character (e.g. birds, insects, fish and mammals).
4. Inbreeding and reproductive success (e.g. birds, insects, fish and mammals).

Examples of meta-analysis

• Arnqvist G, Nilsson T, 2000. The evolution of polyandry: multiple mating and female fitness in insects. Animal Behaviour 60:145-164. (an excellent method section on how to choose studies for meta-analysis and nice summary tables)
• Dubois F, Cezilly F, 2002. Breeding success and mate retention in birds: a meta-analysis. Behavioral Ecology and Sociobiology 52:357-364. (with phylogenetic analysis)
• Griffith SC, Parker TH, Olson VA, 2006. Melanin-versus carotenoid-based sexual signals: is the difference really so black and red? Animal Behaviour 71:749-763. (Relevant to Recommandation 1 above)
• Jennions MD, Moller AP, Petrie M, 2001. Sexually selected traits and adult survival: a meta-analysis. Quarterly Review of Biology 76:3-36. (One very large meta-analysis)
• Nakagawa S, Ockendon N, Gillespie DOS, Hatchwell BJ, Burke T, 2007. Assessing the function of house sparrows' bib size using a flexible meta-analysis method. Behavioral Ecology 18:831-840. (Relevant to Recommendation 1 and an introduction to an advanced method in meta-analysis).
• Schino G, 2001. Grooming, competition and social rank among female primates: a meta-analysis. Animal Behaviour 62:265-271. (Schino has several meta-analysis papers on primates behaviour and they are all very nicely done – check them out if you are keen)