Genome scan methods against more complex models: when and how much should we trust them?

Pierre de Villemereuil*, Eric Frichot, Eric Bazin, Olivier Francois, Oscar E. Gaggiotti

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

175 Citations (Scopus)


The recent availability of next-generation sequencing (NGS) has made possible the use of dense genetic markers to identify regions of the genome that may be under the influence of selection. Several statistical methods have been developed recently for this purpose. Here, we present the results of an individual-based simulation study investigating the power and error rate of popular or recent genome scan methods: linear regression, Bayescan, BayEnv and LFMM. Contrary to previous studies, we focus on complex, hierarchical population structure and on polygenic selection. Additionally, we use a false discovery rate (FDR)-based framework, which provides an unified testing framework across frequentist and Bayesian methods. Finally, we investigate the influence of population allele frequencies versus individual genotype data specification for LFMM and the linear regression. The relative ranking between the methods is impacted by the consideration of polygenic selection, compared to a monogenic scenario. For strongly hierarchical scenarios with confounding effects between demography and environmental variables, the power of the methods can be very low. Except for one scenario, Bayescan exhibited moderate power and error rate. BayEnv performance was good under nonhierarchical scenarios, while LFMM provided the best compromise between power and error rate across scenarios. We found that it is possible to greatly reduce error rates by considering the results of all three methods when identifying outlier loci.

Original languageEnglish
Pages (from-to)2006-2019
Number of pages14
JournalMolecular Ecology
Issue number8
Early online date5 Apr 2014
Publication statusPublished - 8 Apr 2014


  • False discovery rate
  • Power simulation study
  • Genome scan
  • Adaptation
  • Bayesian methods


Dive into the research topics of 'Genome scan methods against more complex models: when and how much should we trust them?'. Together they form a unique fingerprint.

Cite this