The power of effective study design in animal Experimentation: Exploring the statistical and ethical implications of asking multiple questions of a data set

Abstract

One of the chief advantages of using highly standardised biological models including model organisms is that multiple variables can be precisely controlled so that the variable of interest is more easily studied. However, such an approach often obscures effects in sub-populations resulting from natural population heterogeneity. Efforts to expand our fundamental understanding of multiple sub-populations are in progress. However, such stratified or personalised approaches require fundamental modifications of our usual study designs that should be implemented in Brain, Behavior and Immunity (BBI) research going forward. Here we explore the statistical feasibility of asking multiple questions (including incorporating sex) within the same experimental cohort using statistical simulations of real data. We illustrate and discuss the large explosion in sample numbers necessary to detect effects with appropriate power for every additional question posed using the same data set. This exploration highlights the strong likelihood of type II errors (false negatives) for standard data and type I errors when dealing with complex genomic data, where studies are too under-powered to appropriately test these interactions. We show this power may differ for males and females in high throughput data sets such as RNA sequencing. We offer a rationale for the use of alternative experimental and statistical strategies based on interdisciplinary insights and discuss the realworld implications of increasing the complexities of our experimental designs, and the implications of not attempting to alter our experimental designs going forward.