An efficient protocol for the global sensitivity analysis of stochastic ecological models

Abstract

Stochastic simulation models requiring many input parameters are widely used to inform the management of ecological systems. The interpretation of complex models is aided by global sensitivity analysis, using simulations for distinct parameter sets sampled from multidimensional space. Ecologists typically analyze such output using an “emulator”; that is, a statistical model used to approximate the relationship between parameter inputs and simulation outputs and to derive sensitivity measures. However, it is typical for ad hoc decisions to be made regarding: (1) trading off the number of parameter samples against the number of simulation iterations run per sample, (2) determining whether parameter sampling is sufficient, and (3) selecting an appropriate emulator. To evaluate these choices, we coupled different sensitivity-analysis designs and emulators for a stochastic, 20-parameter model that simulated the re-introduction of a threatened species subject to predation and disease, and then validated the emulators against new output generated from the simulation model. Our results lead to the following sensitivity analysis-protocol for stochastic ecological models. (1) Run a single simulation iteration per parameter sample generated, even if the focal response is a probabilistic outcome, while sampling extensively across the parameter space. In contrast to designs that invested in many model iterations (tens to thousands) per parameter sample, this approach allowed emulators to capture the input-output relationship of the simulation model more accurately and also to produce sensitivity measures that were robust to variation inherent in the parameter-sampling stage. (2) Confirm that parameter sampling is sufficient, by emulating subsamples of the sensitivity-analysis output. As the subsample size is increased, the cross-validatory performance of the emulator and sensitivity measures derived from it should exhibit asymptotic behavior. This approach can also be used to compare candidate emulators and select an appropriate interaction depth. (3) If required, conduct further simulations for additional parameter samples, and then report sensitivity measures and illustrate key response curves using the selected emulator. This protocol will generate robust sensitivity measures and facilitate the interpretation of complex ecological models, while minimizing simulation effort.