Quantifying the Power to Detect Change: methodological development and implementation using the R package MRSeaPower.

This work presents the results of marine and freshwater scientific work carried out for Marine Scotland under external commission. The work was funded through the Scottish Government’s Contract Research Fund and by Scottish Natural Heritage and the Crown Estates.
New developments may impact animals and birds which use the development site; for example, with an offshore windfarm marine mammals and seabirds may move away from the site entirely or redistribute themselves within the site. Detecting these changes is difficult because the number of animals may change at a site, or they may move within the site, regardless of any disturbances. Hence, the challenge is to determine if any changes in abundance and distribution are due to an impact (either directly or indirectly) or if these changes would have occurred anyway in the absence of any development. Surveys of the site are thus generally conducted before any development takes place, during construction and after construction in order to reliably determine any effects. Statistical methods can be used to identify both temporal and spatial changes at the site. The ability of a study to detect change is a statistical
concept called ‘power’ and essentially quantifies the chance that a study will correctly identify a genuine change.
This project undertook a literature review of existing methods used to estimate power considering a range of taxa (e.g. seabirds and marine mammals) and survey types (e.g. aerial or vantage point surveys) which were likely to be relevant to studies undertaken to monitor marine developments in Scottish waters. A development could be a large scale, off-shore wind farm or a smaller scale, near-shore wave or tidal installation. Overwhelmingly, the power analysis methods used previously were simulation-based, underpinned by actual survey data from the baseline condition (in the absence of any impacts); post impact data being generated by manipulation of the baseline situation to show the presumed effect. This was the approach taken for this project.
The project created user-friendly software that implemented the identified power analysis approach for a wide range of users. The software generates power analysis results alongside abundance and distribution maps, and is anticipated to be of value to the renewable energy industry, Scottish Government and statutory advisers alike.
The approach taken was based on simulation and the basis of a simulation-based approach is as follows. Data collected pre development is used as the basis for simulating baseline conditions where the true abundance, or distribution, is known. Post-impact data is then created by exposing the baseline condition to a variety of impact scenarios (e.g. a site wide decrease in the number of animals). Statistical models, which include terms that describe the change scenario, are then fitted to the simulated data. For each simulated reality, a metric is obtained from each model and this indicates whether a change has been detected (e.g. by the change term being statistically significant). The metric then informs the power estimate; for example, if
an overall (site-wide) change in abundance was detected for 85 out of 100 simulated sets generated with these characteristics, this would return a power estimate of 85%. It was important to ensure that the methods and software being developed were appropriate for a variety of surveys and also for a data-rich (abundant) species and a data-poor species. Therefore, when developing the software, two types of survey and two species were considered; a large scale survey and a small scale vantage point survey and an abundant seabird species and less frequently detected marine mammal. While simple in concept, a simulation-based approach can be complicated to implement to ensure that the simulated data is realistic and satisfactorily resembles the observed data. Therefore, a substantial part of the project was devoted to characterising the
baseline data. Three key components were considered: the signal underlying the process of interest, the variability observed in the data and patterns that could not be explained by covariates used in the data generation model. Visual diagnostic checks are included in the software to allow a user to assess if these components have been adequately captured. An important part of the assessment of the generated data, was the development of a new test to identify autocorrelation in sequences in data.
Subsequent to the generation of realistic simulated data, a power analysis can be undertaken. The method involves the following stages: inducing changes to the generated data; fitting model(s) to the new set of generated data; quantifying the power to detect change and sense-checking the results.
The post intervention changes available for consideration are: 1. an overall (site-wide) change in abundance, 2. a redistribution of the baseline abundance across the site, and 3. changes in both the site-wide abundance post impact (i.e. an increase or decrease) and different distribution patterns pre and post impact.
The software ( MRSeaPower), developed as part of this project, is a free package designed to integrate with the package MRSea (Marine Renewables Strategic environmental assessment). Both packages are designed to run using
R , a free software environment for statistical computing and graphics. Comprehensive documentation and case studies are available as part of the package to allow users easy access to the methods and software.