Step 5: Collect and Evaluate Available Data

Step 5: Collect and evaluate available data to determine sampling sufficiency and the validity of the strata (if available)

  • In this step, you will use existing data to determine if you need to make any adjustments to the samples sizes that you identified in step 4. Consult with the NOC, USU, and Jornada to implement this step.This step addresses the following question: “How much data should be collected across the study area to address the management goals and monitoring objectives?” Analysis of existing data and monitoring objectives will provide information about the number of points required to detect whether an objective for a particular indicator has been met (e.g., the number of sites needed to determine whether 70% of areas with the potential to support sagebrush have greater than 15% sagebrush cover).
  • Consider sample size requirements in terms of your management objectives and the information needed for the decision at hand. Look at multiple indicators and take a preponderance of evidence approach.  For example, if one indicator requires many more samples than the others, then you may be able to rely on the preponderance of evidence from the other indicators to make your decision. If many indicators are showing insufficient information, then you likely need more monitoring points.
  • Most AIM efforts seek to estimate the proportion of a resource (in acres for terrestrial ecosystem and kilometers for perennial streams) within the project area that are meeting or not meeting objectives, within a certain level of confidence. Given the goals of estimating condition, the general recommendation for such monitoring efforts is to take an approach that minimizes the likelihood of not detecting a difference in conditions when a difference actually exists (i.e., Type II errors).
  • From a statistical standpoint, the sample size required (e.g., number of plots or stream reaches) to determine the proportion of the resource that is achieving the desired conditions will depend on three factors: 1) the amount of existing AIM-compatible data (e.g., WRSA points, LMF plots), 2) estimated proportion of data meeting an objective, and 3) the desired confidence level.
    • For many new AIM projects, data are already available from other AIM monitoring efforts or from the Landscape Monitoring Framework (LMF) or Western Rivers and Streams Assessment (WRSA). Always evaluate and consider using pre-existing data when determining sample sizes.
    • Depending on monitoring objectives and previous sample date and condition, LMF, WRSA, and other AIM data may be used to offset sample size requirements for new monitoring objectives. At a minimum, these data can be used to help assess the proportions of a resource that are meeting an objective and help estimate the required sample size for your monitoring objectives.
    • If you seek to have a high degree of confidence (e.g. 95%) in the condition estimates derived from your data you will require large sample sizes. To balance the desire to minimize Type II errors (i.e. failure to detect a difference) with the need for a realistic workload, the specific recommendation is to establish sample sizes using an 80% confidence interval. If monitoring data are to be used to support a contested management decision, higher percent confidence interval with smaller margin of error may be necessary.
  • To answer your question “How much data do I need to address my management goals and monitoring objectives,” follow the steps below:
    1. Identify the indicators of interest and the proportion of the landscape that is likely be be in a given condition (e.g, % of landscape having suitable or unsuitable habitat). It can be helpful to look at pre-existing data to estimate the proportion of sites currently meeting monitoring objectives as a starting point.
    2. Select an appropriate confidence level for the monitoring objective.
    3. With the information identified above, you can then estimate your initial sample sizes with these sample sufficiency tables.
  • After each year of sampling, it is recommended to do a more formal sample sufficiency analysis with the collected monitoring data to determine if your current sampling intensity is appropriate or if you need to plan to increase this intensity to obtain a larger sample size.
  • Additional points can be added to a monitoring effort to increase the precision and accuracy of estimates as needed. If adding more points is not feasible, an alternative approach is to accept a lower level of confidence for some reporting units. In these cases, data from other sources (e.g., remote sensing, use data) can be valuable for a preponderance of evidence approach.


Step 5 Example: Collect and Evaluate Available Data Example

Step 5: Collect and evaluate available data to determine sampling sufficiency and the validity of the strata (if available)

Terrestrial – Terrestrial sample sufficiency analysis focused on the proportion of the area meeting a benchmark based on pre-existing data. Pilot data were available from an adjacent field office that has similar ecosystems and environmental characteristics. We looked at five different indicators: bare soil, total foliar cover, shrub cover, perennial grass cover, and perennial forb cover. In general, at most 27 samples were sufficient to estimate the proportion of the area meeting objectives for all indicators with 80% confidence and 10% margin of error. In cases where the observed proportion of the landscape meeting objectives was far away from the required proportion, fewer samples were required. Thus, the current design is sufficient to report out in any given year at the District or Field Office scale and over 5 years in the Sheeprocks Sage Grouse Population Area at these error levels. Reporting in the Rich and Box Elder SFA’s, which have smaller sample sizes in this design, will result in a higher margin of error (e.g., 15% or 20%).

Aquatic – No pilot data were available so we were not able to incorporate any previously collected data into our sample sufficiency analysis and strata validity.

With the help of the NOC, we determined that our initial approach should be to collect data at 25 points and then do a sample sufficiency analysis to determine if our target sample size of 50 stream reaches will be enough to characterize conditions with enough confidence. We based this number on the worst case scenario of observing the maximum allowable variance for estimating a proportion (50%), with a 90% confidence level. This scenario will only allow one to detect degraded stream conditions when 50% (± 15%) of streams are in most degraded condition – an unacceptable amount from a management perspective. The actual variance observed at these initial 30 sample points will be used to determine the final sample size of this monitoring effort.


Helpful Documents and Links:

Sample sufficiency tables

Comments are closed