Data analysis provides the means to effectively apply monitoring data to decisions. Proper analysis provides the answers to management questions, and incorporates monitoring data, land use-based data, and other information. Our efforts to streamline the process for field offices are ongoing and input is always welcome!
Before we dive into the details of conducting analysis with AIM data, first it’s important to know the concepts behind data analysis which includes many statistical terms. Most of the terms below can be found on our glossary page, but below goes into more detail about concepts that are relevant to data analysis.
Kinds of AIM Analyses
Some management decisions or questions only require monitoring data from a single plot or stream reach in order to address them. For example, in evaluating the effectiveness of restoration or land treatments at a specific location, only the plot/reach specific condition is needed to decide whether management was successful.
Alternatively, you may only have data from one plot or reach in the area where you need to address a management questions. For example if you are doing a permit authorization renewal you can look for AIM data to supplement your data collected at key areas. You may only have one AIM point that falls in the desired area. However, even one point can provide one line of evidence in a multiple lines of evidence approach.
Even if only one point is available or needed to answer management questions, monitoring data including AIM data from a broader area can be used to set benchmarks.
Unweighted vs. Weighted
Summaries of resource conditions across an area (multiple plots/reaches) are often needed to support management decision-making. For example, a Land Health Evaluation will often use multiple monitoring locations across a grazing allotment. Likewise, Land Use Plan Evaluation requires knowledge of conditions at locations across a Land Use Planning area. Both unweighted and weighted analysis approaches can be used to summarize data within an area. Weighted analyses take into account the number of acres or stream kilometers that each monitoring site represents, otherwise known as a weight. Weights are generated from the original sample design information.
Many management decisions will be made without explicitly taking into account the weight of each plot or stream reach. In general, these are unweighted analyses. To do an unweighted analysis, one can simply count the number of plots/reaches in a given condition. One can also divide by the total number of plots/reaches in the reporting unit to obtain the percentage of the plots/reaches in a given condition. When conducting unweighted analyses, it is important to keep in mind how plots or reaches were selected, because that may influence results and interpretation of the data. The Benchmarks Tools can assist you with completing unweighted analyses (see Tools for Data Analysis).
To understand the percentage of the resource in a given condition, a weighted analysis is required. Point weights can be calculated from the total extent or amount of a monitored resource divided by the number of monitoring points. For example, if a 10,000 acre grazing allotment has 10 points in it, the weight of each point is 1,000 acres (10,000 divided by 10). Weights are used to generate statistical estimates of resource status or condition across the landscape. Specifically, the weight is used to adjust the relative influence each point has on the final estimates; points with larger weights have more influence, and points with smaller weights have less. The weight of each point depends on the design and how it was implemented as well as the reporting area of interest. Calculating weights can be complex. Contact the NOC to request the percentage of the resource in a given condition from a weighted analysis.
See Step 6 of Preparing For Analysis for more information when deciding whether to use a weighted analysis approach.
Other critical concepts related to analysis
Management Objective/Goal vs. Monitoring Objective
What exactly is the difference between these two? Why do you need these set up in order to do analysis?
Management Goal: Broad goals or desired outcomes land managers are trying to achieve with land management. Management objectives and goals provide the context for why monitoring information is needed and how it will be used. Often, these are derived from planning documents and policy. Examples include maintaining forage production for livestock or high-quality habitat for big game animals.
Monitoring Objective: Quantitative statements that provide a means of evaluating whether management objectives or goals were achieved. Monitoring objectives should be specific, quantifiable, and attainable based on available resources and the sensitivity of the methods. Quantitative monitoring objectives may be available in your resource management plans (e.g., for sage grouse, Clean Water Act requirements) or they may be developed in the monitoring planning process. At a minimum, monitoring objectives should include: 1. indicator; 2. benchmark for the indicator; 3. a time frame for evaluating the indicator, and 4. the reporting unit(s) over which the monitoring results will be reported. If making inference to a broader amount of resource (i.e. beyond the individual site scale) is pertinent to your objective, be sure to include the proportion of the resource that is desired to achieve certain conditions (i.e. benchmarks) and a confidence interval in the objective.
- Example objectives:
- Bare ground in Loamy ecological sites is between 15 and 35% for 80% of the land use plan area with 80% confidence with three years of data.
- Maintain bank stability of greater than or equal to 75% for 80% of perennial wadeable streams in the planning area with 95% confidence over 10 years.
Subsets of the study area where you need information, such as indicator means and confidence intervals. A study area can have different types of reporting units. Knowing the units ahead of time helps ensure adequate sampling. Reporting units may be different than stratification. Watersheds, allotments, and Greater Sage-grouse habitat units are all examples of reporting units.
The target population refers to the resource to be described. Sample points (see site or plots) are selected from within the population. The definition of the target population should contain specific information on the resource of interest, such as its spatial extent, its ownership status, and its size (e.g., all stream sizes? just first order streams?). The definition should be specific enough that an individual could determine whether a sample point is part of the target population. In some cases, membership in the target population might be determined after data have been collected at the sample point (e.g., sage-grouse seasonal habitat). Examples of the target population include: all BLM lands within a reporting unit, all perennial, wadeable streams on BLM land, and sage grouse habitat on BLM lands. (Monitoring Resources, 2017)
Stratification refers to dividing a population or study area up into sub-groups or subunits called strata for the purposes of sampling or data analysis. Reason to stratify: 1) variability in indicators is different across types of land; 2) ensure different types of land or uncommon portions of a study area get sampled; 3) to deal with differences in land potential. Examples of strata include biophysical settings (see BpS), stream order (see Strahler stream order) , management unit boundary, and ecological sites (see Ecological Sites) (Monitoring Manual for Grassland, Shrubland, and Savanna Ecosystems, Volume II).
Indicator values, or ranges of values, that establish desired conditions and are meaningful for management. Benchmarks are used to compare observed indicator values at assessed points to desired conditions. For example, achieving a benchmark value of plant density may tell you that a seeding project was successful; failure to achieve it may trigger reevaluation of seeding methods. Likewise, observed conductivity values characterize the amount of dissolved cations and anions in water at an assessed point, but without appropriate benchmarks, such values lack context and cannot be used to assess condition or the attainment of management objectives. Benchmarks for a given indicator may vary by potential (e.g., Ecological Sites) thus different benchmark groups may be necessary within a project area so that points are understood as meeting or not meeting an objective relative to potential. See here for a more detailed discussion of benchmarks.
A geographic area or group of monitoring points that have the same benchmark for evaluating the success of a particular monitoring objective. For example, if you have points across your entire field office but want to evaluate a sage grouse habitat objective, only the points that are within sage grouse habitat should be considered for that particular objective. Likewise, the ecoregion, ecological site or stream type, the evaluation area or stream type must be taken into account for determining whether an objective is met when benchmarks vary by ecoregion or ESD.
Predicted Natural Conditions
An approach to setting benchmarks where the conditions expected to occur at a plot or reach in the absence of anthropogenic impairment are derived from empirical models. Such models use geospatial predictors (e.g., soil, climate and topographic attributes) to account for natural environmental gradients. Observed field values are compared to potential natural indicator values and any deviation is assumed to result from anthropogenic impacts. This approach is advantageous because it provides spatially explicit predictions of expected conditions with known levels of accuracy and precision. Unfortunately, due to data limitations and the current state of the scientific literature, this approach is only available for a few aquatic AIM indicators.
Percentiles of Regional Reference
An approach to setting benchmarks that uses reference sites or plots grouped by a landscape classification schema (e.g., ecoregions) to create a distribution of reference site indicator values. Benchmarks can then be set by assuming that sites in reference condition should fall within certain percentiles of the reference site distribution of a similar physiographic region. For example, the 90th and 70th percentiles of reference site floodplain connectivity values for the Colorado Plateau can be used to separate “major departure,” “moderate departure,” and “minimal departure” from reference conditions, respectively. For aquatic AIM, this approach can be used for indicators that lack models to compute predicted natural conditions. For terrestrial AIM, this approach is dependent on identifying and establishing a group of regional reference points.
Percent (Proportion) Achieving Desired Conditions
The desired percentage of a resource with one or more indicator values that meet benchmark value(s). For instance, a desired percentage may be (80%) of the landscape with <20% bare ground, or 80% of sage-grouse summer habitat scored as suitable (based on multiple indicators). Percentages are derived from weights (see weight definition) of monitoring points or plots, where a point or plot weight indicates the extent of the resource represented by a point or plot.
The status of a renewable resource in comparison with a specific reference value or benchmark (adapted from Bureau of Land Management Rangeland Resource Assessment-2011). When describing condition, a condition category may be assigned (e.g., Suitable, Marginal, Unsuitable or Minimal, Moderate, or Major departure) relative to the benchmark or reference value.
Range of values that likely includes the true value of a population mean, helps to understand uncertainty in the estimate. For example, an 80% confidence interval indicates that 80% of sampling events will result in estimates that fall within this range; 20% will not. The confidence level (e.g., 80%) indicates the probability that the confidence interval includes the true value and is chosen by the monitoring data user. Also see Elzinga et al. 2003 or any statistics textbook.