Conducting Accuracy Assessment

Introduction

There are several steps in conducting an accuracy assessment. Ideally the accuracy assessment should be planned out ahead of time before you conduct your analysis or produce your maps.

General Steps for Performing Accuracy Assessment:

Sampling Methodology

When collecting your reference data your locations should be unbiased. To achieve this you need to develop some sort of sampling scheme. You reference locations must also be large enough to find with certainty on your classified image. The size of your sampling locations may depend on the spatial resolution of your data set. You also need a sufficient number of reference locations. When thinking about collecting reference data you should first consider the sampling methodology.

Random Sampling

In simple random sampling is the most straight forward sampling design. There are no rules for the sampling points, the points are simply randomly distributed throughout your study area.

Systematic sampling

In systematic sampling the sampling points are spaced uniformly at fixed intervals or along a grid. This ensures regular, complete coverage of an entire area.

Stratified Random

In stratified random sampling the number of random points is allocated based thematic classes. This sampling scheme can be applied in a couple of ways. One way is to stratify the number of random points so that larger classes will receive more points, this ensures classes that cover a large area will receive more sampling points, but it also ensures smaller area classes will still have sampling points. In equalized stratified random sampling each class with have an equal number of random points. For example if we have 10 classes and 100 sampling points, each class would be allocated 10 sampling points.

Collecting Reference Data

Field Collection

Vegetation Inventory field data collection Credit: National Park Service

Field data collection or ground truthing is the ideal way to collect reference data. Usually a GPS is used to find pre-determined reference sites (unbiased). When collecting the reference data it is important to have clear data collection procedures. This is done to ensure that the reference data is consistent, especially if you have multiple people or teams collecting the data. It is also a good idea to develop an efficient plan for visiting the site and to have a backup plan in case any of the sampling sites are unreachable.

Field collected data can also be used as training data and as reference data to build models based on remote sensed data. Often when collected reference data in the field you may want to collect a large number of sites. This allows you to use half of the data as training data and the other half of the sites can be used to conduct an accuracy assessment to validate your maps or models.

High Resolution Reference Imagery

While field collected data is often the preferred method to obtain reference data, it is not always feasible. High resolution remotely sensed data to sometimes used to assess the accuracy of low resolution data. In this case aerial photos, air videography, or high resolution satellite imagery is used as reference data. The requires some ground work to “train” yourself to accurately interpret the reference data. This method can be relatively inexpensive, is generally faster than collecting data in the field. It is also a good option for difficult to reach or inaccessible areas. The downfall is that it may negate the assumption of accurate or true reference data since there is a chance the reference data doesn't represent what's actually on the ground.

Compare Reference Data to Classified Data

After the data has been collected and the geospatial analysis has been conducted it is time to compare the results of the two. This is done relatively easily by comparing the classification or data to the reference data. Programs like ArcMap, ENVI and Excel can be used to extract and compare the data. Once the data has been gathered and extracted, an error matrix is computed to determine the accuracy.

← Back

Next →

Module Home