Lab 2: Image Processing Techniques in ENVI

Introduction

In this lab we will cover the ENVI processing techniques for radiometric calibration, atmospheric correction and masking. This lab will also cover applying masks and calculating spectral indices.

Learning Outcomes

• Perform Radiometric Calibration in ENVI
• Conducting atmospheric correction (dark object subtraction)
• Evaluating Image Statistics
• Creating and Applying Masks
• Comparing and extracting data
• Calculating Spectral Indices
• Visualizing Data

About the Data
The primary data is this lab is Landsat Collection 2 Level-1, Landsat Collection 2 Level-2 Surface Reflectance data will be viewed for comparison.
The specific data used in the lab is a Landsat 8 image of the central Sierra Nevada mountain range acquired June 10th 2024.

Accessing and Calibrating Landsat Data in ENVI

You will begin by creating a folder structure or "workspace" that will keep your files organized and prevent any confusion as you work with multiple datasets in the future. It is important to have the files you are working with on a local drive (C: in our case). Many processes in ENVI and ArcGIS do not work well or at all when data is stored in the cloud (i.e. Google Drive G: Drive) or on portable storage devices. Back up your data when you are done!

1. Create a new folder on the desktop or documents folder for your lab data (recommended naming convention of the folder “GSP326_Lab_#”). Create three subfolders: original files, working and final.

   When you are done, you will want to back up your final work onto a USB drive, or cloud based storage like Google Drive or the Course Share Z: Drive. The hard drives (including the desktop) on the lab computers will be deleted every 24 hours. Be sure to always save and backup your work!

2. Download the Lab 2 data files from the shared Google Drive folder or the class share network drive (Z:). The data consists of two Landsat "tarball" files (.tar). Extract the two files into your original data folder, note that each of the files should be extracted into separate folders. When the process is finished you should have two folders that each contain Landsat data files, one is Level-1 data while the other is Level-2 data.. Open ENVI (ENVI 6.0).
3. In ENVI go to File → Open and navigate to the location of the Lab 2 original data folder. Find the folder with the Level-1 Landsat 8 data (hint: look at the file name, level-1 data is designate with L1TP) and select the metadata text file (it will end in "_mtl.txt") and click open.
4. First we will calibrate the Level-1 data so the pixel values represent real-world units, in this case percent reflectance. From the Toolbox, select Radiometric Correction → Radiometric Calibration or type “radiometric” in the toolbox and open the Radiometric Calibration tool.
5. In the Radiometric Calibration window select the data (Input Raster) for calibration, for this exercise will be using the multi spectral data "....._MTL_MultiSpectral". Set the calibration type to "Top of the Atmosphere Reflectance". Leave the default scale factor and data type (1, Float, respectively). Save the calibrated image in your Working folder as "TOA_Reflectance.dat". When the process is complete the calibrated image should automatically open.

View Image Statistics

6. Now we will calculate and view statistics for the image. Right click on the "TOA Reflectance" file in the Layer Manager and select "Quick Stats". A dialogue box will come up as ENVI calculates various statistics for the image. Once the calculations are complete the statistics view window will open.
7. The basic stats include the minimum, maximum and mean pixel values for each of the bands. Scroll down to see more detailed distribution statistics for each of the bands. The values for reflectance data should range from ~0-1, representing 0 to 100% reflectance. Note that there may be values outside this range that you may want to investigate.
8. For each band (seven in total) write down or copy/paste the minimum value, this is the lowest (non-zero) pixel value for that band. You can highlight the Band and associated Min value columns and copy them by selecting the Copy to Clipboard icon, then paste this information into a spreadsheet. We will use this information for our atmospheric correction process (note that values will differ from image below).

Atmospheric Correction (Dark Subtraction)

We will use the Dark Subtraction Tool to estimate and remove the effects of atmospheric scattering from the image by subtracting pixel values that represents a background scattering from each band. We will be using the band minimum values we researched and wrote down in the previous steps.

9. From the Toolbox, select Radiometric Correction → Dark Subtraction or type "Dark" in the Toolbox and open the Dark Subtraction Tool. The Dark Subtraction Input File window appears.
10. In the Dark Subtraction Input File dialog select the file "TOA Reflectance" and click OK.
11. The Dark Subtraction Values dialog appears, under Subtraction Method select "User Value". Click on each band (Bands 1-7) and enter in the minimum value you wrote down in the previous steps. Once you have entered in a value for Band 1, click on Band 2 and enter in the appropriate value. Do this for all seven bands. Note the image below is an example and that values will differ.

12. Once you have entered in the values for all seven bands name your output file "surface_reflectance.dat" and save it in your Final Folder. Click OK to start the Dark Subtraction process, it may take a minute or two to complete the process. The image should automatically open in your viewer.

Masking with ROIs - Create a No Data Mask

Masks are used to exclude certain pixels from image processing or when computing image statistics. Masked pixels appear as transparent in the display. Masks can be created several different ways, using the FMask tool, ROIs or the Build Raster Mask Tool. The next steps will go over several methods.

13. Notice that the outside "no data' areas of the Landsat images are no longer transparent after the dark subtraction process. We can apply a mask to properly exclude these values from display and analysis. Right-click on the Surface Reflectance file in the Layer Manager, and select New Region of Interest. Name the ROI no-data in the ROI tool window.
14. In the ROI Tool, click the Threshold tab. Click the Add New Threshold Rule button . In the File Selection dialog, select the first band associated with the Surface Reflectance file and click OK. A histogram of the band is displayed in the Choose Threshold Parameters dialog.
15. Use the Min and Max values or the histogram to select all of the 'no data' areas outside the image. These values should be negative. Check the preview box to see the areas selected by the range of values. Click OK when the appropriate areas are selected. If you want to save the ROI for futures use (in another work session), select File > Save As and save the selected ROIs as an XML file that can be read by ENVI.
16. We will now use the ROI to mask the image. To apply the mask we will use the Save As operation to create a new files with the no-data areas masked. In the Main toolbar, select File → Save As → Save As.. (ENVI). In the Data Selection Window select the "reflectance" as the Input File. Click the Mask button below. In the Mask Selection Window select the "no-data" ROI and check the inverse mask button, then click OK. Click OK in the Data Selection window to save the file. Note that the default is to mask the areas outside of the ROI, if you would like to mask the area selected by the ROI the inverse mask button should be checked.
17. Set the data ignore value to 0 and save the file as SR_masked.dat in your final folder. When the new image appears the "no data" areas should be hidden.
    

Compare the Data Processed Level-1 Data to the Level-2 Surface Reflectance Data

18. For comparison open the Landsat Level-2 data by opening up the metadata file associated with the data (LC08_L2SP_....._MTL.txt). This file is the Landsat Level-2 Surface Reflectance data that has been calibrated used the NASA/USGS algorithm (see note below). We will use this data to compare it to our processed data, first visually compare the two Surface Reflectance images.

Landsat 8-9 OLI Collection 2 Surface Reflectance data are generated using the Land Surface Reflectance Code (LaSRC) (version 1.5.0), which makes use of the coastal aerosol band to perform aerosol inversion tests, uses auxiliary climate data from MODIS, and a unique radiative transfer model.

19. In the Layer Manager make sure your have the masked Surface Reflectance data selected. Click the Spectral Profile icon on the toolbar or select Display → Profiles → Spectral from the menu bar. This opens the Spectral Profile window. Move the cursor around to look at the spectral profiles (or reflectance curves) of different areas. The x-axis is the the wavelength and the y-axis displays the percent of light reflected (
20. In the Spectral Profile window click Options → Additional Profiles → Add File and select the "TOA_Reflectance.dat" and click OK. Repeat this process to added the Level-2 Surface Reflectance data (LC08 ..L2SP (Surface Reflectance)). You should now see three spectral profiles in spectral profile plot.
21. In the Options menu select Legend to add a legend to the graph. Explore the spectral profile of different pixel types (i.e. Water, Vegetation, Snow, Clouds etc) by moving the cursor around the display window. Make note of any differences between the spectral profiles of the three different processed images.
22. Export at least three different data points for comparison (e.g. vegetation, water clouds for example). Select "Export" → ASCII. This will save the data associated with the plot as a text file, the text file lists the actual pixel values (reflectance) for each of the bands, for all three images. Select "Export" → "Image" from the Spectral Profile menu bar to save a copy of the spectral profile graph. Save your spectral plot image and data in your Final folder as you will need this for your lab report, you should have three plot images and three data text files. You will need to use this data to to discuss the differences in the processing. See GSP 216 Lab 3 steps 26-34 for how to style spectral profiles and export text data into Excel.

Masking Specific Features

You can create a mask from a region of interest (ROI), from a shapefile, by creating binary rasters, or by using options available in the Build Raster Mask tool. In addition to excluding the no data areas, we also want to exclude water and cloud pixels from a vegetation analysis. There are many different ways to create clouds (and other feature) masks and the next section will cover a few techniques.

Fmask Algorithm Cloud Mask

23. The Calculate Cloud Mask Using Fmask tool can be used to create a cloud mask for Landsat and Sentinel-2 data. We will test out the tool on out dataset. From the Toolbox, select Feature Extraction > Calculate Cloud Mask Using Fmask Algorithm.
24. Click the Browse button next to the Input Rasters field. In the File Selection dialog, select all datasets that correspond to the Landsat Level-1 files and click OK in the File Selection dialog.
25. Keep all of the default parameters. Select the working folder as the output file location and name your file Fmask.dat. Check the Display result option and click OK to run the process.
26. When processing is complete, the cloud mask is displayed and added to the Layer Manager. The cloud mask is a binary image where cloud (masked) pixels have values of 0 and non-cloud (non-masked) pixels have values of 1. Visually evaluate the effectiveness of the Fmask process by toggling the layers. Note what features are included in the mask and which are not.

View the Landsat QA Band and Create a Custom Mask

Landsat data products also include Quality Assessment (QA) bands to identify the pixels that exhibit adverse instrument, atmospheric, or surface conditions. This can include clouds, cloud shadows. This information can also be used to generate masks to exclude or isolate certain pixel types from analysis. There are several external tools available to extract information from the QA bands : https://www.usgs.gov/landsat-missions/landsat-level-1-quality-assessment-tools. Typically Level-1 data is used to generate mask, but Level-2 data can be used as well.

27. Now we will open the Landsat QA band, select file open and in your Originals folder and select the file "LC08_L1TP_.....QA_PIXEL.tif" and click OK. This is the Landsat Quality Assessment band.
28. Use the cursor value tool to investigate what some of the pixel values are for various features (clouds water etc). You may want to open a new view or toggle between the Landsat surface reflectance layer and the QA band layer for comparison. Make note of some of the QA pixel values associate with clouds, shadows and other features that you might want to mask.
29. Now we will create a ROI (Region of Interest) to use as out mask. Multiple ROIs can be employed to create a mask. Right click on the SR_Masked.dat file and select New Region of Interest. You can select specific features using several different methods. Use the threshold option to select certain pixel values in the ROI. Experiment using different bands and the QA bands to try and isolate certain features in the image. For example clouds tend to have relatively high reflectance across the spectrum while water and cloud shadows have low reflectance. Depending on the scene and masking requirements you may want to create multiple separate ROIs for different features. You can also create multiple threshold rules for an ROI.
30. Using one of the above techniques create a mask that masks specific features, note that clouds should be masked in all masks. You will apply this mask when calculating a spectral index of your choice in the next step. Some possibilities for masking are:
    1. Mask all non-vegetated areas - ideal for using with spectral vegetation indices like NDVI, NBR etc
    2. Mask all non-water area - ideal for studying water
    3. Mask all non-snow areas - ideal for studying snow reflectance

31. Now we will build the raster mask. From the Toolbox, select Raster Management > Build Raster Mask. The Build Mask Input File dialog appears.

32. Select the input file of SR_masked.dat and click OK. The Mask Definition dialog appears. From the Mask Definition dialog menu bar, select Options, here you can choose to add specific values to mask or ROIs to include in the mask. Under options select the import ROIs option and add the ROI(s) you created in the previous step. If the input file has a data ignore values (no data values), then the dialog opens with these values automatically entered in these fields.
33. Under Options change the mask option to "Selected Areas Off". This will produce a mask where the selected ROI(s) and no data values are masked or "off". Save the file as mymask.dat in the working folder and click OK. Compare the Fmask and your mask results.

Calculate Spectral Index and Apply Mask

ENVI can calculate a variety of different spectral indices.

34. From the Toolbox, select Band Algebra → Spectral Indices or type "Spectral Indices" in the Toolbox and open the Spectral Index tool. The Spectral Indices dialog appears. Select the SR_Masked.dat as your Input Data and click Mask button.
35. In the Mask selection window select the preferred mask from previous steps (either the Fmask or the custom mymask that you created) and click OK.
36. Now we will select the desired Spectral Index from the Spectral Index list. Select a spectral index designed to measure vegetation: Name your output raster appropriately and save it in the finals folder as a .dat file. Click OK and by default the spectral index image should appear in the viewer.

Visualize the Spectral Index

37. Right click on the "spectral index" layer in the Layer Manager and select "Change Color Table" and select one of the color tables. Applying a Color Table makes it easier to see the differences in spectral indices (or any index) values.
38. Another way to display and investigate data values is using the Raster Color Slice tool. Right click on the layer and select New Raster Color Slice. Select the desired layer to create a raster color slice of.
39. Use the Chip to View option or Snipping Tool to capture an image for export of either the index layer with the color table applied or the raster color slice. A simple legend or color bar can be added to the screen under the annotations option.

Turn In - Summary Report

A Word document with:

• An overview of the data used and techniques employed.
• Compare the processing techniques and results:
  • How did the radiometric calibration and dark subtraction atmospheric correction of the Level-1 data compare to the pre-processed Level-2 data? Were there differences in specific bands or certain features? Include the spectral data and graphs to back up your conclusions.
  • What were the advantages/drawbacks of the different masking techniques? Which masking technique worked the best and why?
• Figure showing the spectral index image (e.g. NDVI)