Lab 3: Image Statistics and Data Processing

Introduction

In this lab we will cover the ENVI processing techniques for layer stacking, ,mosaicking and masking datasets. This lab also covers reviewing image statistics and histograms.

Learning Outcomes

• Search for and download Sentinel Data from ESA Copernicus Browser
• Open and evaluate Sentinel-2 data
• Perform Layer Stacking operations to combine Sentinel-2 bands
• Mosaicking datasets
• Creating and Applying Masks
• Evaluating Image Statistics

About the Data
The primary data in this lab is Sentinel-2 Level2A data downloaded from the ESA Copernicus Browser. The goal of this exercise is to download and process Sentinel-2 imagery for Sonoma County, CA for eventual use in image classification. The desired data will be cloud-free, mosaic from June 2024.

Download Sentinel-2 Data and opening in ENVI

1. Create a new folder for your lab data, create two subfolders, for original files, and final processed files. Log into myHumboldt and navigate to your Humboldt Google Drive, navigate to the Lab 3 Data folder and download the ZIP file. The compressed folder contains vector files (shapefile and KMZ approximation) of Sonoma County.
2. Go to https://browser.dataspace.copernicus.eu/ and log into your account if you already have and if not click the “Register” button. To register, complete the form and validate your email address to activate your account then log into the Copernicus browser.
3. Our goal is to download imagery covering Sonoma County captured earlier this year for eventual use in image classification. To select an area of interest, you can draw a rectangle or polygon or upload an existing file. For the upload, only limited file types are supported, KML/KMZ, GeoJSON, and overly complex, large polygons (many vertices) are also not supported. Upload the simplified Sonoma County KMZ file provided to create an area of interest that represents Sonoma County.
4. Click on the Search Tab to search and download Sentinel data.
5. Under Data Sources Select Sentinel-2 L2A (Level-2 Surface Reflectance Data) and set the cloud filter to less than 10% or less by dragging the bar below the data source selection.
6. Scroll down and set the date range for June 1-30, 2024 (leaf-on, growing season imagery) then click the search button. You could also
7. There will be many different files that cover the area. Focus on finding two images that cover the majority of Sonoma County and were taken over a close time period (within 2-3 days). Click the Visualize button to see a visual preview of the image. Note the the imagery browser will switch views and you will need to return to the Search tab to download.
8. Select the appropriate images and download the files by click the Download Product icon for each associated scene.
9. After the downloads are complete, extract the files into your original data folder, keep the two files in separate folders. Note that you may need to rename the Sentinel-2 ZIP file to a shorter name, as the file names can be too long. When the extracting process is finished you should have a series of folders that contain various data files. Open ENVI.
10. In ENVI, select File → Open and navigate to your Originals folder where you extracted your data and find the folder that contains the Sentinel-2 data. Open the metadata XML file that begins with "MTD_MSI...". Repeat the process to open the second dataset.
11. You should now see the true color Sentinel-2 images. ENVI will automatically display the 10m resolution true color view of the scene and the other bands (20m and 60m) files will be available in the Data Manager.

    Band Number	Central Wavelength (nm)	Bandwidth (nm)	Spatial Resolution (m)
    B1	443	20	60
    B2	490	65	10
    B3	560	35	10
    B4	665	30	10
    B5	705	15	20
    B6	740	15	20
    B7	783	20	20
    B8	842	115	10
    B8a	865	20	20
    B9	945	20	60
    B10	1375	30	60
    B11	1610	90	20
    B12	2190	180	20

12. Spend some time looking at the different bands available in the data manager. The bands are packaged in "bundles" based on resolution and data type. See the below table for reference. You can select different band combinations and add them to your viewer.

Bands	Description
10m-S2MSI	Surface reflectance for 10m bands (RGB, NIR)
10m-AOT	Aerosol optical thickness (unitless)
10m-WVP	Water vapor
10m-TCI	True color image - an RGB image built from the B02 (Blue), B03 (Green), and B04 (Red) Bands. The reflectance are coded between 1 and 255
20-SCL	Scene classification data at 20m resolution

13. Now we will create a Layer Stack to create one raster that includes the first 10 bands of the data in one file (the 10m and 20m resolution data). In the Toolbox select Raster Management → Build Layer Stack. We will do this process twice, once for each dataset. We will also need to subet the data after stacking. If you would like you may install the Custom ENVI Task Layer Stack and Subset (found in the class Google Drive Folder).
14. In the Layer Stack window click to add the layer beside Input Rasters. In the Data Selection Window select the 10 and 20m data associated with one of the datasets (hold Ctrl key to select multiple items). Click OK.
15. In the main Layer Stack dialog, under Grid Definition select From Dataset. Select the 20m Sentinel data file as the reference dataset. This will set the coordinate system and cell size based on this dataset. Click OK.
16. In the dialog save your output Layer Stack file as the date of the image (e.g. June52024.dat or 06052024.dat) in your finals folder. This file will now contain the first 10 bands of the Sentinel-2 data. Click OK to create and save the layer stack. Wait to the image to load and for the pyramid layers to fully load. The stacked image should appear in your viewer when the process is complete. Check to make sure all bands (10) are included. Try making different false color composites to see the different bands.
17. Now repeat the layer stacking process for the other image date, including the 10m and 20m layers in the stack. You should have two stacked images when you are completed.
18. The last step is to subet the images. Using the Sonoma County shapefile provided, subset to the two stacked images. This will reduce the file size and improve the mosaicking process. You may remove the original datasets from the Data Manager, leaving only the stacked images.

Mosaic and Subset/Mask Image

18. Now we will mosaic the two datasets together to create one, seamless dataset that can be used for image classification. From the Toolbox launch the Seamless Mosaic Tool. Note that when the tool launches all files will temporarily disappear from the view, don't worry, this is expected and the files are still there!
19. Add the two layer stacked images from the previous steps. Check the Show Preview box to see a preview of the mosaic image. Note the color differences, since they are significant we will use Color Correction techniques.
20. Click the Color Correction Tab and check the Histogram Matching box. Try the different options (Overlap Area/Entire Scence) and select the one with the best results. You can also adjust the reference/adjust image in the main tab.
21. Click the Seam-lines icon and select Auto Generate Seam lines. Click through the various display icons in the mosaic tool to see the different view options. Turn on off all footprints and lines to see a preview of only the final mosaic.
22. Make any adjustments to the color settings and seam-lines and then click the Export Tab. Keep the default setting (Data Ignore Value should be 0) and save the mosaic file in your folder. The process may take a while.
23. Once the mosaic appears in the view we will perform the final data preparation steps by subsetting and masking the mosiac to the Sonoma County boundary. Use the Save As function to subset and mask the mosaic to the Sonoma County boundary (a shapefile was provided in the data download for this purpose). Save the final mosaic in your Final data folder.

24. During the mosaic process, some of the metadata can be lost from the original imagery, we can add this information back to the file by editing the ENVI header file (.hdr) From the Toolbox, select Raster Management > Edit ENVI Header. Select the mosaic raster and click OK.
25. Click on the Display Tab and update the Band Names to include a more descriptive name, e.g. NIR or Red. Keep the band number in parentheses, these are the original Sentinel-2 band references. Also update the Description under the main tab to include more information about the dataset - for example what type of data and the dates of the original imagery.
26. Click the Add button to select Wavelength and Wavelength Units. Use the information provided in the table at the beginning of the lab instructions to add the correct wavelengths to each of the bands and look up appropriate band names.
27. Click OK when you are done and the header file and metadata will be updated. Now the bands names and wavelengths should be visible in the Layer Manager.

View Image Statistics and Histograms

28. Now that we have a mosaic with all of the desired bands, we will calculate and view statistics for the image. In the tool box select Statistics → Computer Band Statistics.
29. In the Compute Statistics input file select the final Sentinel-2 mosaic you just created and under Mask Options select Mask Data Ignore Values (All Bands) and click OK. This will ensure the data ignore values are properly excluded from the statistics.
30. In the parameters windows select Basic Stats, Histograms, Covariance. Check the boxes to Output the data to the screen as well as a Statistics File and Text File. Save the Statistics (.sta) and text file (.txt) in your finals folder. Click OK to run the statistics.
31. Check out the statistics on the screen. The basic stats include the minimum, maximum, mean and standard deviation for each of the bands. Scroll down to see more statistics and detailed distribution statistics for each of the bands. The values for reflectance data should range from ~0-10000, representing 0 to 100% reflectance. Note that there may be values outside this range that you may want to investigate.
32. Below the basic stats are the correlation and covariance tables showing the relationship between bands. All of this same data is also included in the statistics text file (.txt) you save in the previous steps. In windows explorer, find the text file that you save and open in notepad or similar text reader. Review the stats in the text file.
33. Now we will view the histograms for all of the bands. In the upper left hand corner click "Select Plot" and select "All Histograms". Look at the distribution and shape of the histograms. You can save the histogram by selecting Export → Image. Also check out some of the other plots available. Close the Statistics Windows when you are done. You may need to revisit some of this information to complete the worksheet.

Investigate Low and High Pixel Values

Use the ROI threshold tool or the Raster Color slice tool to investigate various pixel values in the image.

34. Create a new ROI on the Sentinel-2 mosaic. Use the ROI threshold tool to find the features in the image with the lowest overall reflectance (across all of the bands) and the highest overall reflectance. Make note of what and where these features are in the image.
35. Now try using the raster color slice tool to investigate specific values in the images. Right click on the Sentinel stack layer and select New Raster Color Slice. Select the band you want to use to create a color slice. Specifically locate features that have greater than 100% reflectance in a band.
36. Review the worksheet questions. Answer the questions based on the statistics files and the above investigations.
37. Back up your final folder. We will need this data for next week's lab.

Turn In - Worksheet and Text file

Upload to Canvas

• Completed Lab 3 Statistics Worksheet
• ENVI generated statistics text file for the Sentinel-2 mosaic(.txt).
• ENVI Header file for the final mosaic image