Lab 12: Lidar in ArcGIS

Introduction

In this lab we will work with a lidar data set ArcGIS Pro. The lidar data is stored as a LAS file. A LAS file is an industry-standard format for storing X, Y, and Z point data (lidar). We will looks at different ways to visualize the lidar data in both 2D and in 3D using ArcGIS Pro. In this lab we will also learn how to create some lidar based raster datasets. Common lidar products are digital surface models. In this lab we will use the Lidar data to create high resolution a digital elevation model (DEM), digital surface model(DSM) and finally a canopy height model (CHM).

About the Data: The lidar point cloud data for this lab was acquired through the USGS 3D Elevation Program (3DEP). This data is a section of lidar that was collected during the summer months of 2019 and 2020 in Glacier Bay National Park and Preserve, Alaska.

Learning Outcomes

• Use ArcGIS to visualize lidar data in 2 and 3 dimensions.
• Create Digital Elevation Models (DEM) from lidar point data.
• Create Digital Surface Models for lidar point data.
• Create canopy height models from DEM and DSM
• Visualize data in 3D using Arc GISPro.

Set up your Workspace

First we need to set up our workspace.

1. Locate the Desktop and create a new folder named Lab_12. Create two subfolders, one name Originals and one named Final.
2. Navigate to the Lab data folder in the Shared Google Folder and locate the file Lab 12 data zip file and extract the data into the Originals folder. There are several separate LAZ files that store the lidar point cloud data, each file covers approximately 0.5 mile x 0.5 mile square or around 150 acres. LAZ is the compressed format of the standard LAS files, the LAZ files can be decompressed using ArcGIS Pro (next steps in lab instructions) or using LAS zip tools.

Creating and Viewing LAS Datasets

3. Start ArcGIS Pro. Make sure the Spatial Analyst and 3D Analyst extensions are enabled (see Lab 10) and create a new project and map in your Lab 12 Final folder. In the Catalog pane (View → Catalog Pane), add a folder connection to your Lab 12 folder for easy access to all of you data. Before we open and process our lidar data, we need to confirm the spatial reference (coordinate system) for the data. In the Catalog pane right click on the XML file (which is a metadata file) and select View Metadata. In the metadata window you will most likely see and message that the FGDC metadata can't be read by ArcGIS Pro, click the View Content link to open the metadata file in a new window.
4. Search for both the Vertical (Z or height, vertdef tag) and Horizontal Coordinate System (XY, horizsys metadata tag). Write this information down or leave the window open for reference, we will need this information for the next step.
5. The Lidar files from the USGS are delivered in the compressed LAZ (.laz) format and will need to be extracted to the LAS (.las) format before they can be loaded in ArcGIS Pro. The Convert LAS tool can extract the files to the .las format and create a LAS Dataset to view and process the lidar tiles. From the Geoprocessing pane search for or navigate to the Convert LAS tool.
6. The Convert LAS tool can convert single files or entire folders. To convert all LAZ files, select your "Originals" folder as the Input LAS. This will convert all applicable files (.LAZ) located in this folder. Select the same folder (Originals) as the Target Folder. This will place the extracted LAS files into the Original folder when the process is complete. Compression should be set to No Compression (this extracts or decompresses files) and leave all of the LAS Options unchecked. In the Define Input Coordinate System select LAS Files with No Spatial References. Set the spatial references as noted from the metadata (step 4). You will need to step both the XY and Z coordinate system. Save the output LAS Dataset as AKLidar in your Lab 12 folder and click Run to convert the files and create the LAS dataset.
7. After the tool runs the LAS Dataset should automatically be added to your map. You will see only a red outline of the data extent. This is normal, by default not all of the points are displayed because there are too many. If you zoom in you should be able to see the data. You may need to zoom in significantly for the points to appear (scale of 1:1000).
8. Now we will review the statistics for the LAS Dataset. In the Catalog pane, right-click on the LAS Dataset and select Properties. This will bring up the LAS Dataset properties and statistics. Now is a good time to open the Lab 12 Worksheet. In the different Dataset properties you will find the total number of LAS points in all of the files, statistics for the individual LAS files and summaries of the returns, classifications and other data attributes.
9. Click on the Statistics option and click the Calculate button to produce the statistics for this LAS dataset from the LAS files you added.
   Now you will see information about the LAS files, including the point counts, classifications and returns. The LAS Files tab includes information about the linear point spacing for each of the LAS files while the Statistics includes information about the classification and attributes. Note that you can highlight and copy/paste the LAS file statistics into a spreadsheet to view the information. You will need some of this information to questions on the lab worksheet. Also check the Coordinate System information by clicking on the Coordinate System tab.  Make sure the Horizontal and Vertical Coordinate System match the metadata and note the units. Close the LAS Dataset properties when you are done.
10. Return to your map and zoom back into the LAS data until you can see points. Now we can check out different ways to visualize the data points in 2D. Click on the .lasd file in the contents pane to select it as the active layer. Now there will be a LAS Dataset Layer tab in the main ribbon. Click the LAS Dataset Layer tab. This controls how the LAS is displayed.
11. First change the Point Thinning options. This allows some control over how many points can be rendered on the map. The density allows for more or less points. The maximum Display Limits is 100,000,000. Try increasing the display limits (if your computer is having a hard time rendering the points you may want to reduce the limit and density) and changing the Density.
12. Check out the Symbology options by clicking the arrow below the Symbology icon. You can visualize the points by different attributes, including Elevation, Class and other attributes (vary depending on the data). You can also symbolize the layer as a surface rather than points. Try out all of the different symbology options.
13. The Filters group on the LAS Dataset Layer tab set allows you to change the display of the data and display selected classes or returns. Once the filter options have been chosen, any further analysis or symbology changes will honor the selected filters. This means any statistics run or files created will be created using only the filtered points. Make sure the LAS Dataset is selected before trying to change the filters. Try different combinations of filters and symbology options. Reset the filters to All Points when you are done exploring and the symbology to Points.

View LAS Data in 3D

14. Any 2D map can be converted to a 3D scene for data visualization, exploration, or analysis. Click View from the main ribbon and select Convert >To Local Scene. This will convert the current map to a 3D enabled scene.
15. Check out the LASD layer in 3D. In the 3D view you can see all points and pan/move the area around using the Explore tool (under the Map tool ribbon), see more: 3D Navigation in ArcGIS Pro. Explorer the scene by zooming in and panning around
16. You can control the display of the LAS points in the 3D map the same as you can in 2D. Play around with the LAS data filters and symbology options. When you are done return to the 2D map, we will return to the 3D scene later.

Creating a Digital Surface Models from LAS Datasets

First we will create a Digital Elevation Model (DEM) from the Ground Returns. A Digital Elevation Model (DEM) is a representation of the bare ground (bare earth) topographic surface of the Earth excluding trees, buildings, and any other surface objects. To create a DEM we will filter the points to only include data points that are classified as ground or water.  

17. In the 2D Map, filter the LAS dataset so only ground points are displayed. This will ensure only ground points are processed when we create our DEM in the next step.
18. Now we will use the LAS Dataset to Raster tool to generate different digital surface models starting with a bare earth DEM. Either select the LAS Dataset to Raster tool from the geoprocessing menu or go to the Data tab under the main ribbon and select Export → Export to Raster.
19. Under Input LAS Dataset, select the LAS Dataset (.lasd). Make sure you select this from the drop-down not by clicking on the file folder, this will preserve the filter we set in the previous steps.  Save the output raster as "DEM.tif” in your Final folder.  Keep the Value Field as “Elevation” as we want to create an elevation raster.  
20. Under Interpolation Type keep “Binning” selected and under Cell Assignment Type select “Average”.  The last thing we are going to change is the Sampling Value, set this to “1”.  This defines the spatial resolution of our output raster, which will be 1 meter (the unit of the sampling value is determined by the horizontal coordinate system). 
21. The Digital Elevation Model (DEM) should now be in your viewer. This is a "bare earth" model that represents the terrain without any vegetation. Make note of the minimum and maximum values of the DEM, these represent the highest and lowest areas of terrain (ground) in the image.

Creating Digital Surface Model of the First Returns (Canopy)

22. Right click on the LASD layer in the Table of Contents and select Properties. In the LAS Filter tab this time, check 1 under Returns and make sure all of the classes are selected. If there are classes labeled as Noise or High Noise these should be unchecked and left out of the analysis. Note that not all LAS fields have Noise classifications. This filters the points so only the first returns will be shown, the first returns represent the highest point in the area. You can also filter to first return by using the drop-down on the LAS Points option on the main toolbar.

23. We will now follow a similar process that we used to create the DEM but this time we will be creating a digital surface model of the first returns (canopy). Either select the LAS Dataset to Raster tool from the geoprocessing menu or go to the Data tab under the main ribbon and select Export Export to Raster.
24. Keep the Value Field as “Elevation” as we still want to create an elevation raster. Under Interpolation Type keep “Binning” selected and under Cell Assignment Type select “Maximum”, this will use the maximum point elevation. The last thing we are going to change is the Sampling Value, set this to “1” to define the spatial resolution of our output raster (1 m in this case). Save the output raster as “DSM.tif” in your folder. Press “OK”. Now the DSM file should have automatically been added to your viewer.

Creating a Canopy Height Model (CHM)

With both the DEM and DSM generated from the lidar data it is possible to estimate the canopy height above the ground. The DSM is essentially the height of the canopy above sea level. We want to create a raster that solely represents the height of canopy above ground. This can be done by simply subtracting the DEM from the DSM, which produces a canopy height model (CHM).

25. Using the Spatial Analyst Math tools we will create a canopy height raster by subtracting the two rasters.  Find the Minus tool in the Geoprocessing pane (Spatial Analysis Tools→ Math → Minus).
26. For Input Raster 1 select the canopy surface model "DSM.tif" and for Input Raster 2 select the DEM "DEM.tif", so DSM-DEM = CHM (Canopy Height Model).  Save the output raster in your Final folder as “CHM.tif” and click OK.
27. You will now see the canopy height raster in your viewer.  The values in this raster represent the height of the vegetation above ground (in meters in this example).  You may want to change the symbology to a different color ramp that better displays the data. Make note of the maximum value of the CHM.
28. Depending on the data, you may encounter negative CHM values, these values are due to noise that may not have been properly filtered out the data (like birds or other features). We will remove any negative values (noise) using the Con geoprocessing tool to recalculate any negative values to a value of zero. In the geoprocessing pan select the Con tool by searching for it or by locating it under Spatial Analysis Tools→ Conditional → Con.
29. In the Con tool window, select the CHM files has the Input conditional raster. In the Expression box enter Value < 0, this will apply the tool only to pixels where the canopy height value is less than 0. Type "0" in the Input true raster or constant value. This will replace any values less than 0 (negative values) with 0. In the next box, Input false raster or constant value select the CHM file. This will use the existing CHM values for all areas where the values are greater or equal to 0. Save the file as CHM-final.tif in the Finals folder and hit OK.


You now have a completed canopy height model where canopy heights are displayed in meters.  Sometimes pixels along the edge can have aberrant values due to the lack of data on the edges. If necessary, an easy way to remove these pixels is to clip or mask the CHM using a shapefile.

30. You now have a cleaned-up CHM. You also have all the information needed to complete the Lab 12: Worksheet in Canvas. You may need to revisit steps 8-10 to view the LAS Dataset properties and Statistics in Catalog to complete the worksheet.

Creating Hillshades with Raster Functions

Raster functions are operations that apply processing directly to the pixels of imagery and raster datasets, as opposed to geoprocessing tools, which write out a new raster to disk. Calculations are applied to the pixels of the original data as the raster is displayed, so only pixels that are visible on your screen are processed. These tools are a great way to quickly preview an operation or to create layers for visualization. If you need to save the file or use it in future analyses the geoprocessing tools are more appropriate. Hillshades are rasters that appears to be a photo from above the sea, shaded by the sun. They rarely used for analysis but are one of the most common backgrounds used with maps. You can make a simple hillshade very quickly with just a DEM file. Hillshades are also useful base layers to use when layering imagery over a surface.

31. Under the Imagery Tab navigate to the Raster Functions button and click on the Raster Functions. This will launch the Raster Functions pane on the side. There are many different functions to choose from.
32. Navigate to the Surface functions in the Raster Functions tool pane. Select the Hillshade option.
33. Create a hillshade using the DEM as the input file (you can keep all of the default settings).
34. Repeat the above process an create two more hillshades, one for the CHM and one for the DSM, you should have three hillshades when you are done.

Arranging Data and Creating Maps

35. First we will create layer groups to organize out layers. Right click on Map in the Contents and select New Group Layer.  Click on the name and rename it to “Elevation”.  Repeat this again and create a second group, and rename the second layer group as “Canopy”. Drag the layers so the final CHM and CHM & DSM hillshades are under the Canopy group layer an the DEM and DEM hillshade under the elevation group. This will allow you to turn on and off all layer in the group with one click and makes organizing data easier.
36. Let’s update the symbology for the DEM. Select the DEM file and go the Symbology and change the Stretched symbology to color ramp instead of the default grayscale ramp. The colors correspond to the different elevations.
37. In order to display the elevation data as a layer on top of the hillshade we will need to set the transparency of the elevation layer. Under the Appearance option change the transparency to 50% for the DEM.   This will set the transparency to 50% for this layer allowing the hillshade layer to show through, note that the DEM layer should be on top of the hillshade. 
38. Repeat this same process for the Canopy Height (CHM) layer, choose different color symbology for the DEM and CHM. Keep the default grayscale symbology for the Hillshade layers.
39. Remove the default esri basemaps. Create a map layout or two separate map layouts, (your choice) that shows the Elevation (map 1) and Canopy Height (map 2). Each map should include the surface model (DEM or CHM) and applicable hillshade and should also include titles, north arrow, scale and legend. The maps can be any size or layout style. Tips and Trick for Working with Legends in ArcGIS Pro
40. Export map (or maps) in PDF or JPG format at 300 dpi resolution.

Create a 3D View and Map

41. Return to the 3D map and add your elevation (DEM) and canopy height rasters (CHM). You may want to uncheck or remove the LAS Dataset layer to make the map render more quickly.
42. Right-click on the Ground element under the Elevation Surfaces group at the bottom of the Content Pane and select Add Elevation Source. Select the DEM.tif and DSM.tif rasters from the project folder then click OK. Remove the default "WorldElevation3D/Terrain3D" layer.
43. Any layer added to the 2D layers group will be draped over the elevation surface. Note that the DEM is the bare ground elevation and terrain, while the DSM includes the elevation of the above ground vegetation in addition to the ground. Turn the two elevation surface layers on and off to see the differences between the two.
44. Now add the Imagery Basemap to your 3D map. This will display a recent satellite or aerial image or the area. Try turning on and off the different layers and experimenting with the display options. Pick a layer to display in your final 3D map, either the imagery, the elevation or the canopy height.
45. The last step is to create a simple map layout with our 3D image. Create a new layout and select the 3D map as the Map Frame. Zoom out and rotate the map so the 3D nature is apparent. Add a title and north arrow to your map. Export your image by going to Share → Export Map. Save your image in your Lab 12 folder as a JPG.

Complete the Lab 12: Worksheet in Canvas

Lidar Derived Maps and Images (Upload to Canvas)

Upload the following to Canvas. Note: the first two can be combined into one map with side-by-side data frames.

• Map that shows the Elevation with shaded relief. Include title, legend, north arrow and scale (PDF/JPG).
• Map that shows the Canopy Height (CHM)with either the CHM or DSM shaded relief. Include title, legend, north arrow and scale (PDF/JPG).
• Exported 3D map image with imagery/elevation/canopy height (your choice) draped over 3-D surface model.