Batch Processing With Python

1. Batch Processing

Batch processing typically refers to processing a "batch" of files. The files are usually very similar such as DEMs or satellite images for a large area. The processing we'll look at here will work for a few files or thousands. Here is a tiny DEM to use for this.

The approach we'll use for batch processing (my personal favorite) will be to process all the files in a folder into another folder. The code below will use the "os" function "listdir()" to get a list of the contents of the target folder. One challenge is that folders, especially created by ArcGIS, will have lots of different file types and folders. We're using IMG files partly because they will be relatively easy to find in the folder. The "os.path" package has functions that let us determine if one of the elements in the folder is a directory or a file. Then we can "split" the file name to get it's extension.

import os

# set the input to the path where the original files are

InputPath="C:/Temp/Inputs/"
OutputPath="C:/Temp/Outputs/"

# get a list of all the files in the input folder
TheList=os.listdir(InputPath)

for TheFile in TheList:
	# break up the file name into pieces based on periods
	TheFileName, TheFileExtension = os.path.splitext(TheFile)

	# create the full path to the file
	InputFilePath=InputPath+TheFileName+TheFileExtension

	# determine if this is a file (not a directory), has just a name and extension, and the extension is "img"
	if (TheFileExtension==".img"):
		print("Found a file to process:"+InputFilePath)

		# put additional processing steps here

Now we can add any processing we like into the "if" statement and it will be executed on all the files in the "Inputs" folder.

Additional Resources

Python Documentation: OS