[GIS] Industry-based examples of using ArcPy in Python for Geoprocessing

Question

This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.

I was asked recently by some students what types of operations GIS Analyst/Developer are usually trying to automate by using Python with geoprocessing in ArcGIS and ArcPy site package. I guess it might be useful to know when searching for some exercises to do to be sure that what you create is relevant for the industry and might be re-used later already at the workplace.

The easiest answer would be "read the Esri help and go through the samples", but I was looking for more specific scenarios which are most common to implement. Thus, sharing workflows sush as "we are getting a .zip file with shapefiles, we use Python to unpack it, project them all to the X coordinate system, load into an ArcSDE geodatabase, grant users access to these data" are most welcome. Please feel comfortable to provide a concise description of the workflow, no extreme details are required.

Answer 1

For me, as your question suggests, I use Python a lot for automating batch processing in particular but also for creating any repeatable specialist calculations. These days I don't use ArcPy because I can't afford the ESRI licences as a freelance GIS Consultant. I use GDAL/OGR, Shapely, PostGIS, Numpy and SciPy a lot, though everything in my list could be done with ArcPy (and some of it was). Examples include:

1. Deriving Zonal Statistics for the whole of the UK that first requires mosaicing 20km raster tiles of two different data types, performing some "mapematics" on those rasters, merging the equivalent area of 10km vector polygon tiles, calculating the zonal stats of the result of the raster mapematics and joining the table of stats to the original vector data before outputting to shapefiles in a logical directory structure and burning to CD for the client.
2. Performing sequential visibility calculations every 100m along a road or track and then assigning the results of the calculation as M values back in the route data.
3. Automated process to creating 3D landscape models by mosaicing/merging tiles of raster and vector data, clipping to the required area and then converting to a proprietary (non-GIS) 3D format. I use the little Python library I developed for this a lot in my freelance work.
4. One huge project I worked on in a team used ArcPy to create batch processes to convert or derive new data from GIS data into a format with features which could be consumed by a procedural computer-game-asset-generator. The geoprocessing scripts were called by a batch processing 'slave-driver' also written in Python and running through Django.
5. Python is very useful even for little tasks, particularly where there is any repetition (e.g. feature by feature processing). ArcGIS' Model Builder is much improved with the flow-controls that came in in version 10 but even so, it still frequently can't provide the necessary control and/or it is quicker and easier just to write the process in ArcPy than try to coerce Model Builder.
6. I have created a tool in Python to perform swept-path analysis (to calculate whether a very long vehicle can follow a given route and where the trailer will be likely to get wedged in tight turns between buildings. This is another specialist tooll in by freelance arsenal.
7. Generating output from Mapnik
8. Before ArcGIS became multithreaded, I used Python to allow me to spawn subprocesses which could sometimes speed up long, slow calculations without the overhead of ArcMap cluttering up the memory.

Python in commercial geoprocessing is great because you have all the speed and brevity of scripting that Python provides and the speed of processing the compiled C-style code provides because, while Python is interpreted it is mostly calling compiled C-style code under the hood. Python provides the glue that can hold a lot of sequential geoprocessing tasks together and the list above is just a tiny snapshot of some of the things I personally use it for. In the 'Good Old Days' we would set up a Watch file and have ArcInfo record our command-line input and then clean up the AML (who remembers Arc Macro Language!) to make a reusable process of geoprocessing calls glued together with AML. It's not so different these days, except we use Python or C# as the glue.