[GIS] Handling large datasets with Erase/Dissolve in ArcGIS (Dissolve alternative?)

arcgis-desktopdissolveeraseperformance

I have what should be a simple problem: erase a large multipart polygon layer from a larger polygon (see below for example). However, I believe the Arc Erase tool is trying to erase each part of the multipart from the larger polygon. I'm currently pushing 24 hours processing with ample storage and 12gb ram…

I have tested the Erase vs. Dissolve –> Erase workflow on a smaller dataset, and Erase runs substantially faster with the single dissolved polygon (~2 minutes for Erase with multipart, ~1 second with single feature in test dataset). Unfortunately, Dissolve also takes around 2 minutes with the same dataset.

So, is there an alternative to Dissolve that will return the outer extent of my feature class? I can not simplify the boundary with a convex hull or minimum bounding extent, I just need the feature mask. I'm open to python options as well.

UPDATE: Unfortunately, none of the provided answers gave me any great improvement in processing. I ended up buffering the inner dataset by 1 cm (smallest integer value available in Arc Buffer tool, and difference between inner and outer layers is ~150m) and erasing with that single polygon dataset.

Best Answer

An alternative to dissolve is to create lines, keep the outer boundaries and then build polygon from the outer boundaries. This will also take some time, but it should be a little bit faster than dissolve because you don't merge all the small polygons.

In practice, you can identify the outer boundaries based on the -1 attribute after running "Polygon to line" (not feature to line).

Related Solutions

ArcGIS – Combining Adjacent Polygons with Similar Traits

Try the Dissolve tool. From the help:

Use the Dissolve tool when you want to aggregate features based on a specified attribute or attributes. For example, you could take a feature class containing sales data collected on a county-by-county basis and use Dissolve to create a feature class containing contiguous sales regions based on the name of the salesperson in each county. Dissolve creates the sales regions by removing the boundaries between counties represented by the same salesperson.

_{(source: arcgis.com)}

To avoid creating multi-part features, uncheck the "Create multipart features" box.

R – Fast Algorithm to Calculate Difference Between Two Sets of Polygons Comparable to QGIS

The rmapshaper package offers a faster function for "erasing" or finding the difference between two polygon layers in R. Below is a comparison of the st_erase approach with sf and the ms_erase function in rmapshaper.

# Let's try an example using publicly available Census vector layers through `tigris`
if(!require(dplyr)){install.packages("dplyr")}
library(dplyr)
if(!require(sf)){install.packages("sf")}
library(sf)
if(!require(tigris)){install.packages("tigris")}
library(tigris)
if(!require(rmapshaper)){install.packages("rmapshaper")}
library(rmapshaper)

# First download two separate layers that overlap. We'll use a layer of Census 
# tracts and a layer of areal water features for Worcester County in Massachusetts.
# worcester_tracts has 172 obs and  10 variables; about 285KB in size.
worcester_tracts <- tracts(state = "MA", county = "Worcester", cb = TRUE) %>% 
  st_transform(., crs = 2805) %>% # use projected layers when doing geometric stuff
  st_make_valid()
# worcester_water has 3,270 obs and 9 variables; about 4.9MB in size. 
worcester_water <- area_water(state = "MA", county = "Worcester") %>% 
  st_transform(., crs = 2805) %>% 
  st_make_valid()

# Start with st_erase function. Note that I altered function slightly by removing 
# the st_combine function because it throws an error and isn't necessary. 
# Takes about 27 secs.
st_erase = function(x, y) st_difference(x, st_union(y))
system.time(erased_tracts1 <- st_erase(worcester_tracts, worcester_water))

# Now try it with ms_erase. Takes about 4.3 secs; over 6x faster. 
system.time(erased_tracts2 <- ms_erase(worcester_tracts, worcester_water))

The results are identical. You can speed things up even more by retaining as few variables, or attributes, as possible in the layers when doing these operations. Also, if your computer memory is being overwhelmed, consider breaking up the process into smaller geographic chunks (e.g., municipalities, counties, a grid), rather than working on the whole dataset at one time.

Related Question