Geopandas Left Join using Merge – Step-by-Step Guide

attribute-joinsgeopandas

Is there a way to do a "left join" when using the "merge" command on a geopandas df to merge by attribute?

The problem I am running into is that I am joining a geopandas df with CA counites to a pandas df that does not contain all the counites. However, I want the final geopandas df to contain all of the counties, but by default the merge command does an inner join. There is no information on the geopandas merge documnetation page: http://geopandas.org/mergingdata.html

Best Answer

When merging by attributes, you are actually using the pandas merge (or join) method or function (see its docstring: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html).

This method has a how keyword to specify whether to do a left/inner/outer/right join:

gdf.merge(df, on='key', how='left')

Related Solutions

GeoPandas Shapefile Attributes – Read Specific Attribute Columns of a Shapefile with GeoPandas/Fiona

Building on gene's answer, you can use GeoDataFrame.from_features

The following should do the trick:

import fiona

def records(filename, usecols, **kwargs):
    with fiona.open(filename, **kwargs) as source:
        for feature in source:
            f = {k: feature[k] for k in ['id', 'geometry']}
            f['properties'] = {k: feature['properties'][k] for k in usecols}
            yield f

And then

gpd.GeoDataFrame.from_features(records(filename), ['prop1', 'prop2'])

I'd be curious to know if this speeds up your code, or if the properties need to be ignored in Fiona rather than the GeoDataFrame construction.

[GIS] Finding problematic rows – spatial join in GeoPandas

There are various reasons why this error can occur, here are the ones I have experienced and the solutions:

Your input data sets do not have clean sequential indices (i.e. there are gaps in the sequence due to prior exclusion of rows).

I'm not sure exactly why this causes the error but it can be resolved by calling

pd.reset_index(drop=True)

on both input GeoDataFrames before applying sjoin.

There are invalid geometry objects in your polygons data frame.

If your polygons were drawn by hand (i.e. manually on a GIS) they may have overlaps or self-intersections that don't translate well further in the process. Or your polygons could be empty which can happen in PostGIS with complex function sequences.

The solution is to ensure that all your polygons are of the correct type and are valid. In PostGIS you can use the functions ST_IsValid and ST_IsEmpty to check for this and remove or amend any problems. You should also check that you have Polygons or MultiPolygons not GeometryCollections.

Best Answer

Related Solutions

GeoPandas Shapefile Attributes – Read Specific Attribute Columns of a Shapefile with GeoPandas/Fiona

[GIS] Finding problematic rows – spatial join in GeoPandas

Related Question