GeoPandas – Fixing ValueError in Python Spatial Join

geodataframegeopandaspythonspatial-joinvalueerror

I'm working on a spatial join between a polygon from a shapefile and points from a CSV file, using geopandas.sjoin. I want to assign each point with a name of the province it is located in. I do know how to do it in R, but I'm restricted to use Python in this context.
I found an example of a sjoin over here, so I tried it:

My data looks as follows:

import pandas as pd
import geopandas as gp
from shapely.geometry import Point
from geopandas import GeoDataFrame

#My observation points:
obs = pd.read_csv('data/butterflies.csv')
pts = gp.GeoDataFrame(obs, geometry=geopandas.points_from_xy(stations.lon, stations.lat))

pts.head()

#Polygons of some provinces in The Netherlands:
nlpoly = gp.read_file('data/provinces_NL.shp')
nlpoly

Now when I try the spatial join as was suggested in the example above, it returns me a ValueError:

gp.sjoin(pts, nlpoly[['name']],how='left',op='intersects')

>>ValueError: 'right_df' should be GeoDataFrame, got <class 'pandas.core.frame.DataFrame'>

I checked the types for both objects but they were all GeoDataFrames as it should be, yet sjoin does not recognize my rigth_df as a GeoDataFrame.
I guess the bug is in the referencing of the column "name" that I want to assign to the observation points, but I have no clue how to do it right.

Best Answer

If you do nlpoly[['name']] you get only DataFrame with one column name. You have to select geometry column as well to keep GeoDataFrame.

gp.sjoin(pts, nlpoly[['name', 'geometry']], how='left', op='intersects')

Related Solutions

[GIS] Converting Pandas DataFrame to GeoDataFrame

Convert the DataFrame's content (e.g. Lat and Lon columns) into appropriate Shapely geometries first and then use them together with the original DataFrame to create a GeoDataFrame.

from geopandas import GeoDataFrame
from shapely.geometry import Point

geometry = [Point(xy) for xy in zip(df.Lon, df.Lat)]
df = df.drop(['Lon', 'Lat'], axis=1)
gdf = GeoDataFrame(df, crs="EPSG:4326", geometry=geometry)

Result:

    Date/Time           ID      geometry
0   4/1/2014 0:11:00    140     POINT (-73.95489999999999 40.769)
1   4/1/2014 0:17:00    NaN     POINT (-74.03449999999999 40.7267)

Since the geometries often come in the WKT format, I thought I'd include an example for that case as well:

import geopandas as gpd
import shapely.wkt

geometry = df['wktcolumn'].map(shapely.wkt.loads)
df = df.drop('wktcolumn', axis=1)
gdf = gpd.GeoDataFrame(df, crs="EPSG:4326", geometry=geometry)

[GIS] Finding problematic rows – spatial join in GeoPandas

There are various reasons why this error can occur, here are the ones I have experienced and the solutions:

Your input data sets do not have clean sequential indices (i.e. there are gaps in the sequence due to prior exclusion of rows).

I'm not sure exactly why this causes the error but it can be resolved by calling

pd.reset_index(drop=True)

on both input GeoDataFrames before applying sjoin.

There are invalid geometry objects in your polygons data frame.

If your polygons were drawn by hand (i.e. manually on a GIS) they may have overlaps or self-intersections that don't translate well further in the process. Or your polygons could be empty which can happen in PostGIS with complex function sequences.

The solution is to ensure that all your polygons are of the correct type and are valid. In PostGIS you can use the functions ST_IsValid and ST_IsEmpty to check for this and remove or amend any problems. You should also check that you have Polygons or MultiPolygons not GeometryCollections.

Related Question