[GIS] Pandas dataframe to Shapely LineString using GroupBy & SortBy

linestringpandaspolyline-creationpythonshapely

I have a pandas dataframe that contains information to construct (poly)lines, and I want to use shapely & geopandas tools to make a SHP.

In the example below, I have 3 lines differentiated by "myid" and the order of the vertices is in "myorder."

Making shapefile from Pandas dataframe? is a great explanation for making a point shapefile, but I am looking for a polyline SHP. Creating Shapely LineString from two Points let's me know I need to use from shapely.geometry import LineString to make the polylines, but I don't understand from the answer there (nor the shapely documentation) how to specify groupby("myid") and sortby("myorder").

How would I do this?

Using Windows 10, Python 3.7.6, Conda 4.6.14.

myid = [1, 1, 1, 2, 2, 3, 3]
myorder = [1, 2, 3, 1, 2, 1, 2]
lat = [36.42, 36.4, 36.4, 36.49, 36.48, 36.39, 36.39]
long = [-118.11, -118.12, -118.11, -118.09, -118.09, -118.10, -118.11]
df = pd.DataFrame(list(zip(myid, myorder, lat, long)), columns =['myid', 'myorder', 'lat', 'long']) 
display(df)

Best Answer

You can do this with geopandas by building a geodataframe, then sorting and grouping and applying a lambda to build the lines.

import pandas as pd
import geopandas as gpd
from shapely.geometry import LineString

myid = [1, 1, 1, 2, 2, 3, 3]
myorder = [1, 2, 3, 1, 2, 1, 2]
lat = [36.42, 36.4, 36.4, 36.49, 36.48, 36.39, 36.39]
long = [-118.11, -118.12, -118.11, -118.09, -118.09, -118.10, -118.11]
df = pd.DataFrame(list(zip(myid, myorder, lat, long)), columns =['myid', 'myorder', 'lat', 'long']) 

# Convert to GeoDataFrame
gdf = gpd.GeoDataFrame(
    df, geometry=gpd.points_from_xy(df['long'], df['lat']))

display(gdf)

# Sort and group points to make lines
line_gdf = gdf.sort_values(by=['myorder']).groupby(['myid'])['geometry'].apply(lambda x: LineString(x.tolist()))
line_gdf = gpd.GeoDataFrame(line_gdf, geometry='geometry')

display(line_gdf)

# Write out
line_gdf.to_file("lines.shp")

Related Solutions

[GIS] Making shapefile from Pandas dataframe

Yes, that can be done with shapely and geopandas.

Supposed that your pandas dataframe kind of looks like this:

import pandas as pd
data = [
        {'some_attribute': 'abc', 'lat': '50.1234', 'lon': '10.4023'},
        {'some_attribute': 'def', 'lat': '40.5678', 'lon': '8.3365'},
        {'some_attribute': 'ghi', 'lat': '60.9012', 'lon': '6.2541'},
        {'some_attribute': 'jkl', 'lat': '45.3456', 'lon': '12.5478'},
        {'some_attribute': 'mno', 'lat': '35.7890', 'lon': '14.3957'},
        ]

df = pd.DataFrame(data)
print(df)

=>

       lat      lon some_attribute
0  50.1234  10.4023            abc
1  40.5678   8.3365            def
2  60.9012   6.2541            ghi
3  45.3456  12.5478            jkl
4  35.7890  14.3957            mno

First, make sure that geopandas and shapely are installed properly which sometimes is not easy because they come with some dependencies (e.g. GEOS and GDAL). If does not work at first try via pip install geopandas shapely, search for the error on Google or StackOverflow/Gis.Stackexchange because most probably there will be an answer available solving that problem for you.

Then, it is just a matter of creating a new geometry column in your dataframe which combines the lat and lon values into a shapely Point() object. Note that the Point() constructor expects a tuple of float values, so conversion must be included if the dataframe's column dtypes are not already set to float.

from shapely.geometry import Point

# combine lat and lon column to a shapely Point() object
df['geometry'] = df.apply(lambda x: Point((float(x.lon), float(x.lat))), axis=1)

Now, convert the pandas DataFrame into a GeoDataFrame. The geopandas constructor expects a geometry column which can consist of shapely geometry objects, so the column we created is just fine:

import geopandas
df = geopandas.GeoDataFrame(df, geometry='geometry')

To dump this GeoDataFrame into a shapefile, use geopandas' to_file() method (other drivers supported by Fiona such as GeoJSON should also work):

df.to_file('MyGeometries.shp', driver='ESRI Shapefile')

And that is what the resulting shapefile looks like when visualized with QGIS:

[GIS] Assign a point to polygon using pandas and shapely

If you're working with spatial data and Pandas you should take a look at GeoPandas.

The example below demonstrates how to perform a spatial join in GeoPandas (which uses Shapely). A GeoDataFrame object is created from a list of cities and their coordinates and is joined to an ESRI Shapefile containing countries.

import pandas
import geopandas
import geopandas.tools
from shapely.geometry import Point

# Create a DataFrame with some cities, including their location
raw_data = [
    ("London", 51.5, -0.1),
    ("Paris", 48.9, 2.4),
    ("San Francisco", 37.8, -122.4),
]
places = pandas.DataFrame(raw_data, columns=["name", "latitude", "longitude"])

# Create the geometry column from the coordinates
# Remember that longitude is east-west (i.e. X) and latitude is north-south (i.e. Y)
places["geometry"] = places.apply(lambda row: Point(row["longitude"], row["latitude"]), axis=1)
del(places["latitude"], places["longitude"])

# Convert to a GeoDataFrame
places = geopandas.GeoDataFrame(places, geometry="geometry")

# Declare the coordinate system for the places GeoDataFrame
# GeoPandas doesn't do any transformations automatically when performing
# the spatial join. The layers are already in the same CRS (WGS84) so no
# transformation is needed.
places.crs = {"init": "epsg:4326"}

# Load the countries polygons
countries = geopandas.GeoDataFrame.from_file("ne_110m_admin_0_countries.shp")
# Drop all columns except the name and polygon geometry
countries = countries[["name", "geometry"]]

# Perform the spatial join
result = geopandas.tools.sjoin(places, countries, how="left")

# Print the results...
print(result.head())

Note that the spatial join feature is still fairly new and hasn't made it into the stable branch yet - you'll need to download and install the development version.

https://github.com/geopandas/geopandas

The result looks like this:

       name_left             geometry  index_right      name_right
0         London    POINT (-0.1 51.5)           57  United Kingdom
1          Paris     POINT (2.4 48.9)           55          France
2  San Francisco  POINT (-122.4 37.8)          168   United States

You can also use GeoPandas to plot the data with matplotlib:

import matplotlib.pyplot as plt
fig, ax = plt.subplots(1)
countries.plot(ax=ax, color="#cccccc")
places.plot(ax=ax, markersize=5, color="#cc0000")
plt.show()

Best Answer

Related Solutions

[GIS] Making shapefile from Pandas dataframe

[GIS] Assign a point to polygon using pandas and shapely

Related Question