Python GeoPandas – Querying ESRI ArcGIS REST API Feature Service

arcgis-rest-apigeopandaspython

I would like to query a feature service from a public resource, such as the Department of Public Works, to see if they have any buildings near a latitude and longitude. However, whenever I query the resource I only get empty json responses or errors about how my query could not be completed. I know the goal is to create a URL, and I was trying to use the website to reverse engineer the query parameters. Then I would like to build the URL programaticaly with the Python requests and GeoPandas libraries.

Here is the REST service I am working with:

https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query

Can someone walk me through the basics of creating a query using a point lat/lon as a starting point and getting nearby facilities? The ESRI documentation is not friendly to newcomers.

I know I can just download the data source because it is so small. This is just a public feature service for the purpose of the question.

Best Answer

The easiest way to create a query to an ArcGIS service is to first use the HTML interface, i.e. open https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query in a browser, and then inspect the URL it creates after you click on the Query (GET) button.

Documentation for all query parameters can be found here.

The URL will contain a lot of unused parameters that you can ignore in many cases. For the most basic query, you only need to set the following parameters:

  • Where: 1=1 (return all features)
  • Out fields: * (return all fields)
  • Format: json (or pjson to get a nice readable json format, or even geojson if that suits your needs)
  • Return ID's only: False

So, the URL would be https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query?f=json&outfields=*&where=1%3D1

Most REST services will limit the returned data on an open query like the one just described. However, once you have the top 1000 rows you can see all the available data. One key piece of data which you might not have yet is the coordinate reference system (CRS). This is in the returned data as "wkid": 102645. You can read about this reference system here: http://epsg.io/102645.

Here is a screenshot of what that query returns in a JSON format and with the well known ID (CRS) highlighted.

Query JSON response with CRS highlighted

There are two ways to query features nearby a certain location. The easiest way is to specify a point and a distance, but not all services support this option. In the geometry below, the units are in the native CRS 102645. If you want to use the latitude and longitude from Google Maps, you will need to declare the CRS you are using in the input field "Input spatial Reference" as 4326.

  • Geometry: 6560874.3,1863451.1
  • Geometry Type: Point
  • Query By Distance: Any distance
  • Units: e.g. esriSRUnit_Meter
  • The Where option can be omitted if a geometry is set.

Note that the HTML interface incorrectly sets the name of the distance parameter to queryByDistance, and that it doesn't provide a field to set the units. Example: https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query?f=json&outfields=*&geometry=6560874.3,1863451.1&geometryType=esriGeometryPoint&distance=100&units=esriSRUnit_Meter

The other way is to generate a buffer around a point, and use that as geometry. This option should be supported by most, if not all, services. You may have to use a POST request rather than GET, because the URL may become too long due to the size of the geometries JSON.

  • Geometry: {"rings":[[[6560878,1863351],[6560869,1863351],[6560861,1863351],[6560852,1863353],[6560843,1863355],[6560835,1863358],[6560827,1863362],[6560819,1863367],[6560812,1863372],[6560805,1863378],[6560799,1863384],[6560794,1863391],[6560789,1863398],[6560784,1863406],[6560781,1863414],[6560778,1863422],[6560776,1863431],[6560774,1863440],[6560774,1863448],[6560774,1863457],[6560775,1863466],[6560777,1863475],[6560779,1863483],[6560783,1863491],[6560786,1863499],[6560791,1863507],[6560796,1863514],[6560802,1863521],[6560809,1863527],[6560816,1863532],[6560823,1863537],[6560831,1863541],[6560839,1863544],[6560848,1863547],[6560856,1863549],[6560865,1863550],[6560874,1863551],[6560883,1863550],[6560891,1863549],[6560900,1863547],[6560908,1863544],[6560917,1863541],[6560924,1863537],[6560932,1863532],[6560939,1863527],[6560945,1863521],[6560951,1863514],[6560956,1863507],[6560961,1863499],[6560965,1863491],[6560968,1863483],[6560971,1863475],[6560973,1863466],[6560974,1863457],[6560974,1863448],[6560973,1863440],[6560972,1863431],[6560970,1863422],[6560967,1863414],[6560963,1863406],[6560959,1863398],[6560954,1863391],[6560948,1863384],[6560942,1863378],[6560935,1863372],[6560928,1863367],[6560921,1863362],[6560913,1863358],[6560904,1863355],[6560896,1863353],[6560887,1863351],[6560878,1863351]]]}
  • Geometry Type: Polygon

Example: https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query?f=json&outfields=*&geometry=%7B%22rings%22%3A%5B%5B%5B6560878,1863351%5D,%5B6560869,1863351%5D,%5B6560861,1863351%5D,%5B6560852,1863353%5D,%5B6560843,1863355%5D,%5B6560835,1863358%5D,%5B6560827,1863362%5D,%5B6560819,1863367%5D,%5B6560812,1863372%5D,%5B6560805,1863378%5D,%5B6560799,1863384%5D,%5B6560794,1863391%5D,%5B6560789,1863398%5D,%5B6560784,1863406%5D,%5B6560781,1863414%5D,%5B6560778,1863422%5D,%5B6560776,1863431%5D,%5B6560774,1863440%5D,%5B6560774,1863448%5D,%5B6560774,1863457%5D,%5B6560775,1863466%5D,%5B6560777,1863475%5D,%5B6560779,1863483%5D,%5B6560783,1863491%5D,%5B6560786,1863499%5D,%5B6560791,1863507%5D,%5B6560796,1863514%5D,%5B6560802,1863521%5D,%5B6560809,1863527%5D,%5B6560816,1863532%5D,%5B6560823,1863537%5D,%5B6560831,1863541%5D,%5B6560839,1863544%5D,%5B6560848,1863547%5D,%5B6560856,1863549%5D,%5B6560865,1863550%5D,%5B6560874,1863551%5D,%5B6560883,1863550%5D,%5B6560891,1863549%5D,%5B6560900,1863547%5D,%5B6560908,1863544%5D,%5B6560917,1863541%5D,%5B6560924,1863537%5D,%5B6560932,1863532%5D,%5B6560939,1863527%5D,%5B6560945,1863521%5D,%5B6560951,1863514%5D,%5B6560956,1863507%5D,%5B6560961,1863499%5D,%5B6560965,1863491%5D,%5B6560968,1863483%5D,%5B6560971,1863475%5D,%5B6560973,1863466%5D,%5B6560974,1863457%5D,%5B6560974,1863448%5D,%5B6560973,1863440%5D,%5B6560972,1863431%5D,%5B6560970,1863422%5D,%5B6560967,1863414%5D,%5B6560963,1863406%5D,%5B6560959,1863398%5D,%5B6560954,1863391%5D,%5B6560948,1863384%5D,%5B6560942,1863378%5D,%5B6560935,1863372%5D,%5B6560928,1863367%5D,%5B6560921,1863362%5D,%5B6560913,1863358%5D,%5B6560904,1863355%5D,%5B6560896,1863353%5D,%5B6560887,1863351%5D,%5B6560878,1863351%5D%5D%5D%7D&geometryType=esriGeometryPolygon

If you want to use python to build a query, here's how.

import requests
import urllib.parse
import geopandas as gpd

url_base = r'https://dpw.gis.lacounty.gov/dpw/rest/services/PW_Open_Data/MapServer/22/query?'

Note the addition of the ? at the end of the URL.

Now define all your parameters.

params = {
    'geometry': '-118.21637221791077, 34.094916196179504',
    'geometryType': 'esriGeometryPoint',
    'inSR': '4326',
    'distance': '10000', 
    'units': 'esriSRUnit_Meter', 
    'returnGeometry': 'true', 
    'outFields': 'OBJECTID,DIVISION,FACILITY_N,ADDRESS,CITY', 
    'f': 'pjson'
}

Encode the URL:

url_final = url_base + urllib.parse.urlencode(params)

Use requests to query the server:

response = requests.get(url=url_final)

Convert the response to text so geopandas can read it:

data = response.text

Now load it into a geodataframe:

gdf_temp = gpd.read_file(data)
Related Question