[GIS] Query one overpass api process multiple times

localhostopenstreetmapoverpass-api

I using a local instance of the overpass api, because I need to make a lot of queries (over 100k). The solution I'm using right now is to call bin/osm3s_query for every single query.

This is very slow, because osm3s_query has to start, answer the query and shut down for every single query.

Is there a way of having some kind of daemon waiting for my queries?

I guess if osm3s_query doesn't have to start and shutdown for every single query, this should accelerate things a lot.
The "daemon" for running a web server seems to do exactly what I'm doing right now (firing up one osm3s_query process for every query). Therefore I couldn't find anything helpful in the documentation

Best Answer

The best option in your case is probably to fetch all traffic light at once according to your gps track's bounding box. Cache that result locally in whatever language you're processing the Overpass API result and you're probably all set. Don't worry about the size, even a 300km by 300km area just takes some seconds to run.

Right now, there's no option in stock Overpass API to handle several requests in a row. Apache calls the interpreter binary via CGI for each single query. Usually, that's not that much of a concern, as most queries take more time, so start up costs are negligible.

BTW: There's also a FastCGI prototype for Overpass API on Github to reduce some of the startup costs. However, even with the support of FastCGI, each query usually has some specific fix cost in the range of maybe 50+ milliseconds, and as data needs to be fetched from disk (resp. buffer cache) and decompressed for each single query, you're better off with fewer but larger queries.

Related Solutions

Overpass API Nodes – How to Get Nodes in Correct Order

(I edited my answer for your follow-up questions)

You have to use the order in which the node IDs are referenced by the corresponding <way> element in the XML file. In contrast, the order in which the <nodes> elements appear in the XML file is completely irrelevant.

Furthermore, as you already discovered, a relation can contain multiple separate ways forming a single polygon. Here, the relation doesn't necessarily has to reference the ways in the correct order. Consequently the correct order of the ways has to be determined by looking at the first and last node of the way. Consecutive ways will share the same last and first node.

For a neat sorting algorithm you can take a look at the source code of JOSM.

Alternatively you might take a look at overpass turbo which provides a GeoJSON output which Leaflet should be able to handle.

[GIS] Overpass API query error: query ran out of memory

By default Overpass API has a certain memory limit in place, to control the overall memory consumption of queries. However, you can easily override that default by providing the maxsize parameter, like shown in the following example. The increased maxsize limit would allow queries up to 2GB. The maximum possible value is a configuration setting on the server.

/*
This is a simple map call.
It returns all data in the bounding box.
*/
[out:xml][maxsize:2000000000];
(
  node({{bbox}});
  <;
);
out meta;

Also, downloading huge amounts of data in overpass turbo and displaying the results in the browser window is highly discouraged, as it will likely crash your browser. Be sure to use Export -> raw data directly from Overpass API instead to save the results in a file.

Best Answer

Related Solutions

Overpass API Nodes – How to Get Nodes in Correct Order

[GIS] Overpass API query error: query ran out of memory

Related Question