[GIS] Where to find street names for each city in US

dataopenstreetmapstreets

I'm sorry if my question is stupid but I had to ask.

I'm building a database of US states, cities, streets AND their basic relations to each other. I already have state and city tables.

Where can I find street data to purchase?

I know nothing about GIS and I'm too busy to learn it. So I have to buy it i believe.

Best Answer

You might be able to extract some information or use Open Street Maps as a Base.

Here you can search for some datasets through data.gov

ArcGis provides street and highways at a certain resolution.

A quick google search may yield you more results depending on what exactly you're looking for. But the above should serve you well!

There may be other resources for actual purchase, be sure to read the fine print on each of these websites depending on the purposes of your project.

Options that won't work well

Hadoop

Hadoop and similar tools aren't the solution, as it's entirely possible to do this type of analysis on a reasonably powerful server. You may not have a reasonably powerful server, in which case Hadoop wouldn't be a good option since it needs a cluster.

If you happen to have a Hadoop cluster and are an expert in using it, it's reasonable, but otherwise it's more development time for no gain.

Vector tiles

Vector tiles don't remove any processing steps, they just allow some of the work to be shared by multiple styles. As you've seen with Mapbox Streets' styling, buildings aren't often in low-zoom vector tiles, so you'd have to generate them yourself.

You could stitch together low-zoom vector tiles, but you'd have to use your own rendering toolchain for that, and it would be complex.

Reasonable options

OSM has about 160 million building ways, 35 million addresss nodes, and 21 million ways with addresses. Most of the last are also buildings.

osm2pgsql

osm2pgsql can handle this on reasonable hardware, if you take care to exclude other data. To do this you want a custom .style file which includes only address and building tags. Starting with empty.style, the suggested starting point, we can get

node,way addr:unit text linear node,way addr:housename text linear node,way addr:housenumber text linear node,way addr:street text linear way building text polygon

Everything below here is copied from empty.style, with the "building" and z_order lines removed. The former is above as an actual column, the latter isn't relevant for this use way_area is included, as it is useful

way abandoned:aeroway text phstore way abandoned:amenity text phstore way abandoned:building text phstore way abandoned:landuse text phstore way abandoned:power text phstore way area:highway text phstore node,way aeroway text phstore node,way amenity text phstore way building:part text phstore node,way harbour text phstore node,way historic text phstore node,way landuse text phstore node,way leisure text phstore node,way man_made text phstore node,way military text phstore node,way natural text phstore node,way office text phstore node,way place text phstore node,way power text phstore node,way public_transport text phstore node,way shop text phstore node,way sport text phstore node,way tourism text phstore node,way water text phstore node,way waterway text phstore node,way wetland text phstore way way_area real linear # This is calculated during import

Save it as buildings.style

You can then import the planet, using something based on the suggested osm2pgsql command line for the planet osm2pgsql -c -d buildings --style /path/to/buildings.style --slim --drop -C <cache size> --flat-nodes <flat nodes> /path/to/planet-latest.osm.pbf

where

<cache size> is 24000 on machines with 32GiB or more RAM or about 75% of memory in MiB on machines with less
<flat nodes> is a location where a 24GiB file can be saved.

There's a couple of different options used

--drop gets rid of tables used only during the import and during updates, as I'm assuming you'll update by reimporting

--style /path/to/buildings.style specifies to use the style we wrote above

This will take a day or two on a reasonably powered server.

Once the import is done, there's a couple indexes you can add which will help performance

CREATE INDEX planet_osm_polygon_area_18250_idx ON planet_osm_polygon USING gist (way) WHERE way_area > 18250; CREATE INDEX planet_osm_polygon_area_1140_idx ON planet_osm_polygon USING gist (way) WHERE way_area > 1140; CREATE INDEX planet_osm_polygon_area_71_idx ON planet_osm_polygon USING gist (way) WHERE way_area > 71;

When defining the layers in Kosmtik, Tilemill, or another map design studio, include the condition WHERE way_area > 0.05*!pixel_width!::real*!pixel_height!::real on any polygon layers.

Some things to watch for when visualizing

Rendering lots of little polygons doesn't work well. That, as well as performance, is why there's the area cutoff for layers.
It's still going to be slow, but pre-rendering the US should be entirely reasonable
Test on a small area first

https://switch2osm.org/loading-osm-data/ has more information on setting up PostgreSQL and installing osm2pgsql

libosmium

libosmium is a library for working with OSM data, and probably the best option for operating directly on the planet file.

QGIS or ArcGIS

If using this much data in QGIS or ArcGIS, you probably want to script it, but you could do a more sophisticated analysis.

osm2pgsql multi-backend

This is very similar to the above, but features a more flexable backend, which can result in tables better suited for exporting to another format.

[GIS] Downloading OpenStreetMap data with street names and city attributes for ArcGIS Desktop

To do this I would use the tools of ArcGIS Editor for OpenStreetMap to download a file geodatabase in your area of interest.

OSM Editor makes it easy for you to download OSM data, use it for analysis in your research and maps, and upload changes back to the OSM database if desired.

If you prefer shapefiles over the feature classes that it creates then they are easily converted.