OpenStreetMap – Improving Tile Pre-Rendering Speed Using osm2pgsql

openstreetmaposm2pgsqlrendering

I have created a OSM tile serving server using osm2pgsql with entire asia tiles. I am using Ubuntu 18.04, 32 CPU cores, 132 GB RAM, 1 TB HDD. This I believe is a reasonable hardware. Now, I feel that de facto rendering of tiles is extremely slow for production environments when i cross zoom level 5-6. So in order to circumvent it I have started pre-rendering for zoom level 1-15.

I have used : render_list -m default -a -z 0 -Z 15 –num-threads=20

I feel that this process is extremely slow.

Here is a sample output :

Nov 02 14:56:02 test-virtual-machine renderd[19184]: DEBUG: Sending render cmd(3 default 8/104/112) with protocol version 2 to fd 31
Nov 02 14:56:40 test-virtual-machine renderd[19184]: DEBUG: DONE TILE default 8 104-111 128-135 in 117.281 seconds
Nov 02 14:56:40 test-virtual-machine renderd[19184]: DEBUG: Sending render cmd(3 default 8/104/128) with protocol version 2 to fd 32
Nov 02 14:56:54 test-virtual-machine renderd[19184]: DEBUG: DONE TILE default 8 104-111 136-143 in 129.875 seconds

As you can see quite frequently its taking 100 sec+ for iterations. Also it has been running for more than 3 hours now and I only see /var/lib/mod_tile/default with 109MB size and 929 files. I am assuming this is where the tiles will be extracted by the render_list process.

Is this standard behavior or are there things I can do to fasten the process?

Best Answer

I am using Ubuntu 18.04, 32 CPU cores, 132 GB RAM, 1 TB HDD.

Is this standard behavior or are there things I can do to fasten the process?

Swith to SSD, preferably NVMe. Many processes in OSM rendering and database processing need fast random access.