[GIS] Pulling a CSV file from website into QGIS with ability to automatically refresh

csvqgis

I am attempting to pull a CSV file from a website that is updated daily. I would like to have a process that can automatically perform the process and save daily as a new layer. The CSV file is related to tracking forest fires and has multiple attributes. The CSV download url is http://www.wildfire.alberta.ca/reports/activedd.csv. I attempted to use the a vrt driver to load the file into QGIS as a layer but I was unable to pull the attributes as well as the lat/long locations.

I am fairly new to QGIS.

Best Answer

You can load the CSV file directly (or have ftp load the file to your local drive) QGIS will detect the change and reload the data on the fly.

If you want it to change use the 'watch file'

enter image description here

result enter image description here

use wget to get the csv (every minute,hour,day etc) (task scheduler)

enter image description here

Related Solutions

[GIS] Pulling CSV file directly from website into QGIS

You could use an OGR Virtual Format.

This link has a section called "Reading CSV containing spatial information" which shows how to create a VRT file that reads a CSV file with a lat/lon column. This example is a basic template but could be manipulated to suit your case.

Basically you copy the code below into a text file and save it with a file extension of ".vrt". You can then load this into QGIS and theoretically each time you load the VRT file you should see the latest version of the CSV contents. I assume you could change the datasource to be a http end point, I haven't tried that yet. But consuming local csv files this way works.

<OGRVRTDataSource>
<OGRVRTLayer name="test">
    <SrcDataSource relativeToVRT="1">test.csv</SrcDataSource>
    <GeometryType>wkbPoint</GeometryType>
    <LayerSRS>WGS84</LayerSRS>
    <GeometryField encoding="PointFromColumns" x="Longitude" y="Latitude"/>
</OGRVRTLayer></OGRVRTDataSource>

QGIS CSV to Shapefile – How to Join and Export

You don't want to export csv, you should Save as... the shapefile which you joined the csv to. If you export csv table (which was not loaded with points geometry), there won't be any geometry and thus no shapefile.

So it should go like this:

drag and drop shapefile and csv into QGIS
Double click on shapefile in layers list, switch to joins tab
join csv to shapefile table through common attribute
right click on shapefile and Open attribute table to make sure it is joined as expected (now actually this might cause potentionally troubles because shapefiles/respectively dbf tables have only 10 characters limitation for column names and also limit for absolute length - good idea is rename csv to some very short name - only one character or so before starting the whole operation, or use the custom field name prefix when doing join)
once all look good right click and Save as... on shapefile, by default QGIS will automatically load the new file after save
- If you want to save only features with joined information from csv - meaning your csv has IDs only of some smaller subset of features:
- Select only the ones joined through expression: "joined_column_name" is not NULL
- Then click on Save as... and check Save only selected features

(If you try to save csv it will create only dbf (and projection files proj, qpj)) but won't create shapefile because csv is only table, which can be joined to shapefile but not the other way around - thus it will come up with error because there won't be any shp to load)

Best Answer

Related Solutions

[GIS] Pulling CSV file directly from website into QGIS

QGIS CSV to Shapefile – How to Join and Export

Related Question