[GIS] Conversion of netcdf file to csv file using R programming

csvnetcdfr

I have NCEP/NCAR reanalysis data in netcdf format. I want to convert it into csv file. I have a script to convert netcdf to csv. But since the data in netcdf file is too large which exceeds number columns in csv file. How can i split the data in nc file to multiple sheets of csv file.

Script that i have used to convert netcdf file to csv is as follows:

library(ncdf4)
getwd()
workdir <- "I:\\NCEP\\"
setwd(workdir)
ncin <- nc_open("X27.251.228.22.106.3.5.59.nc")
print("ncin")
dname <- "hgt"
lon <- ncvar_get(ncin, "lon")
nlon <- dim(lon)
head(lon)
lat <- ncvar_get(ncin, "lat", verbose = F)
nlat <- dim(lat)
head(lat)
print(c(nlon, nlat))
t <- ncvar_get(ncin, "time")
tunits <- ncatt_get(ncin, "time", "units")
nt <- dim(t)
tmp.array <- ncvar_get(ncin, dname)
dlname <- ncatt_get(ncin, dname, "long_name")
dunits <- ncatt_get(ncin, dname, "units")
fillvalue <- ncatt_get(ncin, dname, "_FillValue")
dim(tmp.array)
title <- ncatt_get(ncin, 0, "title")
institution <- ncatt_get(ncin, 0, "institution")
datasource <- ncatt_get(ncin, 0, "source")
references <- ncatt_get(ncin, 0, "references")
history <- ncatt_get(ncin, 0, "history")
Conventions <- ncatt_get(ncin, 0, "Conventions")
# split the time units string into fields
tustr <- strsplit(tunits$value, " ")
tdstr <- strsplit(unlist(tustr)[3], "-")
tmonth = as.integer(unlist(tdstr)[2])
tday = as.integer(unlist(tdstr)[3])
tyear = as.integer(unlist(tdstr)[1])
chron::chron(t, origin = c(tmonth, tday, tyear))
tmp.array[tmp.array == fillvalue$value] <- NA
length(na.omit(as.vector(tmp.array[, , 1])))
m <- 1
tmp.slice <- tmp.array[, , m]
lonlat <- expand.grid(lon, lat)
tmp.vec <- as.vector(tmp.slice)
length(tmp.vec)
tmp.df01 <- data.frame(cbind(lonlat, tmp.vec))
names(tmp.df01) <- c("lon", "lat", paste(dname, as.character(m), sep = "_"))
head(na.omit(tmp.df01), 20)
csvfile <- "cru_tmp_1.csv"
write.table(na.omit(tmp.df01), csvfile, row.names = FALSE, sep = ",")
tmp.vec.long <- as.vector(tmp.array)
length(tmp.vec.long)
tmp.mat <- matrix(tmp.vec.long, nrow = nlon * nlat, ncol = nt)
dim(tmp.mat)
head(na.omit(tmp.mat))
# create a dataframe
lonlat <- expand.grid(lon, lat)
tmp.df02 <- data.frame(cbind(lonlat, tmp.mat))
options(width = 110)
head(na.omit(tmp.df02, 20))
csvfile <- "cru_tmp_2.csv"
write.table(na.omit(tmp.df02), csvfile, row.names = FALSE, sep = ",")

Best Answer

Use something like this, with the high-level tools in the raster package: 

library(raster)
r <- brick("X27.251.228.22.106.3.5.59.nc", varname = "hgt")
r

How does that look? If all seems well, try a plot(r[[1]]) and make sure everything's interpreted correctly, then test 

tab <- as.data.frame(r[[1]], xy = TRUE)

Then if all is well, consider taking out the 1-st index subsetting there. See ?raster::as.data.frame for long and wide shape options. Once in this data.frame/tab form you can easily write to tabular formats.

The fact is that this format is so general, and the way that conventions within that format get used is so open that you almost always will need to do some upfront investigation and massaging. But once you've done that everything can be really simple and efficient.

I highly recommend using high level facilities like raster, but the upfront cost is in ensuring that the interpretation is correct. There are (some hidden) facilities in raster to deal with almost any 2D+ variable.

Related Question