Is it necessary to do an accuracy assessment for an unsupervised classification done for a Landsat 5 image?
[GIS] Accuracy assessment in Erdas imagine
accuracyerdas-imagineremote sensing
Related Solutions
As many on this forum know, I am often for an R solution. However, in this case it is reinventing the wheel, and in a much less robust way. There is a great piece of free software, Map Comparison Kit (MCK), that implements many published and novel validation statistics for rasters. Of particular interest in this case are the Kappa, fuzzy Kappa and weighted Kappa.
Now, if you want to implement something in R there are many approaches you can take that depend on the complexity of the validation statistic. In a univariate case you can easily pass a function to "focal" to calculate uncertainty within a defined neighborhood. Moving into a bivariate case, you would want to vectorize the problem and define a function that would take two independent data into account. I do not believe that "movingFun" or "focal" will take two rasters into account. You can however, use "overlay", "getValuesBlock" or ideally"getValuesFocal" all of which will operate on stack/block objects.
Here is a worked example of calculating Kappa, using a 3x3 window, with "getValuesFocal". In the for loop the lapply function is reclassifying simulated probabilities [p >= t |1| else |0|], The parameter to adjust the sensitivity is "p" and "ws" adjust the size of the focal window extracted. I wrote this to be memory safe so, it writes a file ("Kappa.img") to disk in the defined working directory.
require(raster)
require(asbio)
setwd("D:/TEST")
ws <- 3 # window size
p=0.65 # probability threshold
# Create example data
pred <- raster(ncol=100, nrow=100)
pred[pred] <- runif(length(pred[pred]),0,1)
obs <- pred
obs[obs] <- runif(length(pred[pred]),0,1)
obs.pred <- stack(obs,pred)
names(obs.pred) <- c("obs","pred")
# Create new on-disk raster
s <- writeStart(obs.pred[[1]], "Kappa.img", overwrite=TRUE)
tr <- blockSize(obs.pred)
options(warn=-1)
# Loop to read raster in blocks using getValuesFocal
for (i in 1:tr$n) {
# Get focal values as list matrix object
v <- getValuesFocal(obs.pred, row=tr$row[i], nrows=tr$nrows[i],
ngb=ws, array=FALSE)
# reclassify data to [0,1] using lapply
v <- lapply(v, FUN=function(x) {
if( length(x[is.na(x)]) == length(x) ) {
return( NA )
} else {
return( ifelse(x >= p, 1, 0) )
}
}
)
# Loop to calculate Kappa and assign to new raster using writeValues
r <- vector()
for( j in 1:dim(v[[1]])[1]) {
Obs <- v[[1]][j,]
Obs <- Obs[!is.na(Obs)]
Pred <- v[[2]][j,]
Pred <- Pred[!is.na(Pred)]
if( length(Obs) >= 2 && length(Obs) == length(Pred) ) {
r <- append(r, Kappa(Pred, Obs)$khat)
} else {
r <- append(r, NA)
}
}
writeValues(s, r, tr$row[i])
}
s <- writeStop(s)
k <- raster("Kappa.img")
plot(k)
The User's Accuracy is the reliability of the classes in the classified image. It is calculated as the fraction of correctly classified pixels with respect to all pixels classified as this class in the image. For instance, based on your sample data, the User Accuracy for:
UA_Class1 is : 3 / 3+3 = 50
UA_Class2 is : 7 / 7 + 1 = 87.5
Therefore, to answer your question, it would seem that the User and Producer's accuracies from the asbio package are switched.
For a thorough explanation of the Error (Confusion) Matrix and related calculations, I refer you to this excellent technical note by D. Rossiter. I have included an excerpt that formally defines the UA.
It is expressed from the point of view of the mapper. Looking across the rows (classes as mapped), an error of commission is said to occur when the mapper incorrectly mapped this class at a reference (ground truth) site where it does not exist. That is, the mapper ‘committed’ the error of over-mapping a class. This leads to a lower user’s ‘accuracy’Ci.
Best Answer
When an accuracy assessment is being performed an error matrix would be generated and kappa. From there you would notice that omission and commision error is there from which the producer and user accuracy can be calculated from the formula given below:
Producer Accuracy = 100 - omission error and User Accuracy = 100 - commission error
From the calculated values above if the user accuracy is between 70-100% then you are good to go otherwise you have to reclassify for good results.