MATLAB: Datastore for custom files

datastorefiledatastorefopenfreadhadoop

I am trying to work with Hadoop and the datastore object. However, the file to be read is binary, hence up until now, I have used fopen and fread to read in the stream of binary. This does not work with datastore, but should work with fileDatastore. So this is what I did:
fds = fileDatastore(fname{1},'ReadFcn',@testfcn)
read(fds);
function output=testfcn(fileName)
    fileID = fopen(fileName);
    output=fread(fileID,'ubit1');
end
During the debugging, I found out that the file actually has been successfully loaded. But I keep gettin an error message at the end:
Warning: The following error was caught while executing
'matlab.io.datastore.splitreader.WholeFileCustomReadSplitReader' class
destructor:
No directories were removed. 
> In matlab.io.datastore.SplittableDatastore/delete (line 108)
  In matlab.io.datastore.FileDatastore/readall (line 11) 
Error using matlab.io.datastore.FileDatastore/readall (line 21)
No directories were removed.
Does anyone know what the problem is? Thanks!

Best Answer

For files on Hadoop, the File Datastore makes a temporary copy of the file on the local drive to be read by the ReadFcn. The ReadFcn here, testfcn, opens the file ID but does not close it afterwards. This prevents the File Datastore from being able to clean up the temporary copy of the file.
Closing the file ID after use should resolve this error:
function output=testfcn(fileName)
    fileID = fopen(fileName);
    output=fread(fileID,'ubit1');
    fclose(fileID);
end
Related Question