MATLAB: Matlab using Java openStream(url) get :Java exception occurred: java.lang.NullPointerException

java openstream exception

Hi,

I want to download one file from a URL. According to this: http://www.mathworks.com/matlabcentral/newsreader/view_thread/294528. all works fine until i go to the line: is = jurl.openStream; This gave one error says Java exception occurred: java.lang.NullPointerException.

Anyone can help? thanks

Best Answer

Yes. so the answer is Matlab urlread does not support authentication with lower level request. So we need to code firectly against the Java URL class to use them:

% Matlab's urlread() doesn't do HTTP Request params, so work directly with Java
jUrl = java.net.URL(url);
conn = jUrl.openConnection();
conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]);
conn.connect();
info.status = conn.getResponseCode();
info.errMsg = char(readstream(conn.getErrorStream()));
s = readstream(conn.getInputStream());
function out = base64encode(str)
% Uses Sun-specific class, but we know that is the JVM Matlab ships with
encoder = sun.misc.BASE64Encoder();
out = char(encoder.encode(java.lang.String(str).getBytes()));
function out = readstream(inStream)
%READSTREAM Read all bytes from stream to uint8
try
    import com.mathworks.mlwidgets.io.InterruptibleStreamCopier;
    byteStream = java.io.ByteArrayOutputStream();
    isc = InterruptibleStreamCopier.getInterruptibleStreamCopier();
    isc.copyStream(inStream, byteStream);
    inStream.close();
    byteStream.close();
    out = typecast(byteStream.toByteArray', 'uint8'); %'
catch err
    out = []; %HACK: quash
end

Hope it will help others later.

Related Solutions

MATLAB: Webread certificate problems – roll your own java works, webread doesn’t. Why

See Jörgens comment above. https://ch.mathworks.com/matlabcentral/answers/376318-webread-certificate-problems-roll-your-own-java-works-webread-doesn-t-why#comment_528714

""" I solved it by setting the options=weboptions; options.CertificateFilename=(''); webread(URL,options); """

I also then needed to set the RequestMethod as POST, and crucially, use the 'webwrite' function, as opposed to the 'webread' function. Suddenly, things burst into life ...

MATLAB: Extract data from URL Human Mortality Database

MATLAB doesn't support well authentication. If you have 2013a, you can try this workaround.

If not, save the following function written I guess by Andrew Janke:

 function [s,info] = urlread_auth(url, user, password)
 %URLREAD_AUTH Like URLREAD, with basic authentication
 %


 % [s,info] = urlread_auth(url, user, password)
 %
 % Returns bytes. Convert to char if you're retrieving text.
 %
 % Examples:
 % sampleUrl = 'http://browserspy.dk/password-ok.php';
 % [s,info] = urlread_auth(sampleUrl, 'test', 'test');
 % txt = char(s)
 % Matlab's urlread() doesn't do HTTP Request params, so work directly with Java
 jUrl = java.net.URL(url);
 conn = jUrl.openConnection();
 conn.setRequestProperty('Authorization', ['Basic ' base64encode([user ':' password])]);
 conn.connect();
 info.status = conn.getResponseCode();
 info.errMsg = char(readstream(conn.getErrorStream()));
 s = readstream(conn.getInputStream());
 function out = base64encode(str)
 % Uses Sun-specific class, but we know that is the JVM Matlab ships with
 encoder = sun.misc.BASE64Encoder();
 out = char(encoder.encode(java.lang.String(str).getBytes()));
 function out = readstream(inStream)
 %READSTREAM Read all bytes from stream to uint8
 try
    import com.mathworks.mlwidgets.io.InterruptibleStreamCopier;
    byteStream = java.io.ByteArrayOutputStream();
    isc = InterruptibleStreamCopier.getInterruptibleStreamCopier();
    isc.copyStream(inStream, byteStream);
    inStream.close();
    byteStream.close();
    out = typecast(byteStream.toByteArray', 'uint8'); %'
 catch err
    out = []; %HACK: quash
 end

and then execute

 url    = 'http://www.mortality.org/hmd/PRT/STATS/Mx_1x1.txt' ;
 user   = 'yourLogin' ;
 pwd    = 'yourPassword' ;
 buffer  = char( urlread_auth( url, user, pwd )) ;

You can write buffer to file using e.g. FWRITE. You can also generate url dynamically using SPRINTF if you want to loop over files.

EDIT: you could actually extract the data on the fly, and save a .mat file instead of a text file. For example..

 % The two following replacements should be fine tuned to your needs.
 buffer = strrep(buffer, '+', ' ') ;         % Eliminate + ..
 buffer = strrep(buffer, ' .', '-1') ;       % Replace . entries with -1 ..
 % Txt -> cell array of num columns.
 data = textscan(buffer, '%f %f %f %f %f', Inf, 'HeaderLines', 3) ;

Best Answer

Related Solutions

MATLAB: Webread certificate problems – roll your own java works, webread doesn’t. Why

MATLAB: Extract data from URL Human Mortality Database

Related Question