MATLAB: Do I get “Error using datastore (line 114) Failure occurred while trying to resolve path ‘s3://thedata08/Input.csv'” while trying to access file from Amazon AWS S3

MATLAB

I am using MATLAB to access a file from Amazon AWS S3. I followed this link from MATLAB documentation and I have set all the environment variables correctly.

https://www.mathworks.com/help/matlab/import_export/read-remote-data.html#bvn_hcu-2

I am using following code to access the file:

data=load('s3://mydata08/Input.csv') 
data=datastore('s3://mydata08/Input.csv')

Both of these give following error message:

ERROR: Error using datastore (line 114) Failure occurred while trying to resolve path 's3://mydata08/Input.csv'.

Please tell me why am I getting this error and how can I access the file?

Best Answer

One of the probable reason behind this issue is that the security settings of the file on Amazon AWS S3 server might be preventing MATLAB from accessing this file. Please try changing the security settings related to the file and provide read/write permission to all Amazon S3 authenticated users. After making these changes, you should be able to access the file from MATLAB.

Related Solutions

MATLAB: How to make a RESTful request through a Kerberos proxy

Direct Kerberos support was introduced in MATLAB R2019b. Hence, if you have access MATALB R2019b or newer, you can pass your certificates file as show below:

site = 'https://www.mySite.com/api/FILE.csv';
options = weboptions('CertificateFilename','C:\PATH\TO\YOUR\FILE.pem');
out = webread(site, options)

Refer to the Server Authentication documentation page below for more information:

https://www.mathworks.com/help/releases/R2019b/matlab/import_export/server-authentication.html

However, this will mostly likely not work in older versions of MATLAB. In those cases, the best alternative is to use system command 'curl' as follows (ask your system administrator for the exact details on how to set up the call to "curl"):

[status, result] = system('curl https://www.mySite.com/api/FILE.csv --proxy https://your-proxy-web --proxy-user ":" --proxy-negotiate --proxy-cacert PATH/TO/YOUR/FILE.PEM ')

MATLAB: Am I getting an error when trying to start the MATLAB Production Server instance after changing the port number in the “main_config” file

The issue in this case is not the result of changing the port number, rather the result of changing from HTTP to HTTPS. An MPS server instance must be specifically configured in order to work using HTTPS. Specifically, to use HTTPS you much generate a certificate chain and private key for your server instance, and then place these PEM files into the appropriate directories as detailed in the doc link here:

https://www.mathworks.com/help/mps/server/enable-security.html

If you are running multiple servers on the same machine, the same* certificate (self-signed or CA) can be used across multiple MPS instances *as the Fully Qualified Domain Name goes into the certificate (thus the identifier for that particular physical machine will be a part of the certificate).

If you are doing this on a larger scale with multiple machines and do not want to have to go through the trouble of setting up a certificate for each machine, or are looking to migrate existing HTTP servers to HTTPS then you could also choose to use a Load Balancer. This would mean that the only externally visible end-point would be that of the Load Balancer, and you could configure your MPS servers to just use HTTP. By then setting the Load Balancer's endpoint to HTTPS, you could still maintain the same secure layer between the client and the Load Balancer. This topology works best if all the instances are deploying the same CTF, but even if they do not, then a LB can still be configured and used mostly for authentication.

Best Answer

Related Solutions

MATLAB: How to make a RESTful request through a Kerberos proxy

MATLAB: Am I getting an error when trying to start the MATLAB Production Server instance after changing the port number in the “main_config” file

Related Question