MATLAB: Run Matlab file on each Node of Hadoop Cluster

1MATLAB Runtime Serverparallel computing

Hi everybody,

I want to buid a cluster which using Hadoop. But I don't know that I can run file Matlab with each Datanode ?

I will install Matlab on each Node or there is another way to solve problem?

Please help me. Thanks a lot.

Best Answer

This documentation page describes how to configure a Hadoop cluster so that client MATLAB sessions can submit to it. The first step in that workflow, "Cluster Configuration", links to a page that describes how to install MATLAB on the worker nodes.

I don't think this process has changed in recent releases, but if you're using a release older than the current release (the online documentation is for the current release, right now that is release R2019b) you probably want to find the equivalent of that page in the installation of one of the clients who are planning to submit to the Hadoop cluster. Alternately find the correct release's documentation from the documentation archive.

Related Solutions

MATLAB: How to run the example ‘Run mapreduce on a Hadoop Cluster’

Hi Jingyu,

Try editing the mapred-site.xml to set the value of this property: mapreduce.admin.user.home.dir

which will be set to a value of that of your hadoop directory: './home/rjy/soft/hadoop-1.2.1'

Maybe this will force it to look to your hadoop installation instead of /homes.

Referencing: http://www.mathworks.com/matlabcentral/answers/82060-mcr-problems-with-linux-hadoop

Hope this can solve the problem.

Cheers, Esther

MATLAB: How to csvwrite an array in HDFS

write command is able to write only tall array as .seq file. So, if you want to save csv files to HDFS, you should first save csv files on locla then upload to HDFS using Hadoop commands. From MATLAB,

csvwrite('hoge.csv', Array1);
!hadoop fs -copyFromLocal ./hoge.csv /user/savecsvfile

Best Answer

Related Solutions

MATLAB: How to run the example ‘Run mapreduce on a Hadoop Cluster’

MATLAB: How to csvwrite an array in HDFS

Related Question