MATLAB: How to check Hadoop and Matlab Integrated Properly or Not ??

hadoopMATLABmatlab guiMATLAB Parallel ServerParallel Computing Toolbox

Hi
We integrated Matlab R2016b with Hadoop-2.7.2 …but we are not sure it is working properly ….how we can check the program is running on cluster and each node is contributing in processing …
1. It is taking more time for map-reduce (on cluster with 50 Nodes ) compare to matlab map-reduce with single computer..
2. Where to set Matlab Distributed Computer Server Properties …like how many nodes are there, parallel pool etc .
3.How to see Matlab+Hadoop Cluster Configuration in Matlab Interface ??
Please provide me all detail in answer Thanks

Best Answer

To look at whether MATLAB is running on the Hadoop cluster correctly, your best bet is to look at the Hadoop/Yarn Web UI. By default, this is:
http://hadoophostname:8088/
Where hadoophostname should be replaced by the hostname of the head node of Hadoop. During a mapreduce operation in MATLAB, you should see a running job in the web UI.
If you don't see a job running, it might be possible that the Hadoop installation you provided to MATLAB is not configured to run jobs in cluster mode. This can happen if the Hadoop property mapreduce.jobtracker.address found in ${HADOOP_INSTALL}/etc/hadoop/mapred-site.xml has not been set or has been set to "local". This property should be set to the hostname of the headnode of the cluster.
In a Hadoop cluster, the number of workers that are launched are controlled by Hadoop. By default, it will run as many workers as it can fit in the memory given to it.
Related Question