MATLAB: Parpool local too slow

Question

Why does it take so long to start a parallel pool on a local profile? It's only 10 seconds or so, but for a job that should simply thread across the cores of the host machine's CPU it should be nearly instantaneous.

I spent some time reworking a bit of code we use frequently to run in parallel. The runtime went from 10 seconds to around 2 which I was very happy with. Our use case, though, only requires running a few jobs at a time. Over time it would add up to a pretty nice time savings except that every time the user runs the code they need to wait for parpool to run. The 15 seconds there means that it's only worth using the parallel code I worked so hard on if they're going to be running more than a handful of jobs at a time which is rare. The result is that my time producing parallel code was wasted because Matlab inexplicably needs time to set up a pool of workers on the host machine that Matlab itself is running on. Is there any way around this because it makes no sense?

Answer 1

Unfortunately there is no way around this, other than to open a parpool when you first need one (or let it open automatically if you're using >= R2013b), and don't close it until you've done everything you need to do. In R2013b, there's also an automatic timeout so that the pool will shut down when you're not using it.

The reason for the amount of time taken is that the pool launches separate MATLAB worker processes (which are essentially the same as a normal MATLAB process, except they are missing the display), rather than simply creating additional threads.