If the error is "object too big" then your first priority is to switch to a 64 bit version.
If the error is "out of memory" then your first priority is to set the /3GB flag in your boot.ini If your /3GB boot.ini is already set, then you will need to switch to a 64 bit version.
If you run a 64 bit version of MATLAB on a system with 4 Gb, and use a lot of memory, then you are probably going to end up using swap space. This could make the simulation take a long time. But it would at least run. Because a simulation that finishes eventually is better than one that doesn't run at all.
You'd be better off going 64 Gb and getting more memory.
Best Answer