When designing filters you need normalized coefficients, and when they get very close or abut to 0 or 1, IIR goes unstable with the Signal Processing Toolbox. This effect is magnified for higher orders of the filter.
The IIR filters designed in Filter Design Toolbox are more stable.
Best Answer