MATLAB: Do I get “Maximum variable size allowed on the device is exceeded.” error when running the “semanticseg” function on the image and custom network

Computer Vision Toolboxconvolveforward2dsemanticseg

I am trying to run the "semanticseg" function on an image with size (8557×11377) and my own custom network, and I receive the error:
Error using nnet.internal.cnngpu.convolveForward2D
Maximum variable size allowed on the device is exceeded.
I expect MATLAB to tile the image, run the semantic segmentation, and reassemble the tile, but I am receiving this error on a windows laptop with a GTX 1070 card.
The two questions I have are:
1) Why do I receive this error message?
2) I would like to know what the maximum allowed image size I can have with my own custom network when running the "semanticseg" function.

Best Answer

Currently, the maximum value for the total number of array elements on the GPU is limited by CUDA's CUBLAS/CUFFT APIs to " intmax('int32'). The error is likely because the number of elements of the array input to "gpuArray" exceeds this limit. Internally, "semanticseg" creates a gpuArray, and due to the size of the image and the customers network, the limit of  intmax('int32') is being exceeded.
Since this is a very large image to push into a GPU along with the additional memory required for the network and all of its intermediate data, at the moment, there is no tool to support these large image sizes with deep learning.
One workaround in MATLAB R2017b would be to tile the image separately and pass the tiles into the "semanticseg" function. A function like "blockproc" can help with the tiling.
A version of a tile-based semantic segmentation helper function is also attached, which can be adapted for your specific application.