All examples I found using deep belief or convolutional neural networks use them for image classification, chatacter detection or speech recognition.
Are deep neural networks also useful for classical regresion tasks, where the features are not structured (e.g., not arranged in a sequence or grid)? If yes, can you give an example?
Best Answer
The characteristics of images that makes them amenable to classification with a deep neural network is there are a ton of features (possibly millions if not billions of pixels with RGB, intensity, etc.) and if you have accurate labels, it's not noisy data. Cameras these days are very good and they aren't mis-measuring anything. Thanks to the Internet, we now have a lot of accurately labeled images. A deep network can express arbitrarily complicated functions, which is a problem with noisy data because you can very easily overfit the noise, hence why many learning methods tend to penalize complicated models. In the case of image recognition, however, the true function seems to actually be very complicated, we have no idea what the functional form looks like, and we don't even know what the relevant features are in many cases. A many-layer network can automatically discover and extract relevant features, which doesn't make it completely unique, but that is one attractive part of the model.
This doesn't mean you can't use deep networks to learn functions having nothing to do with images. You just need to be very careful about the downsides, mostly that it is very prone to overfitting, but also that it's computationally expensive and can take a long time to train (not as much an issue these days with parallelized SGD and GPUs). The other downside is you have very little to no model interpretability, which doesn't really matter for image classification. We're just trying to get computers to recognize the difference between a chimp and an orangutan. Human understanding of the formula doesn't matter. For other domains, especially medical diagnostics, policy research, etc., you want or might even need human understanding.