From many definitions that I read, I concluded that a DNN (deep neural network) is an ANN (artificial neural network) that have more than one hidden layer.
Knowing that CNN (convolutional neural network, a kind of a DNN) includes a stage of feature extraction (through convolution operations then pooling), my question is:
Does any DNN naturally considers feature extraction? For example, if we assume a simple architecture that includes three fully-connected hidden layers. I think not. If it is really not, how a step of feature extraction can be introduced within such a DNN? Should I necessarily introduce convolution like in CNN?
Best Answer
If your problem is "nice enough" that CNNs can do a good job on it, congratulations! Most of the work has been abstracted away! But if you're unlucky enough to be working on any of the myriad of problems that don't naturally lend themselves to CNNs or similar "feature-free" neural networks, then you'll have to spend a lot of time figuring out how to collect and represent your data in a useful way.
One example is compute security. A portable executable file is billions or more bytes and usually contains heterogenous data types. Feature engineering, the painstaking process of measuring various attributes of the file, is critically important to representing this data in a format that is useful and yields strong results. Feature-free attempts at analyzing PE files have not yet achieved parity with handcrafted feature vectors. (This might happen in the future, but it hasn't happened yet.)
The discussion at Why do neural networks need feature selection / engineering? is also helpful here.