From what I found out until now:
A layer in a neural network consists of a parameterizable number of neurons. A neural network consists of multiple layers. A neuron consists of a function f(x1, x2, …, xn), a sigmoid function which uses f as input and gives a binary output and a weight factor which is multiplied with with the sigmoid function and determines how much this neuron is considered for the output of the layer.
Is this correct and what is the function f/how does it combine the inputs to the output?
(I tried to read the literature and some course material myself but they all either ignore this part/call it a black box or assume you have a PhD in Theoretical Math but my university only focused on Applied Informatics. So if you have literature understandable for someone like me I would also appreciate it)
Best Answer
You are correct in your overall view of the subject. The neuron is nothing more than a set of inputs, a set of weights, and an activation function. The neuron translates these inputs into a single output, which can then be picked up as input for another layer of neurons later on.
While details can vary between neural networks, the function $f(x_1, x_2, \ldots , x_n)$ is often just a weighted sum: $$ f(x_1, x_2, \ldots , x_n) = w_1\cdot x_1 + w_2\cdot x_2 + ... + w_n\cdot x_n $$ Each neuron has a weight vector $w = (w_1, w_2, ..., w_n)$, where $n$ is the number of inputs to that neuron. These inputs can be either the 'raw' input features — say temperature, precipitation, and wind speed for a weather model — or the output of neurons from an earlier layer.
The weights for each neuron are turned during the training stage such that the final network output is biased toward some value (usually 1) for signal, and another (usually -1 or 0) for background.
Non-linear behavior in a neural network is accomplished by use of an activation function (often a sigmoid function) to which the output of $f$ is passed and modified. This allows neural networks to describe more complicated systems while still combining inputs in a simple fashion.
I found the Deep Learning in a Nutshell series by Tim Dettmers to be a very approachable introduction to the subject of machine learning in general.