Hello, I'm trying to get my head around this neural network stuff. So a neuron's output is the sigmoid function applied to the total sum of all weights added together? So then where does the treshold fit in, and what is a bias? Am I missing anything? Thankyou. Have a good day.
The terminology is a bit confusing. In the common type of feed-forward back-propagation artificial neural network using a sigmoid transfer function the sigmoid function is applied to the sum of the weighted inputs i.e. to the sum of w_i * v_i where w_i is the weight associated with the input i and v_i is the current value of input i. Applying the sigmoid function to just the sum of the weights would mean that the neuron's output didn't change when the inputs changed, which would be useless.
The bias is an offset value applied to this sum - in effect, it is the weight of an input whose value v_bias is always 1. The bias shifts the centre of the sigmoid function away from the origin. The term originates in signal processing filter design, I think.
Threshold is used for basic perceptrons, neurons whose outputs take discrete values (e.g. where the output is either 0 or 1). If the value of the sigmoid function crosses the threshold, then the neuron's output switches from one value to another. I get the impression that switching perceptrons are rarely used in data interpretation these days except in a teaching context. Setting the neuron's output equal to the value of the sigmoid function, instead of using the sigmoid function value as a trigger to switch between different set output values, is more powerful. If your network's output values must be discrete (e.g. you are encoding a logic circuit as an exercise) then using a switching perceptron might be appropriate, at least in the final layer.
The terminology I'd recommend is feed-forward back-propagation artifical neural network. Feed-forward identifies how a network with a given set of connection weights and biases processes the input values to produce the output values. Back-propagation identifies how the network is trained using sets of known input/output data to adjust the values of the weights and biases. Again, the bias is simply a weight applied to an input whose value is always one. Once trained, an artificial neural network can be run in feed-forward mode only.
powered by Yahoo Answers