As it is mentioned in the previous section, the transmission of a signal from one neuron to another through synapses is a complex chemical process in which specific transmitter substances are released from the sending side of the junction. The effect is to raise or lower the electrical potential inside the body of the receiving cell. If this potential reaches a threshold, the neuron fires.

It is this characteristic that the artificial neuron model proposed by McCulloch and Pitts, [McCulloch and Pitts 1943] attempt to reproduce. The neuron model shown in Figure 1.6 is the one that is widely used in artificial neural networks with some minor modifications on it.

Activation

The artificial neuron given in this figure has N inputs, denoted as u₁, u₂, ...u_N. Each line connecting these inputs to the neuron is assigned a weight, which are denoted as w₁, w₂, .., w_N respectively. Weights in the artificial model correspond to the synaptic connections in biological neurons. If the threshold in artificial neuron is to be represented by , then the activation is given by the formula:

(1.2.1)

The inputs and the weights are real values. A negative value for a weight indicates an inhibitory connection while a positive value indicates an excitatory one. Although in biological neurons,  has a negative value, it may be assigned a positive value in artificial neuron models. If  is positive, it is usually referred as bias.  For its mathematical convenience we will use (+) sign just before  in the activation formula. Sometimes, the threshold is combined for simplicity into the summation part by assuming an imaginary input u₀ having the value +1 with a connection weight w₀ having the value . Hence the activation formula becomes:

(1.2.2)

Vector Notation

Furthermore the vector notation:

(1.2.3)

Is useful for expressing the activation for a neuron. Here, the j^th element of the input vector u is u_j and the j^th element of the weight vector of w is w_j. Both of these vectors are of size N. Notice that,  is the inner product of the vectors w and u, resulting in a scalar value. The inner product is an operation defined on equal sized vectors. In the case these vectors have unit length, the inner product is a measure of similarity of these vectors.

Output Function

The output value of the neuron is a function of its activation, and it is analoguous to the firing frequency of the biological neurons:

(1.2.4)

 

Originally the neuron output function f(a) in McCulloch Pitts model proposed as threshold function, however linear, ramp and sigmoid  functions (Figure 1.6.) are also widely used output functions:

• Linear:		(1.2.5)
• Threshold:		(1.2.6)
• Ramp:		(1.2.7)
• Sigmoid:		(1.2.8)

Exercise 1.2.1

The applet on right-hand side is provided to demonstrate the behaviour of the sigmoid output function. By changing the value of the activation "a", observe how the output value changes. Also by changing the value of constant K, observe how the sigmoid function approaches to the sign; that is a threshold function taking valus of -1 and +1. You can observe the behaviour of the function in a larger range by changing a range.

Exercise 1.2.2

An example of a four input neuron is given in the applet on right-hand side. You can change the input values, connection weights and threshold using the associated scrollbars. Also the output function can be chosen using radio buttons. Using the applet find out how the output value is affected by changing the input values, connection weights, threshold and output function.

Example 1.2.1

Though its simple structure, McCulloch-Pitts neuron is a powerful computational device. McCulloch and Pitts proved that a synchronous assembly of such neurons is capable in principle to perform any computation that an ordinary digital computer can, though not necessarily so rapidly or conveniently.

When the threshold function is used as the neuron output function, and binary input values 0 and 1 are assumed, the basic boolean functions AND, OR and NOT of two variables can be implemented by choosing appropriate weights and threshold values, as shown in Figure 1.8. The first two neurons in the figure receives two binary inputs u₁, u₂ and produces y(u₁, u₂) for the Boolean functions AND and OR respectively. The last neuron implements the NOT function.

Exercise 1.2.3

The applet given on the right-hand side is provided to demonstrate how the logical AND OR and NOT gates can be implemented using single neurons.

Change the input values and see how the output neuron value changes. Compare the neuron output with the output values of the corresponding logical Boolean functions.