Latest Updates from Tech world

Wednesday 22 November 2017

What is artificial neural network and how it works?

What is a neural network?How it works and how it is can be the key element of Artificial Intelligence and Machine learning?

Artificial neural network is inspired by biological nervous system.
An Artificial Neural Network (ANN) is an information processing system that is inspired by the biological nervous systems. Artificial Neural network is an attempt to mimics animal brain. Neural network consist of large number of highly interconnected processing elements (neurones) working in unison to solve complex problems. Whether it is biological or artificial neural network neurons are the building blocks. The concept of Artificial Intelligence and neural networks established before the advent of computers, but many important advances have been boosted by the use of computer emulations.
Artificial intelligence and neural networks already applied in information processing systems, pattern recognition, medical diagnosis etc. Still there is a long way to go. This field offers infinite opportunity in coming decades.

Biological neurons V/S artificial neurons

Human brain is composed of billions of neurons and other specialised cells. Brain is the least explored and most complex organ of our body. We know only less than 1% of our brain. But we have a vague idea about learning mechanism of our brain.


Structure of biological neuron


A neuron receives signals from other neurons through a host of fine structures called dendrites. The neuron sends out spikes of electrical signals through a long, thin stand known as an axon, which splits into thousands of branches. At the end of each branch, a structure called a synapse converts the activity from the axon into electrical effects that inhibit or excite activity in the connected neurons. When a neuron receives excitatory input that is sufficiently large compared with its inhibitory input, it sends a spike of electrical activity down its axon. Learning occurs by changing the effectiveness of the synapses so that the influence of one neuron on another changes. In other words information is stored in these junctions between each neurons.

Biological to artificial neurons

Now let us see how an artificial neuron model is designed to mimic the basic functions of a neuron. An artificial neuron is a device with many inputs and one output. Dendrites will be replaced by input, processing unit will function as cell body and axon will be replaced by output line. We can program/ teach cellbody/processing unit to get desired output.

A simple neuron

The neuron has two modes of operation; the training mode and the using mode. In the training mode, the neuron can be trained to produce a particular output, for a set of inputs. In the using mode, when a taught input pattern is detected at the input, its associated output becomes the current output. If the input pattern does not belong in the taught list of input patterns, the firing(excitation) rule is used to determine the output state.

Artificial neural network example

For example, a 3-input neuron is taught to output 1 when the input (X1,X2 and X3) is 111 or 101 and to output 0 when the input is 000 or 001. Then, the truth table is;

X1:
0
0
0
0
1
1
1
1
X2:
0
0
1
1
0
0
1
1
X3:
0
1
0
1
0
1
0
1
OUT:
0
0
?
?
?
1
?
1

Output for the inputs 010,011,100 and 110 is not known to neuron. Here comes the firing rule Take the pattern 010. It differs from 000 in 1 element, from 001 in 2 elements, from 101 in 3 elements and from 111 in 2 elements. Therefore, the 'nearest' pattern is 000 which belongs in the 0-taught set. Thus the firing rule requires that the neuron should not fire when the input is 001. On the other hand, 011 is equally distant from two taught patterns that have different outputs and thus the output stays undefined (0/1).

By applying the firing in every column the following truth table is obtained;

X1:
0
0
0
0
1
1
1
1
X2:
0
0
1
1
0
0
1
1
X3:
0
1
0
1
0
1
0
1
OUT:
0
0
0
?
?
1
1
1

Therefore the firing rule enables neuron to respond 'sensibly' to patterns not seen during training.
More complicated neuron architecture

Above neuron doesn't do anything that conventional computers don't do already.In a neural network output of one neuron is given as input to another neuron. So, more sophisticated and meaningful neuron can be designed obtained by giving weight to each input. The effect that each input has at decision making is dependent on the weight of the particular input. The weight of an input is a number which when multiplied with the input gives the weighted input. These weighted inputs are then added together and if they exceed a pre-set threshold value, the neuron fires. In any other case the neuron does not fire.

Output=f(X1W1+X2W2+………..+XnWn)
This neuron has the ability to adapt to a particular situation(inputs) by changing its weights and/or threshold.

We have seen in biological neurons ‘Learning occurs by changing the effectiveness of the synapses so that the influence of one neuron on another changes’

In artificial neurons learning occurs by changing its weights on particular input, thus changing the effect of one neuron on another. Various algorithms exist that cause the neuron to 'adapt'.

Neural network



A neural network is an interconnection of large number of neurons. Output of one neuron will be fed to input of one or more neurons. Apart from the input and output layers , hidden layers are include so that output will be fine tuned and error free. As the number of neurons in a network increases, intelligence of that network will also improve(as in biological neural system).

Neuron:An example

In above figure neuron is trained to recognise the patterns X and Y. The associated patterns are all black and all white respectively as shown below.Let us assume input is captured by 3 X 3 sensor array
.
If we represent black squares with 1 and white squares with 0 then the truth tables for the neuron is.
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
0
1
0
1
0
1
0
1
X
The truth table for above pattern is as follows.
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
0
1
1
1
1
0
1
0
Y
Above two input sets are the taught input sets. Now, let us see what will be the output of the neuron if another set of input is introduced.
Truth table for the above pattern is given below. Initially output of the above pattern unknown to the neuron, but, on application of firing rule neuron will conclude above pattern is similar to taught input corresponds to ‘X’.
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
1
1
0
1
0
1
1
1
X
Neuron will look for the similarity of this input with the taught inputs. This truth table is similar to first truth table in 7 ways(7 inputs are similar) and to the second table in 4 ways only. So, neuron will give the output as ‘X’.
By applying firing rule neuron will conclude above pattern is ‘Y’ since it’s 5 input are similar to taught inputs of ‘Y’ and only 3 inputs are similar to taught inputs of ‘X’.
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
0
1
0
1
0
0
1
0
Y
Similarly observing other patterns given below you can see that , with only two taught input sets , system can identify up to five set of inputs. If a single neuron can do this much, what would be the capability of a network of tens, thousands or millions of neurons.? Today this algorithm can be implemented using simple computer programs.
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
0
1
0
1
0
1
1
1
X
A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
1
1
0
1
0
1
0
1
X

A1
A2
A3
A4
A5
A6
A7
A8
A9
o/p
1
0
1
1
1
1
0
0
1
Y
Artificial neural network application

  • Neural networks are best in pattern identification, identifying particular trends in a set of data, they are well suited for prediction or forecasting needs in sales, marketing, customer research, planning etc..
  • Medicine: Artificial neural nets are capable of disease diagnosis and treatment suggestions. Neural networks learn by example so the details of how to recognise the disease are not needed. What is needed is a set of examples that are representative of all the variations of the disease. Their problem solving ability and precision improves after each task(like humans).
  • Humanoid Robots: Artificial intelligence can be incorporated into robotic brains using neural networks.
  • Software :Pattern Recognition in facial recognition, optical character recognition, etc.
  • Self driving vehicles and autopiloting
Many applications of Neural Networks are yet to be explored. Their ability to learn by example makes them very flexible and powerful. With the advancement of high power computing we have a lot to gain from neural networks.Scientists believe that some day a 'consious' networks might be produced.For that neural network will be the biggest contributor along with other technologies.

1 comment:
Write comments
  1. "Learning about data modeling has significantly improved my comprehension of sophisticated informational systems. Even yet, I have concerns about its capacity to respond to quick technological changes.
    Data Analytics Courses in India

    ReplyDelete

Join Our Newsletter