An Artificial Neural Network (ANN) is an information
processing paradigm that is way biological nervous systems, such as the brain,
process inspired by the information. The key element of this paradigm is the
novel structure of the information processing system. It is composed of a large
number of highly interconnected processing elements (neurones) working in
unison to solve specific problems. ANNs, like people, learn by example. An ANN
is configured for a specific application, such as pattern recognition or data
classification, through a learning process. Learning in biological systems
involves adjustments to the synaptic connections that exist between the
neurones. This is true of ANNs as well.
Simplified view of an artificial neural network
Neural
network simulations appear to be a recent development. However, this field was
established before the advent of computers, and has survived at least one major
setback and several eras. The first artificial neuron was produced in 1943 by
the neurophysiologist Warren McCulloch and the logician Walter Pits.
Needs :
Neural
networks, with their remarkable ability to derive meaning from complicated or
imprecise data, can be used to extract patterns and detect trends that are too
complex to be noticed by either humans or other computer techniques. A trained
neural network can be thought of as an "expert" in the category of
information it has been given to analyse. This expert can then be used to
provide projections given new situations of interest and answer "what
if" questions.
Other advantages include:
Other advantages include:
- Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experience.
- Self-Organisation: An ANN can create its own organisation or representation of the information it receives during learning time.
- Real Time Operation: ANN computations may be carried out in parallel, and special hardware devices are being designed and manufactured which take advantage of this capability.
- Fault Tolerance via Redundant Information Coding: Partial destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be retained even with major network damage.
Neural networks versus conventional computers
Neural networks
take a different approach to problem solving than that of conventional
computers. Conventional computers use an algorithmic approach i.e. the computer
follows a set of instructions in order to solve a problem. Unless the specific
steps that the computer needs to follow are known the computer cannot solve the
problem. That restricts the problem solving capability of conventional
computers to problems that we already understand and know how to solve. But
computers would be so much more useful if they could do things that we don't
exactly know how to do. Neural networks process information in a similar way
the human brain does. The network is composed of a large number of highly
interconnected processing elements(neurones) working in parallel to solve a specific
problem. Neural networks learn by example. They cannot be programmed to perform
a specific task. The examples must be selected carefully otherwise useful time
is wasted or even worse the network might be functioning incorrectly. The
disadvantage is that because the network finds out how to solve the problem by
itself, its operation can be unpredictable.
On the other hand, conventional computers use a cognitive approach to
problem solving; the way the problem is to solved must be known and stated in
small unambiguous instructions. These instructions are then converted to a high
level language program and then into machine code that the computer can
understand. These machines are totally predictable; if anything goes wrong is
due to a software or hardware fault. Neural networks and conventional
algorithmic computers are not in competition but complement each other. There
are tasks are more suited to an algorithmic approach like arithmetic operations
and tasks that are more suited to neural networks. Even more, a large number of
tasks, require systems that use a combination of the two approaches (normally a
conventional computer is used to supervise the neural network) in order to
perform at maximum efficiency.
Forexample : If computer asked to make a search for the captain of
Indian Cricket team, it will make search by going through structure step by
step. It will read the profile of each cricket player and by matching the
condition, it will give the result whereas the Neural network like human brain
will directly give the result by jumping to that particular field of data. Result
will be instantaneously comes out.
Architecture of neural networks
1> Feed-forward networks
Feed-forward
ANNs (figure 1) allow signals to travel one way only; from input to output.
There is no feedback (loops) i.e. the output of any layer does not affect that
same layer. Feed-forward ANNs tend to be straight forward networks that
associate inputs with outputs. They are extensively used in pattern
recognition. This type of organisation is also referred to as bottom-up or
top-down.
2> Feedback networks
Feedback
networks (figure 1) can have signals travelling in both directions by
introducing loops in the network. Feedback networks are very powerful and can
get extremely complicated. Feedback networks are dynamic; their 'state' is
changing continuously until they reach an equilibrium point. They remain at the
equilibrium point until the input changes and a new equilibrium needs to be
found. Feedback architectures are also referred to as interactive or recurrent,
although the latter term is often used to denote feedback connections in
single-layer organizations.
An example of a simple feedforward network
An
example of a complicated network
3> Network layers
The
commonest type of artificial neural network consists of three groups, or
layers, of units: a layer of "input" units is
connected to a layer of "hidden" units, which is
connected to a layer of "output" units. (see Figure
4.1)
1> The
activity of the input units represents the raw information that is fed into the
network.
2> The activity
of each hidden unit is determined by the activities of the input units and the
weights on the connections between the input and the hidden units.
3> The
behaviour of the output units depends on the activity of the hidden units and
the weights between the hidden and output units.
This simple
type of network is interesting because the hidden units are free to construct
their own representations of the input. The weights between the input and
hidden units determine when each hidden unit is active, and so by modifying
these weights, a hidden unit can choose what it represents.
We also
distinguish single-layer and multi-layer architectures. The single-layer organization, in which all units are connected to one another, constitutes the
most general case and is of more potential computational power than
hierarchically structured multi-layer organizations. In multi-layer networks,
units are often numbered by layer, instead of following a global numbering.
4> Perceptrons
The most
influential work on neural nets in the 60's went under the heading of
'perceptrons' a term coined by Frank Rosenblatt. The perceptron (figure 4.4)
turns out to be an MCP model ( neuron with weighted inputs ) with some
additional, fixed, pre--processing. Units labelled A1, A2, Aj , Ap are called
association units and their task is to extract specific, localised featured
from the input images. Perceptrons mimic the basic idea behind the mammalian
visual system. They were mainly used in pattern recognition even though their
capabilities extended a lot more.
Characterizations
:
The term 'Neural Network' has two distinct usages:
- Biological neural networks are made up of real biological neurons that are connected or functionally-related in the peripheral nervous system or the central nervous system. In the field of neuroscience, they are often identified as groups of neurons that perform a specific physiological function in laboratory analysis.
- Artificial neural networks are made up of interconnecting artificial neurons (programming constructs that mimic the properties of biological neurons). Artificial neural networks may either be used to gain an understanding of biological neural networks, or for solving artificial intelligence problems without necessarily creating a model of a real biological system.
We
are mainly emphasizing over the Artificial Neural network. An artificial
neural network (ANN), also called a simulated neural network (SNN)
or commonly just neural network (NN) is an interconnected group of artificial neurons that uses a mathematical or
computational model for information processing
based on a connectionist approach
to computation. In most cases an ANN is an adaptive system that changes its structure based
on external or internal information that flows through the network.
A Technical aspect of ANN
A simple neuron
An artificial
neuron is a device with many inputs and one output. The neuron has two modes of
operation; the training mode and the using mode. In the training mode, the
neuron can be trained to fire (or not), for particular input patterns. In the
using mode, when a taught input pattern is detected at the input, its
associated output becomes the current output. If the input pattern does not
belong in the taught list of input patterns, the firing rule is used to
determine whether to fire or not.
A simple
neuron
Firing rules
The firing
rule is an important concept in neural networks and accounts for their high
flexibility. A firing rule determines how one calculates whether a neuron
should fire for any input pattern. It relates to all the input patterns, not
only the ones on which the node was trained. A simple firing rule can be
implemented by using Hamming distance technique. The rule goes as follows:
Take a
collection of training patterns for a node, some of which cause it to fire (the
1-taught set of patterns) and others which prevent it from doing so (the
0-taught set). Then the patterns not in the collection cause the node to fire
if, on comparison , they have more input elements in common with the 'nearest'
pattern in the 1-taught set than with the 'nearest' pattern in the 0-taught
set. If there is a tie, then the pattern remains in the undefined state.
For example,
a 3-input neuron is taught to output 1 when the input (X1,X2 and X3) is 111 or
101 and to output 0 when the input is 000 or 001.
Then, before
applying the firing rule, the truth table is;
X1:
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
|
X2:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
|
X3:
|
0
|
1
|
0
|
1
|
0
|
1
|
0
|
1
|
|
OUT:
|
0
|
0
|
0/1
|
0/1
|
0/1
|
1
|
0/1
|
1
|
As an
example of the way the firing rule is applied, take the pattern 010. It differs
from 000 in 1 element, from 001 in 2 elements, from 101 in 3 elements and from
111 in 2 elements. Therefore, the 'nearest' pattern is 000 which belongs in the
0-taught set. Thus the firing rule requires that the neuron should not fire
when the input is 001. On the other hand, 011 is equally distant from two
taught patterns that have different outputs and thus the output stays undefined
(0/1).
By applying
the firing in every column the following truth table is obtained;
X1:
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
|
X2:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
|
X3:
|
0
|
1
|
0
|
1
|
0
|
1
|
0
|
1
|
|
OUT:
|
0
|
0
|
0
|
0/1
|
0/1
|
1
|
1
|
1
|
The
difference between the two truth tables is called the generalisation of the
neuron. Therefore the firing rule gives the neuron a sense of similarity
and enables it to respond 'sensibly' to patterns not seen during training.
An important
application of neural networks is pattern recognition. Pattern recognition can
be implemented by using a feed-forward (figure 1) neural network that has been
trained accordingly. During training, the network is trained to associate
outputs with input patterns. When the network is used, it identifies the input
pattern
and tries to
output the associated output pattern. The power of neural networks comes to
life when a pattern that has no output associated with it, is given as an
input. In this case, the network gives the output that corresponds to a taught
input pattern that is least different from the given pattern.
Forexample:
The network of figure 1 is trained to recognise the patterns T and H. The associated patterns are all black and all white respectively as shown below.
The network of figure 1 is trained to recognise the patterns T and H. The associated patterns are all black and all white respectively as shown below.
If we
represent black squares with 0 and white squares with 1 then the truth tables
for the 3 neurones after generalisation are;
X11:
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
|
X12:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
|
X13:
|
0
|
1
|
0
|
1
|
0
|
1
|
0
|
1
|
|
OUT:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
Top
neuron
X21:
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
|
X22:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
|
X23:
|
0
|
1
|
0
|
1
|
0
|
1
|
0
|
1
|
|
OUT:
|
1
|
0/1
|
1
|
0/1
|
0/1
|
0
|
0/1
|
0
|
Middle
neuron
X21:
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
|
X22:
|
0
|
0
|
1
|
1
|
0
|
0
|
1
|
1
|
|
X23:
|
0
|
1
|
0
|
1
|
0
|
1
|
0
|
1
|
|
OUT:
|
1
|
0
|
1
|
1
|
0
|
0
|
1
|
0
|
Bottom
neuron
From
the tables it can be seen the following associasions can be extracted:
In this
case, it is obvious that the output should be all blacks since the input
pattern is almost the same as the 'T' pattern.
Here also,
it is obvious that the output should be all whites since the input pattern is
almost the same as the 'H' pattern.
Here, the
top row is 2 errors away from the a T and 3 from an H. So the top output is
black. The middle row is 1 error away from both T and H so the output is
random. The bottom row is 1 error away from T and 2 away from H. Therefore the
output is black. The total output of the network is still in favour of the T
shape.
Applications of neural networks
Neural Networks in Practice
Given this
description of neural networks and how they work, what real world applications
are they suited for? Neural networks have broad applicability to real world
business problems. In fact, they have already been successfully applied in many
industries.
Since neural
networks are best at identifying patterns or trends in data, they are well
suited for prediction or forecasting needs including:
1> sales
forecasting
2> industrial
process control
3> customer
research
4> data
validation
5> risk
management
6> target
marketing
7> Imaging , Vision
8> Medical and Biological evolution , etc
But to give
you some more specific examples; ANN are also used in the following specific
paradigms: recognition of speakers in communications; diagnosis of hepatitis;
recovery of telecommunications from faulty software; interpretation of multimeaning
Chinese words; undersea mine detection; texture analysis; three-dimensional
object recognition; hand-written word recognition; and facial recognition.
Case Application:
Neural networks in medicine
Artificial
Neural Networks (ANN) are currently a 'hot' research area in medicine and it is
believed that they will receive extensive application to biomedical systems in
the next few years. At the moment, the research is mostly on modelling parts of
the human body and recognising diseases from various scans (e.g. cardiograms,
CAT scans, ultrasonic scans, etc.). Neural networks are ideal in recognising
diseases using scans since there is no need to provide a specific algorithm on
how to identify the disease. Neural networks learn by example so the details of
how to recognise the disease are not needed. What is needed is a set of
examples that are representative of all the variations of the disease. The
quantity of examples is not as important as the 'quantity'. The examples need
to be selected very carefully if the system is to perform reliably and
efficiently.
Conclusion
The
computing world has a lot to gain from neural networks. Their ability to learn
by example makes them very flexible and powerful. Furthermore there is no need
to devise an algorithm in order to perform a specific task; i.e. there is no
need to understand the internal mechanisms of that task. They are also very
well suited for real time systems because of their fast response and
computational times which are due to their parallel architecture. Neural
networks also contribute to other areas of research such as neurology and
psychology. They are regularly used to model parts of living organisms and to
investigate the internal mechanisms of the brain.
Finally, I
would like to state that even though neural networks have a huge potential we
will only get the best of them when they are integrated with computing, AI, Fuzzy logic anciently, Machine Learning and Deep Learning.
REFERENCES:
BOOK AUTHOR
EDITION
1> The Handbook
of Brain Theory and Neural Networks. Arbib,
Michael A. (Ed.) (1995).
2>
Nonlinear Programming. Bertsekas,
Dimitri P. (1999).
5>
An
introduction to neural computing. Aleksander, I.
and Morton, H. 2nd edition
6>
Wikipedia.com
7>
Sciencedirect.com
