## 3.2Neural Networks

Artificial neural networks are inspired by the biological neural network. They represent the powerful tool for solving or approximating problems, which are difficult to solve in a traditional analytical way.

A neuron is an abstract device capable to compute the value of a transfer function for any input in one computational step. Each neuron has several connections representing either outputs of other neurons or stimulus from the surroundings (neuron input values xi). Every connection has associated a real number wi that indicates the (synaptic) weight of the connection (i.e. its importance). Every neuron has also value called threshold t. For a scheme of a formal neuron see Fig. 3.2-1. The sum  tells total stimulus, called neuron potential. Neuron responds to this collected potential with the output response z=S(x), where S is non-linear transfer function, usually of sigmoid form (e.g. S(x)=1/(1+exp(-kx)) where k is constant representing the slope coefficient of the transfer function.) If the potential is greater then threshold, point x=[x1,x2,...] lies in the positive halfspace defined by separating hyperplane x=0. Otherwise it lies in the complement halfspace. This neuron facility enables classification of inputs into two classes.

Figure 3.2-1: The basic structure of a formal neuron.

A neural network is a finite set of neurons (input neurons, output neurons and other neurons), oriented interconnections among neurons (network topology), weights and thresholds. A multi-layer neural network is a neural network with a directed acyclic interconnection graph. Its set of neurons consists of a sequence pairwise disjunctive subsets called layers. In the interconnection graph, there are only edges from i-th to the (i+1)-th layer only. The first layer (input layer) is the set of all input neurons and the last layer (output layer) is the set of all output neurons of the network. For an example of such a network see Fig. 3.2-2. The networks distinguish two modes of work - adapting mode (network training) and active mode (network recall):

• Training: At first the user must create training a set. Training set is set of input values and corresponding expected outputs. Then training algorithm can start. The aim of the training algorithm is to find a set of weights that ensures that for each input vector contained in the training set the actual output vector produced by the network equals the desired output vector.
• Recall: Well-trained neural network is able to approximate the problem it was trained for. It gives correct answers for all known inputs and is able to approximate them in the unknown ones.

Figure 3.2-2: An example of a multi-layer neural network.

We decided to use neural networks for evaluating the genes when no analytical function can be used. Trained neural network can compute the fitness function value. Topology establishing and training is performed in separate system component before genetic algorithm is started. Once trained, the network configuration (topology, weights,...) can be saved in a file for further use. The object representing neural network is created at the start of the genetic algorithm computation. When running, the genetic algorithm calls just one neural network method for gene evaluation.