3.2Neural Networks

Artificial neural networks are inspired by the biological neural network. They represent the powerful tool for solving or approximating problems, which are difficult to solve in a traditional analytical way.

A neuron is an abstract device capable to compute the value of a transfer function for any input in one computational step. Each neuron has several connections representing either outputs of other neurons or stimulus from the surroundings (neuron input values xi). Every connection has associated a real number wi that indicates the (synaptic) weight of the connection (i.e. its importance). Every neuron has also value called threshold t. For a scheme of a formal neuron see Fig. 3.2-1. The sum  tells total stimulus, called neuron potential. Neuron responds to this collected potential with the output response z=S(x), where S is non-linear transfer function, usually of sigmoid form (e.g. S(x)=1/(1+exp(-kx)) where k is constant representing the slope coefficient of the transfer function.) If the potential is greater then threshold, point x=[x1,x2,...] lies in the positive halfspace defined by separating hyperplane x=0. Otherwise it lies in the complement halfspace. This neuron facility enables classification of inputs into two classes.

Figure 3.2-1: The basic structure of a formal neuron.

A neural network is a finite set of neurons (input neurons, output neurons and other neurons), oriented interconnections among neurons (network topology), weights and thresholds. A multi-layer neural network is a neural network with a directed acyclic interconnection graph. Its set of neurons consists of a sequence pairwise disjunctive subsets called layers. In the interconnection graph, there are only edges from i-th to the (i+1)-th layer only. The first layer (input layer) is the set of all input neurons and the last layer (output layer) is the set of all output neurons of the network. For an example of such a network see Fig. 3.2-2. The networks distinguish two modes of work - adapting mode (network training) and active mode (network recall):

Figure 3.2-2: An example of a multi-layer neural network.

We decided to use neural networks for evaluating the genes when no analytical function can be used. Trained neural network can compute the fitness function value. Topology establishing and training is performed in separate system component before genetic algorithm is started. Once trained, the network configuration (topology, weights,...) can be saved in a file for further use. The object representing neural network is created at the start of the genetic algorithm computation. When running, the genetic algorithm calls just one neural network method for gene evaluation.

System OverviewProject Antares