Comparison of Neural Networks and GAs

For Both:

1. Climbing/Descending an unknown multi-dimensional landscape.
2. Never being shown the actual shape of the landscape, only deducing its shape from a finite number of sample points on its surface.
3. No method that works like that can ever guarantee to find the global maximum/minimum. It may require an infinite amount of exploration to hit (by luck) the global optimum, if there is no landscape leading up/down towards it.
4. To avoid local optima, we do not do strict descent/ascent. We can make moves in the opposite direction. This probability of this "noise" is high at the start and declines as we go on.

For Neural Nets only:

1. It is given well-chosen, representative exemplars.
2. E is known (distance from correct answer).

3. is known (how error changes as we change the parameters).

4. We can make a directed move.
5. We start off by climbing/descending multiple landscapes at once. Eventually, the weight specialises on climbing/descending a particular family of similar landscapes.

For GAs only:

1. It has to make up its own exemplars.
2. E is not known. We do not know how good the "correct" answer might be. We get a fitness score for our attempt, certainly, but we do not know how good an attempt could possibly be.

3. is not known.

4. Or, where f is fitness, ∂f / ∂w is not known.

5. We can only make a random move.
6. We use a population of multiple climbers/descenders on the same landscape. We follow the best performers. So this may mean we are following a number of trails at once.