Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 36 ยท Projects and Practice

Project: Evolve a Neural Network

Train a small neural network without backpropagation. Evolve its weights with a genetic algorithm on XOR and a two-class dataset, using only NumPy.

Neural networks are normally trained with gradient descent, which needs a differentiable loss. Neuroevolution trains them with an evolutionary algorithm instead: the network's weights are the chromosome and the fitness is how well the network performs. It needs no gradients, so it works when the "loss" is a game score, a robot's distance walked, or any other number you can measure but cannot differentiate. Companies have used it for game agents and for searching network architectures.

In this project you build a two-layer network in NumPy and evolve its weights. You will see both the power and the limits of the idea.

Step 1: a network as a flat vector

A network with 2 inputs, 4 hidden units (tanh) and 1 output (sigmoid) has these parameters: a 2 x 4 matrix W1, 4 biases b1, a 4 x 1 matrix W2 and 1 bias b2. That makes 8 + 4 + 4 + 1 = 17 numbers. We store them in a flat vector of 17 real numbers, our chromosome, and unpack it into matrices when we need to run the network. The flat vector is what crossover and mutation work on.

flowchart LR
    G["Chromosome: 17 real numbers"] --> U["Unpack into W1, b1, W2, b2"]
    U --> N["Forward pass: tanh hidden layer, sigmoid output"]
    N --> L["Loss on the data"]
    L --> F["Fitness = -loss"]

Fitness needs only forward passes: no backpropagation.

Python Evolve a network that solves XOR

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

XOR is solved when the printed predictions are close to 0, 1, 1, 0. If you change the seed, you may see a run that gets stuck at 75 percent accuracy: evolution, like gradient descent, can fall into poor local optima. More population or a restart fixes it.

Step 2: a harder dataset

Let us classify two interleaved clusters of points in 2D (a "two moons" shape, built by hand so we do not need extra libraries). A single straight line cannot separate them, so the network needs its hidden layer.

Python Evolve a classifier for two moons

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The evolved network should reach a clearly higher test accuracy than any single straight line, showing the hidden layer learned a curved boundary.

What you learned about neuroevolution

  • Works with no gradients: useful when the objective is a simulation or a game.
  • Scales badly with size: with 17 parameters evolution is fast. With millions of weights, gradient descent wins by orders of magnitude, because it uses exact direction information while evolution samples blindly. (Large-scale evolution strategies with huge parallel populations can work for reinforcement learning, though.)
  • Mutation size matters: a step that is too big wrecks good networks, one that is too small stalls. We shrank sigma slowly, a simple schedule; self-adaptive step sizes are the standard improvement.
  • Crossover is tricky: two networks can compute the same function with hidden units in a different order (the permutation problem), so mixing their weights can break both. This is why famous systems such as NEAT use speciation and historical markings.

Extension challenges

  1. Replace uniform crossover with no crossover and mutation only. Does the result change?
  2. Evolve the number of hidden units too, by adding a penalty on size.
  3. Use the same code to evolve a small controller: the network reads a state (position, speed) and outputs a force, and fitness is how long a simulated pole stays upright.
  4. Compare with an evolution strategy that perturbs one parent with noise and moves toward the better children.

Common mistakes

  • Evaluating on the test set during evolution. Keep a test set untouched.
  • Starting weights too large. tanh saturates and the fitness is flat; start around 0 with standard deviation 1 or less.
  • No elitism. Mutation can destroy the best network; keep it.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the chromosome in neuroevolution as done in this project?

    Evolution operates on the parameter vector; the network is rebuilt from it for each evaluation.
  2. 2. Why is a hidden layer needed for XOR?

    A model with no hidden layer can only draw one straight decision boundary.
  3. 3. Why does neuroevolution struggle with millions of weights compared with backpropagation?

    Gradient information gives direction in all dimensions at once.
  4. 4. What is the permutation problem in crossover of neural networks?

    Hidden-unit order carries no meaning, so naive weight mixing may combine unrelated units.

Summary

  • Neuroevolution treats network weights as a chromosome and uses forward-pass performance as fitness.
  • No gradients are needed, so it suits games, control and non-differentiable objectives.
  • Selection, uniform crossover and Gaussian mutation with a shrinking step size solved XOR and a moons dataset.
  • It is practical for small networks; for large ones gradient methods dominate.
  • Keep elitism, a separate test set and sensible initial weights.

Next: Benchmarking GA, DE and PSO