Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 36 ยท Projects and Practice

Practice Exercises and Where to Go Next

Test your skills with hands-on genetic algorithm exercises (starter code included), a design checklist for new problems, and a roadmap of libraries and topics to explore next.

You have covered constraints, multi-objective search, swarms, differential evolution, genetic programming, tuning, speed-ups and applications. Knowledge sticks when you use it. This lesson gives you ten exercises in increasing difficulty, a design checklist for any new optimization problem, and a roadmap of what to learn next. Each exercise has runnable starter code in the first few cases. Solve them before looking elsewhere.

A starter kit

This small GA for real vectors is the base for exercises 1 to 3. It minimizes a function over a box.

Python Starter GA: minimize a function over a box

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercises

Easy

  1. Find a hidden number. Define f(x) = (x - 3.1415)^2 on [0, 10] and use the starter GA to find x. Then change sigma to 2.0 and to 0.001. Which settings find 4 decimals?
  2. Constrained minimum. Minimize x^2 + y^2 subject to x + y >= 1, using a penalty. The true answer is (0.5, 0.5). How large must the penalty coefficient be?
  3. Curve fit. Generate data from y = 2 * sin(3x) + 0.5 with a little noise. Use the GA to recover the four parameters of a * sin(b * x + d) + c. The fitness is the mean squared error. Why can different (a, b, d) values give the same curve?
Python Exercise 2 scaffold: add the penalty yourself

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Medium

  1. Knapsack with two limits. Items have a weight and a volume; the bag has a weight limit and a volume limit. Use a repair operator (Repair and Decoders).
  2. Job scheduling. Schedule 30 jobs on 4 machines with random-key chromosomes to minimize the makespan, then add job due dates and minimize total lateness.
  3. Two objectives. Minimize f1 = x^2 and f2 = (x - 2)^2 for x in [-5, 5] with NSGA-II. The Pareto set is x in [0, 2]. Check that your front covers it evenly.
  4. Tune a GA. Pick the trap function of Tuning GA Parameters with 20 blocks. Find settings that solve it more than 80 percent of the time within 20,000 evaluations.

Hard

  1. Evolve a formula. Use symbolic regression to rediscover y = x^2 / (1 + x) on [0.1, 3]. Add sin and exp to the function set and compare complexity of the result.
  2. Vehicle routing. Extend the TSP GA to 3 vehicles with a capacity of 15 units each and customer demands between 1 and 5. Split a permutation into routes with a greedy decoder.
  3. Your own problem. Pick something you actually care about (a timetable, a playlist order, a layout, a hyperparameter search), write the objective, choose an encoding, add an exact or random baseline and use the benchmark harness to compare at least two algorithms.

When stuck, write the fitness function first and test it by hand on three or four chromosomes whose quality you already know. Most GA bugs are fitness-function bugs.

A design checklist for any new problem

  1. What exactly is a solution? Write one by hand.
  2. What is the objective? One number, or several (then use Pareto methods)?
  3. What are the constraints? Hard or soft? Choose penalty, repair or decoder.
  4. Choose the encoding: bits, integers, reals, permutations, trees. Ask: do small changes to the chromosome cause small changes in quality? If not, redesign.
  5. Choose operators that fit the encoding (OX for permutations, SBX/blend for reals, subtree for trees).
  6. Build baselines: random search, greedy, hill climbing or a known exact method.
  7. Start with defaults: population 100, tournament 3, crossover 0.8, mutation about 1/length, elitism 1.
  8. Run many seeds, plot convergence and diversity.
  9. Tune one parameter at a time, at equal evaluation budgets.
  10. Speed up only after profiling: cache, vectorize, parallelize.
  11. Validate on unseen data/instances and report honest numbers.
flowchart LR
    A["Define solution + objective"] --> B["Constraints: penalty / repair / decoder"]
    B --> C["Encoding + operators"]
    C --> D["Baselines"]
    D --> E["Run, many seeds"]
    E --> F{"Better than baselines?"}
    F -- no --> G["Revisit encoding and fitness"]
    G --> C
    F -- yes --> H["Tune, speed up, validate"]

Most of the effort goes into the loop between encoding and baselines, not into parameter tweaking.

Where to go next

Libraries

  • DEAP (Python): a flexible framework for GAs, genetic programming, evolution strategies and multi-objective algorithms.
  • PyGAD: easy GA with built-in support for neural-network training.
  • pymoo: state-of-the-art multi-objective algorithms (NSGA-II, NSGA-III, MOEA/D) and many test problems.
  • scipy.optimize: differential_evolution, dual_annealing and shgo.
  • Optuna, Ray Tune, Nevergrad: hyperparameter and black-box optimization.
  • gplearn, PySR: symbolic regression.
  • Inspyred, Platypus, jMetal (Java), ECJ (Java): further evolutionary frameworks.

Topics

  • Evolution strategies and CMA-ES: the leading method for hard continuous problems with up to a few hundred dimensions.
  • Estimation of distribution algorithms: replace crossover and mutation with a learned probability model.
  • Quality-diversity (MAP-Elites, novelty search): search for a collection of diverse high-performing solutions, widely used in robotics.
  • Co-evolution: populations that evolve against each other (predators and prey, solutions and test cases).
  • Neuroevolution (NEAT, HyperNEAT): evolve network structure and weights together.
  • Bayesian optimization: a different approach for very expensive functions.
  • Theory: schema theorem, runtime analysis of evolutionary algorithms, fitness landscapes.

Reading habits

Read papers of the GECCO and IEEE CEC conferences, and look for benchmark suites such as COCO/BBOB, which are designed to compare optimizers rigorously.

Final self-check

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. You designed a chromosome in which a tiny change often turns a great solution into a terrible one. What should you do?

    Smooth, meaningful neighborhoods are the heart of a good encoding.
  2. 2. What should you build before trusting a new GA result?

    If a GA cannot beat simple baselines, it is not helping.
  3. 3. Which library is the best fit for multi-objective algorithms such as NSGA-III?

    pymoo specializes in multi-objective optimization.
  4. 4. For hard, continuous, black-box problems in up to a few hundred dimensions, which method is often the strongest?

    CMA-ES adapts a full covariance matrix of its sampling distribution.

Summary

  • Practice with the exercises, starting from the starter GA and the earlier lessons.
  • Follow the design checklist: solution, objective, constraints, encoding, baselines, many seeds, tuning, speed, validation.
  • Explore DEAP, pymoo, PyGAD, SciPy and Optuna, and topics such as CMA-ES, quality-diversity and neuroevolution.
  • Honest evaluation (baselines, seeds, held-out data) matters more than any single trick.

Back to the course overview: Genetic and Evolutionary Algorithms