Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 36 ยท Applications

Neuro-Fuzzy Systems and ANFIS from Scratch

Learn how neuro-fuzzy systems learn fuzzy rules from data: Takagi-Sugeno models, ANFIS layers, hybrid training with least squares and gradient descent, in a runnable numpy example.

Writing fuzzy rules by hand works when an expert can describe the behavior. Often there is no expert, only data: pairs of inputs and outputs measured from a machine. A neuro-fuzzy system combines the two worlds. It keeps the fuzzy structure (rules with membership functions, readable and interpretable) and uses neural-network-style learning to fit the parameters to data.

The best-known neuro-fuzzy model is ANFIS, the Adaptive-Network-based Fuzzy Inference System. In this lesson you will build a one-input ANFIS-style model from scratch, so that you understand what the toolboxes do under the hood.

The Takagi-Sugeno model

ANFIS uses first-order Takagi-Sugeno rules, whose conclusions are linear functions:

TEXT
Rule i:  IF x is A_i  THEN  y_i = p_i * x + r_i

Each A_i is a fuzzy set (here a Gaussian bell with center c_i and width s_i). The rule strength is the membership w_i(x). The model output is the weighted average of the rule outputs:

TEXT
y(x) = sum_i  w_i(x) * (p_i * x + r_i)  /  sum_i  w_i(x)

So every rule describes a local linear model valid near its center, and the memberships blend these local models smoothly. Think of fitting a curve with several straight-line pieces that fade in and out.

flowchart LR
  X["Input x"] --> L1["Layer 1<br/>memberships w_i"]
  L1 --> L2["Layer 2<br/>firing strengths"]
  L2 --> L3["Layer 3<br/>normalize"]
  L3 --> L4["Layer 4<br/>times (p x + r)"]
  L4 --> L5["Layer 5<br/>sum = output"]

The five layers of ANFIS. Layer 1 has the premise parameters (centers, widths); layer 4 has the consequent parameters (p, r).

Neuro-fuzzy in a nutshell

Fuzzy part Readable rules
Neural part Learn from data
Result Fitted and explainable

Interpretable models that learn

1 / 2

Hybrid learning

The output is linear in the consequent parameters p_i and r_i when the memberships are fixed. So for given centers and widths the best p and r are found directly with least squares, in one step, no iteration. The centers and widths (the premise parameters) affect the output nonlinearly, so they are improved by gradient descent on the error. ANFIS alternates the two: this is the hybrid algorithm. We use numerical gradients to keep the code short.

We fit a wavy target curve with three rules and see how much learning helps. The "before" number below is the least-squares fit with the starting centers; the gradient steps then improve it.

Python Training a one-input ANFIS-style model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both results come from the same code. More rules give more local models and a better fit (RMSE 0.207 with 3 rules, 0.074 with 6 before learning). Learning lowers each error further (0.192 and 0.056) by moving the centers and widths. Notice that every rule remains readable ("near x = 1.4, the output is about p x + r") unlike the weights of an ordinary neural network.

Predicting with the trained model

After training you want to use the rules, not only fit data. The next example trains a three-rule model (least squares for the consequents) and prints each rule so you can read what was learned.

Python Reading the learned rules

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The learned rules are slopes and intercepts you can interpret: near 2 the output rises steeply, near 8 it falls slowly. Compare this readability with a black-box regression model.

Practical notes

  • Number of rules. The count grows exponentially with inputs (k sets per input and n inputs gives k^n rules). For more than three or four inputs use clustering (such as fuzzy c-means) to place rules only where data exist.
  • Overfitting. More rules fit the training data more closely. Keep a validation set and stop when its error rises.
  • Initialization matters. Gradient descent finds a local minimum; place initial centers evenly over the input range.
  • Tools. MATLAB's Fuzzy Logic Toolbox has anfis; in Python, libraries such as scikit-fuzzy and several GitHub packages exist. This lesson's code is a teaching version of the same idea.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In a first-order Takagi-Sugeno rule, what is the conclusion?

    The consequent is y = p x + r (plus more inputs if there are any), a local linear model.
  2. 2. Why can the consequent parameters be found by least squares?

    With fixed memberships, the model output is a linear combination of p and r.
  3. 3. Which parameters does gradient descent adjust in the hybrid algorithm?

    Premise parameters (centers, widths) affect the output nonlinearly and are tuned by gradient steps.
  4. 4. What is a main reason neuro-fuzzy models are attractive?

    Interpretable rules plus data-driven fitting is the main selling point.

Summary

  • Neuro-fuzzy systems learn fuzzy rule parameters from data while keeping rules readable.
  • ANFIS uses first-order Takagi-Sugeno rules: Gaussian premises and linear consequents blended by normalized strengths.
  • Hybrid learning solves the linear consequents by least squares and tunes centers and widths by gradient descent.
  • Rule count explodes with many inputs; use clustering and validation data.

Next: Project: A Fuzzy Fan Speed Controller