Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 40 ยท Getting Started

Data Basics for Keras

Prepare data for Keras models: features and labels, train/validation/test splits, scaling, one-hot labels, shuffling and batching, with NumPy examples.

A model is only as good as its data, and most "my network does not learn" problems are data problems: unscaled inputs, labels in the wrong format, or a test set that leaked into training. This lesson covers the preparation steps almost every Keras project needs. The NumPy code you can run here does exactly what keras.utils and tf.data do later, and the real Keras calls are shown next to it.

Features and labels

Training data comes in two parts:

  • Features x: what the model sees. A table with shape (n_examples, n_features); images (n, height, width, channels); text as sequences of integer ids.
  • Labels y: what you want it to predict. A class index for classification, a number for regression.

The first axis is always the example (sample) axis. Keras expects NumPy arrays or tensors, not Python lists, and floating-point features (float32).

Python
import numpy as np

x = np.array([[5.1, 3.5], [4.9, 3.0], [6.7, 3.1]], dtype="float32")
y = np.array([0, 0, 1])
print(x.shape, y.shape)   # (3, 2) (3,)

Splitting: train, validation and test

Never judge a model on the data it trained on. Use three sets:

Set Used for Typical share
Training Updating the weights 60-80%
Validation Tuning choices (layers, learning rate, stopping time) 10-20%
Test One final, honest score 10-20%

Shuffle before splitting unless the data has a time order (for time series you split by time instead, see Chapter 6). Fix a random seed so the split is repeatable.

Python Stand-in: shuffle and split into three sets (NumPy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In Keras you can also pass validation_split=0.2 to fit, which holds out the last 20% of the arrays before shuffling. If your data is sorted by class, shuffle it yourself first, or that last slice contains only one class.

Scaling the features

Neural networks train best when inputs are on similar, modest scales (roughly zero mean and unit variance, or between 0 and 1). A feature measured in thousands next to one measured in fractions makes the loss surface stretched and the updates erratic.

  • Standardization: (x - mean) / std.
  • Min-max scaling: (x - min) / (max - min).
  • Images: divide pixel values by 255.

The critical rule: compute mean and std from the training set only, then apply the same numbers to validation and test data. Computing them from everything leaks information about the test set.

Python Stand-in: standardize with training statistics only (NumPy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In Keras the same job is done by a layer, so the scaling travels with the model:

Python
norm = keras.layers.Normalization()
norm.adapt(x_train)            # learns mean and variance from training data

model = keras.Sequential([
    keras.Input(shape=(4,)),
    norm,
    keras.layers.Dense(16, activation="relu"),
    keras.layers.Dense(1),
])

For images, keras.layers.Rescaling(1/255) does the division inside the model.

Labels: integers or one-hot

For a classification with k classes, labels can be stored two ways, and the loss must match:

Label format Example for class 2 of 4 Loss
Integer 2 sparse_categorical_crossentropy
One-hot [0, 0, 1, 0] categorical_crossentropy

Integer labels are simpler and use less memory, so prefer them. When you need one-hot (for example to use label smoothing), convert with keras.utils.to_categorical(y, num_classes=4). For two classes, a single 0/1 label with binary_crossentropy and a sigmoid output is typical.

Python Stand-in: one-hot encoding by indexing (NumPy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Missing values and categories

Neural networks cannot take NaN. Fill or drop missing values before training (for example fill numeric gaps with the training median and add an "is missing" column). Text categories must become numbers: small sets can be one-hot encoded, large ones (thousands of categories) use an Embedding layer, covered in Chapter 6.

Batching and shuffling

Training does not feed the whole dataset at once. It uses mini-batches (commonly 32 to 256 examples). Batches make each step cheap, add helpful noise to the gradient and fit in memory. The data should be reshuffled every epoch so batches differ.

Python
history = model.fit(x_train, y_train,
                    batch_size=32,
                    epochs=20,
                    shuffle=True,                       # the default
                    validation_data=(x_val, y_val))

Below is what a batch generator does, written by hand. Chapter 7 replaces it with tf.data.

Python Stand-in: a mini-batch generator (NumPy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last batch can be smaller (here 2 examples). Keras handles that for you.

Common data mistakes

  • Leakage: scaling or selecting features with the whole dataset, or having the same example in train and test.
  • Wrong dtype: integer pixel arrays, or float64 doubling memory use. Cast with astype("float32").
  • Label/loss mismatch: integer labels with categorical_crossentropy raises a shape error.
  • Unshuffled, sorted data: the model sees one class at a time.
  • Forgetting the same preprocessing at prediction time: put preprocessing layers inside the model.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. From which data should the scaling mean and standard deviation be computed?

    Using only training statistics avoids leaking information from validation and test data.
  2. 2. Which loss matches integer class labels such as 0, 1, 2?

    The sparse variant takes class indices; categorical_crossentropy expects one-hot vectors.
  3. 3. What does validation_split=0.2 in fit do?

    Keras takes the final fraction of the data, so shuffle first if your data is ordered.
  4. 4. Why divide image pixel values by 255?

    Inputs of similar, modest scale make gradient-based training better behaved.

Summary

  • The first axis is the example axis; features are float32 arrays, labels are class indices or numbers.
  • Split into training, validation and test sets; shuffle first unless time order matters.
  • Scale features with statistics from the training set only, ideally inside the model with Normalization or Rescaling.
  • Match label format to loss: integers with the sparse loss, one-hot with the plain categorical loss.
  • Mini-batches are reshuffled each epoch; tf.data automates this later.

Next: Tensors and Shapes