Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 49 ยท Machine Learning in Biology

Sequence Features: k-mers, One-hot Encoding and Logistic Regression

Learn how to turn DNA and protein sequences into numbers with k-mer counts, GC content and one-hot encoding, and train an interpretable logistic regression classifier.

A machine-learning model eats numbers. A DNA sequence is a string of letters. The step between them, called feature engineering or encoding, decides what the model can possibly learn. Give it only the GC content and it can never find a motif. Give it every position in a fixed-length window and it can learn position-specific rules but will struggle when a motif slides around. Choosing the representation is the most biological part of machine learning.

In this lesson you will meet the three workhorse encodings (summary statistics, k-mer counts and one-hot grids), train a logistic regression classifier on synthetic promoter-like sequences, and read its weights to see what it learned. You will also see a result that surprises many beginners: a more detailed encoding is not automatically a better one.

The sequence ATGCGTACG with sliding 3-mer windows, a one-hot grid with rows A, C, G and T, and a 64-cell 3-mer count vector with seven filled cells.
Caption: k-mer counts ignore order but are fixed-length and easy; one-hot keeps every position but needs aligned inputs.

What you will learn

  • How to compute GC content, k-mer counts and one-hot matrices.
  • What each encoding captures and what it throws away.
  • How logistic regression turns features into a probability, and how to read its weights.
  • Why the position of a motif changes which encoding works.

Key terms

k-mer: a substring of length k; there are 4^k possible DNA k-mers (64 for k = 3).

One-hot encoding: representing each base as a vector with a single 1, such as A = (1, 0, 0, 0).

Logistic regression: a model that outputs P(y = 1) = 1 / (1 + exp(-(w . x + b))).

Weight: the coefficient the model learns for a feature; large positive means "pushes toward class 1".

Standardization: rescaling each feature to mean 0 and standard deviation 1.

Three ways to encode a sequence

Summary statistics. The simplest features are single numbers such as length, GC content (the fraction of G and C), CpG count, or for proteins the fraction of charged residues. They are short, interpretable and sometimes surprisingly strong, because GC content differs between gene classes, isochores and organisms. They ignore order entirely.

k-mer counts. Slide a window of length k along the sequence and count how often each k-mer occurs. For DNA with k = 3 that gives a vector of 64 numbers (AAA, AAC, ..., TTT), as in the figure above; with k = 6 it is 4,096 numbers. Divide by the number of windows to get frequencies so sequences of different lengths are comparable. k-mer vectors have a fixed length whatever the sequence length, which is convenient, and they capture local patterns such as motifs and codon usage regardless of where they occur. They throw away the order of the k-mers and the positions. As k grows the vector gets huge and sparse, so most projects use k between 3 and 8, depending on how much data they have. For double-stranded DNA, people often count a k-mer and its reverse complement together (a canonical k-mer), because the sequence could be read from either strand.

One-hot encoding. Map each base to a four-component vector: A = (1,0,0,0), C = (0,1,0,0), G = (0,0,1,0), T = (0,0,0,1), and stack them into a 4 by L grid. This keeps every position and is lossless: you can recover the sequence. It is the standard input to convolutional neural networks. Flattened into one long vector of length 4L, it can also feed a simple model, but then each position is a separate feature, so the model can only learn rules tied to specific positions. For proteins, the same idea uses 20 rows (one per amino acid), or learned embeddings (see the deep-learning lesson).

Logistic regression in a paragraph

Despite the name, logistic regression is a classifier. For features x it computes a score z = w . x + b, a weighted sum, then squashes it into a probability with the sigmoid function p = 1 / (1 + exp(-z)). Training adjusts the weights w and bias b to maximize the likelihood of the labels, which is the same as minimizing the cross-entropy loss -[y log p + (1 - y) log(1 - p)] averaged over the examples. The loss is convex, so gradient descent finds the global minimum. Adding an L2 penalty on the weights (also called ridge regularization) discourages huge weights and fights overfitting.

Its biggest advantage in biology is interpretability. After training on standardized features, a large positive weight on the k-mer TAT means that sequences rich in TAT tend to be in class 1; you can sort k-mers by weight and read off what the model learned, and compare it with known motifs. Weights are only meaningful when features are on comparable scales, which is why we standardize them.

An experiment: finding a motif without being told

To see this in action, the examples below use synthetic data (a teaching model, not real genomes). Positive sequences are 60 bases of random DNA with a promoter-like motif planted at a random position. The motif is TATAAT, the consensus of the bacterial -10 box (the Pribnow box). Each motif base is mutated with probability 0.1, mimicking the fact that real motifs are fuzzy. Negative sequences are made by shuffling the letters of each positive sequence, so they have exactly the same base composition but no motif in a fixed place.

Which encodings will find the motif?

  • GC content carries no information, because shuffling preserves it exactly. Expect chance-level accuracy.
  • 3-mer counts include TAT, ATA, TAA and AAT, the pieces of the motif, so a linear model can pick them up wherever the motif sits. Expect clearly better than chance, with longer k-mers (4-mers such as TATA and ATAA) holding more of the motif and often doing a little better, until the number of features grows large compared with the training data.
  • Flattened one-hot needs the motif at the same position in every sequence. Because ours moves, no single position is informative, and the linear model has little to latch onto. Expect poor accuracy even though one-hot is the "richer" encoding.

The last point is the most instructive. More detailed information is not more useful when the model cannot combine it in a translation-invariant way. Convolutional networks, covered in the next lessons, solve exactly this problem by scanning one pattern detector over all positions.

Common misconceptions

"A richer encoding always gives a better model." One-hot is lossless but, fed to a simple linear model with a floating motif position, it is worse than k-mer counts. The encoding must match the structure of the problem.

"A large weight means a causal effect." Weights reflect association in your training data. Correlated features share weight, and confounders can create weights for features that do nothing biologically.

"I can standardize using the whole dataset." Compute the mean and standard deviation on the training set only and reuse them for the test set; otherwise information leaks.

Try it

First, code the encodings and check them against the figure. Second, run the motif experiment and compare encodings. Third, explore k-mers on your own sequence in the browser.

Python k-mer counts and one-hot encoding

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Which encoding finds the motif?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Explore the k-mers of your own sequence

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. How many different 4-mers are possible for DNA?

    4^4 = 256.
  2. 2. A sequence has length 50. How many 3-mer windows does it contain?

    A sequence of length L has L - k + 1 windows, here 50 - 3 + 1 = 48.
  3. 3. Why does a flattened one-hot encoding struggle when a motif appears at random positions?

    A linear model on a flattened grid learns position-specific rules; a floating motif breaks them.
  4. 4. On which data should you compute the mean and standard deviation for standardizing features?

    Using test data to compute scaling parameters leaks information from the test set.
  5. 5. What does the sigmoid function do in logistic regression?

    p = 1 / (1 + exp(-z)) maps any real score z into the interval (0, 1).

Summary

  • Models need numbers: encode sequences as summary statistics, k-mer frequencies (4^k features) or one-hot grids (4 by L).
  • k-mer counts capture local patterns independent of position but lose order; flattened one-hot keeps positions but is brittle when patterns slide.
  • Logistic regression outputs 1 / (1 + exp(-(w . x + b))), is trained with cross-entropy loss and L2 regularization, and its weights are readable.
  • Standardize with training-set statistics only, and read weights as associations, not causes.
  • The representation must fit the problem: the next lessons show how to evaluate honestly and how convolution filters remove the position problem.

Next: Evaluating Classifiers