Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 37 ยท Channel Capacity

The Gaussian Channel and the Shannon-Hartley Limit

Learn the AWGN channel capacity C = B log2(1 + SNR), convert decibels, find the minimum Eb/N0 of -1.59 dB, and check the formulas with Python simulations.

Wires, radio waves and optical fibres carry continuous signals, not tidy bits. Noise on such links is often well modelled as additive white Gaussian noise (AWGN): the receiver sees Y = X + Z, where Z is a random Gaussian value with variance N that is independent of the signal. The sender may use at most an average power P.

This lesson derives the capacity of that channel, C = 1/2 log2(1 + P/N) bits per real use, and the famous Shannon-Hartley formula C = B log2(1 + S/N) for a band-limited link. You also meet the Shannon limit, the lowest signal energy per bit at which any system can work.

The capacity formula

Mutual information is I(X;Y) = h(Y) - h(Y|X), where h is differential entropy. Given X, the output differs only by the noise, so h(Y|X) = h(Z) = 1/2 log2(2ฯ€e N). Among all signals with power P, the output with the largest differential entropy is Gaussian, with variance P + N: h(Y) = 1/2 log2(2ฯ€e (P + N)). Subtract:

TEXT
C = 1/2 log2(1 + P/N)       bits per real channel use

The ratio P/N is the signal-to-noise ratio (SNR). Capacity grows only logarithmically with power: doubling the power adds just half a bit per use at high SNR. Each additional bit costs a factor of 4 in power.

Bandwidth and the Shannon-Hartley theorem

A channel limited to bandwidth B hertz can carry 2B independent real samples per second (the Nyquist rate). With noise power N = N0 B (where N0 is the noise density), you get:

TEXT
C = B log2(1 + S/N)       bits per second

SNR is usually quoted in decibels: SNR_dB = 10 log10(S/N). A 30 dB SNR is a ratio of 1000.

Python Capacity of real links

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two levers, bandwidth and SNR, are traded against each other. At high SNR extra bandwidth gives a proportional gain, but extra power gives only a logarithmic one. At low SNR the situation reverses.

Checking the formula by simulation

A numerical check of h(Y) - h(Y|X) is straightforward if the input is Gaussian. The mutual information per sample is the average of log2( p(y|x) / p(y) ), where p(y|x) is a normal density with variance N around x, and p(y) a normal with variance P + N.

Python Monte Carlo estimate of I(X;Y)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The estimate sits within a few thousandths of 1.1610 bits. A Gaussian input reaches capacity; a binary input at the same power, as in simple modulations, cannot exceed 1 bit per use, which is why high-rate systems use many amplitude levels (QAM constellations).

Energy per bit and the Shannon limit

Engineers compare systems by Eb/N0: energy per information bit divided by noise density. At spectral efficiency r bits per second per hertz, the signal power is S = r B Eb, so S/N = r Eb/N0. Setting r = C/B in Shannon-Hartley:

TEXT
r = log2(1 + r * Eb/N0)   =>   Eb/N0 = (2^r - 1) / r

As r shrinks toward 0 (using enormous bandwidth), (2^r - 1)/r tends to ln 2 โ‰ˆ 0.693, which is -1.59 dB. Below this value no code at all can be reliable, whatever the bandwidth. This is the ultimate Shannon limit.

Python Minimum Eb/N0 for each spectral efficiency

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The table shows that high spectral efficiency costs energy: sending 8 bit/s/Hz needs about 17 dB more Eb/N0 than the limit. Low-rate systems, like deep-space probes, use enormous bandwidth to operate close to -1.6 dB.

Do not confuse SNR with Eb/N0. SNR compares total signal power to total noise power in the band; Eb/N0 is normalized per information bit, so it lets you compare systems with different rates and bandwidths fairly.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the capacity of the AWGN channel per real channel use?

    C = 1/2 log2(1 + P/N). Multiply by 2B real samples per second to get B log2(1 + S/N) in bit/s.
  2. 2. A 30 dB signal-to-noise ratio corresponds to a power ratio of:

    SNR_dB = 10 log10(ratio), so a ratio of 10^(30/10) = 1000.
  3. 3. What is the Shannon limit on Eb/N0 for reliable communication?

    As spectral efficiency goes to zero, the minimum Eb/N0 tends to ln 2, about -1.59 dB. Below it no code can be reliable.
  4. 4. At high SNR, what does doubling the transmit power add to the capacity per real use?

    Capacity grows like 1/2 log2(SNR), and doubling SNR adds 1/2 log2(2) = 1/2 bit.

Summary

  • On an AWGN channel with SNR P/N the capacity is 1/2 log2(1 + P/N) bits per real use.
  • With bandwidth B, Shannon-Hartley gives C = B log2(1 + S/N) bit/s.
  • Power buys capacity logarithmically; bandwidth buys it linearly.
  • Eb/N0 = (2^r - 1) / r is the minimum energy per bit for spectral efficiency r.
  • No system works below Eb/N0 = ln 2, which is -1.59 dB.

Next: Error Detection: Parity, Checksums and CRC