Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 36 ยท Panel and Time Series

Panel Data: xtset, xtdescribe and xtsum

Declare and explore panel data in Stata with xtset, xtdescribe, xtsum and xttab: long format, balanced versus unbalanced panels, within and between variation.

Panel (longitudinal) data follows the same units, such as people, firms or countries, across several time periods. It is richer than a single cross-section because you can see how each unit changes, and it allows methods that remove unobserved differences between units. Before you can use Stata's panel commands, the data must be in long format and you must tell Stata which variable identifies the unit and which identifies time.

Stata cannot run on this site. Stata commands are read-only blocks. The runnable examples are Python (pandas) stand-ins that reproduce what xtdescribe and xtsum compute on a small simulated panel.

Long versus wide

Panel commands expect long format: one row per unit per period.

Text
id  year  wage  hours
 1  2020  15.2   38
 1  2021  15.9   40
 2  2020  12.1   35
 2  2021  12.1   37

If your file is wide (one column per year, such as wage2020 wage2021), convert it with reshape:

STATA
reshape long wage hours, i(id) j(year)

reshape wide goes back the other way. The i() option names the unit identifier and j() the new time variable. Check afterwards with isid id year, which stops with an error if any pair of values appears twice.

Declaring the panel with xtset

STATA
xtset id year

This one line does not change the data except to sort it; it records the panel structure so that every xt command, and the time-series operators L. and D., work correctly. Stata prints whether the panel is strongly balanced (every unit observed in every period) and the range of time values. Add a delta when time is not in steps of one: xtset id year, delta(5). If your unit ID is a string, first convert it: egen id = group(name) or encode name, gen(id).

If time has gaps, L.wage refers to the observation exactly one period earlier and is missing when that row does not exist; it does not skip to the previous row. That behaviour is what you want, and it is another reason to use xtset instead of sorting and subtracting by hand.

Describing the panel

STATA
xtdescribe
xtsum wage hours
xttab union
xttrans union

xtdescribe shows the participation patterns, for example "1111" for units seen in all four periods and "1.11" for a unit missing in period 2, along with the share of units for each pattern. It tells you at once whether the panel is balanced and how much attrition there is.

xtsum splits the standard deviation of each variable into three parts:

Component Meaning
overall Variation across all rows
between Variation of the unit means across units
within Variation of each unit around its own mean

A variable like sex or birth year has zero within variation; it never changes for a unit. That matters a great deal: a fixed-effects model cannot estimate the effect of any variable with no within variation.

xttab does the same decomposition for a categorical variable, and xttrans shows transition probabilities between categories from one period to the next.

Stand-in: a panel in pandas

The block simulates a small unbalanced panel and computes the patterns and the within/between split the way xtdescribe and xtsum do. It is a stand-in; Stata's own output layout differs.

Python xtdescribe and xtsum by hand (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The female row shows a within standard deviation of zero: it never varies inside a unit. wage has both kinds of variation, mostly between units here because each unit has its own base wage.

Creating lags, leads and differences

Once xtset is done, time-series operators work within each unit:

STATA
generate wage_lag  = L.wage      // previous period
generate wage_lead = F.wage      // next period
generate dwage     = D.wage      // first difference
generate wage_gr   = (wage - L.wage) / L.wage

Because the operators respect the unit boundary, the first row of each unit gets a missing lag, not the last row of the previous unit.

Python Lag and difference within units (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Practical advice

  • Run xtdescribe first and think about why units drop out; attrition that depends on the outcome biases results.
  • Always run xtsum on your key regressors before choosing a model.
  • Keep the unit ID and time variable names consistent across files so merges and xtset are painless.
  • xtset once at the top of the do-file, after any merge or append that might re-sort the data.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In what format must data be for xtset and xtreg?

    Panel commands need one row per unit-time observation; use reshape long to convert wide data.
  2. 2. What does xtset id year do?

    xtset records the panel structure; it does not estimate anything or balance the data.
  3. 3. A time-invariant variable such as birth year has which within standard deviation in xtsum?

    It does not change inside a unit, so within variation is zero.
  4. 4. What does L.wage give for a row whose previous period is absent?

    Lag operators refer to the exact previous time value, so a gap produces a missing lag.

Summary

  • Panel data needs long format; reshape long converts wide data and isid checks uniqueness.
  • xtset id year declares the structure; xtdescribe, xtsum, xttab explore it.
  • xtsum separates between and within variation, which determines what a fixed-effects model can estimate.
  • The pandas blocks are stand-ins for the Stata output.

Next: Fixed effects, random effects and the Hausman test