Skip to content
Tutorials Center
My learning

Topics

Lesson 7 of 37 ยท Environment and Data Steps

The Program Data Vector and DATA Step Execution

Understand the compile and execution phases of the SAS DATA step, the program data vector, _N_ and _ERROR_, implicit OUTPUT and RETURN, and explicit OUTPUT and DELETE.

Behind every DATA step is a memory area called the program data vector (PDV). Once you picture the PDV, many behaviors that look like magic, such as why a variable resets to missing on each row or why if can delete rows, become obvious. This lesson walks through exactly what SAS does when it runs a step.

Two phases

SAS processes each DATA step in two phases.

1. Compile phase. SAS checks syntax and builds the PDV, which has one slot for every variable the step will use, plus two automatic variables, _N_ and _ERROR_. It also records the descriptor portion (variable names, types, lengths) of the output data set. No data is read yet, except to learn variable attributes from an input data set's header.

2. Execution phase. SAS runs the step's statements, one pass of the DATA step loop per observation:

  1. At the top of the step, set all PDV variables to missing (except retained ones and those read from SET).
  2. Read the next observation from the input into the PDV.
  3. Execute the statements in order.
  4. At the bottom (the run;), write the PDV contents to the output data set. This is the implicit OUTPUT.
  5. Return to the top and repeat until there is no more input; then the step ends.
flowchart TD
    A["Compile: build PDV and descriptor"] --> B["Top of loop: initialize variables to missing"]
    B --> C["SET: read next observation into the PDV"]
    C --> D["Run the statements"]
    D --> E["End of step: implicit OUTPUT writes the PDV row"]
    E --> F{"More input?"}
    F -- yes --> B
    F -- no --> G["Step ends"]

Figure: the DATA step loop.

Automatic variables

Variable Meaning
_N_ Number of times the step has begun its loop (starts at 1)
_ERROR_ 1 if the current iteration had a data error (such as invalid input), otherwise 0

They are in the PDV while the step runs but are not written to the output data set. In a step that reads one observation per iteration, _N_ equals the observation number, but they differ once you read several rows or none per loop.

SAS
data first_three;
   set sashelp.class;
   if _N_ <= 3;
run;

Variables are reset each iteration

When a new iteration starts, variables that come from the input data set are overwritten by the next observation. Variables you create are set back to missing at the top of every iteration. Consider this step:

SAS
data demo;
   set sashelp.class;
   count = count + 1;
run;

count starts every iteration as missing, so count + 1 is always missing: the step produces a column of missing values and the log says that missing values were generated. To carry a value from row to row you need RETAIN or the sum statement, covered in RETAIN and Running Totals.

Controlling output

By default each iteration writes one observation. Two statements change that:

  • OUTPUT; writes the current PDV to the output data set right now. Once a step contains an explicit output statement, SAS stops doing the automatic write at the end. You can output a row zero, one or several times per iteration.
  • DELETE; stops the current iteration immediately and does not write the observation.
SAS
data double_rows;
   set sashelp.class;
   output;            /* first copy */
   weight = weight * 2;
   output;            /* second copy, modified */
run;

data adults_only;
   set sashelp.class;
   if age < 16 then delete;
run;

RETURN; sends control back to the top of the loop (and performs the implicit output, when no explicit one exists). It is seldom needed.

What if there is no SET statement?

A DATA step without any input statement runs once. This is perfect for creating a small data set from assignments, or for _NULL_ steps that only write files:

SAS
data tiny;
   x = 42;
   y = x * 2;
run;

Special data set name _null_ tells SAS not to create a data set at all. You will use that when writing reports to files in Writing Data with FILE and PUT.

When the step ends

The loop ends when the SET statement tries to read past the last observation. At that moment the step stops completely: statements after set do not run again. This is why a set inside an if can cause an endless loop in a careless program, and why you can write stop; to end a step on purpose.

Seeing the PDV with PUTLOG

You can inspect the PDV by writing it to the log:

SAS
data _null_;
   set sashelp.class;
   if _N_ <= 2;
   putlog _all_;
run;

_all_ prints every variable in the PDV with its name and value. Using putlog is the quickest debugging technique in SAS.

A PDV simulator

This Python stand-in (not SAS) mimics the loop, resetting created variables to missing (None) on each iteration, with and without a retained counter.

Python Python stand-in: PDV reset versus RETAIN

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the compile phase of a DATA step build?

    SAS checks syntax and creates the PDV with all variables, plus N and ERROR.
  2. 2. What value does a newly created variable have at the top of each iteration?

    Created variables are reset to missing unless retained.
  3. 3. What does DELETE; do?

    The row is not written to the output; the loop continues with the next one.
  4. 4. Are N and ERROR written to the output data set?

    They are automatic variables available in the PDV, not saved to the output.

Summary

  • A DATA step has a compile phase (build the PDV) and an execution phase (loop over observations).
  • Each iteration: reset variables, read a row, run statements, implicitly write the row.
  • _N_ counts iterations; _ERROR_ flags data errors; neither is saved.
  • OUTPUT writes explicitly, DELETE skips the write, and created variables reset to missing unless retained.
  • putlog _all_; shows the PDV for debugging.

Next: Conditional Logic: IF-THEN/ELSE and SELECT