Skip to content
Tutorials Center
My learning

Topics

Python Reference ยท Files, Paths and Text I/O

Iterating over files

Reference for iterating over a Python file object: for line in f, memory use, stripping newlines, enumerate, next(), chunked reading and generators.

A file object is an iterator over its lines. Writing for line in f: reads one line at a time, so you can process a file of any size using almost no memory. Each line arrives with its trailing newline attached.

This is the idiomatic way to read text files that you loop over once. It replaces readlines() in nearly all code, and combines well with enumerate(), generator expressions and sum().

Python
with open("data.txt", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n"))

Try it

Python Looping over lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python next(), shared position and chunks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Syntax

Python
for line in file:
    ...

next(file)
next(file, default)
iter(lambda: file.read(size), "")
Name Description
file A file object in text mode (lines are str) or binary mode (lines are bytes, split on b"\n").
default Value next() returns at the end instead of raising StopIteration.

Return value: each step yields the next line, including its line ending.

Raises: StopIteration from next(file) at the end of the file (the for loop handles this for you), ValueError if the file is closed, and UnicodeDecodeError for undecodable text.

Usage notes

  • Memory friendly. The file is read in buffered blocks and split into lines lazily. A 10 GB log file loops fine; f.readlines() would try to hold it all.
  • Newlines stay. Strip with line.rstrip("\n") to keep other whitespace, or line.strip() to remove both ends. The final line may have no newline at all.
  • One pass only. Once the loop finishes, the file is at the end; a second loop does nothing unless you call f.seek(0) first.
  • Shared position. The loop, next(f), readline() and read() all use the same position. Mixing them works, so take a header with next(f) first, then loop. One catch: in text mode tell() raises OSError after next(f) or inside a for line in f loop.
  • Use enumerate(f, start=1) for line numbers in error messages.
  • The file must be open while you iterate. Returning a generator from a function that closes the file before the caller loops is a common bug; make the function a generator itself (yield inside the with).
  • Universal newlines. In text mode, \r\n and \r are read as \n, so line splitting works for files from any platform.
  • Binary files have no real lines. Iterate with iter(lambda: f.read(size), b"") to process fixed-size chunks.
  • Very long lines are still read whole into memory; use readline(limit) if you must bound them.