Python Reference ยท Files, Paths and Text I/O
Iterating over files
Reference for iterating over a Python file object: for line in f, memory use, stripping newlines, enumerate, next(), chunked reading and generators.
A file object is an iterator over its lines. Writing for line in f: reads one line at a time, so you can process a file of any size using almost no memory. Each line arrives with its trailing newline attached.
This is the idiomatic way to read text files that you loop over once. It replaces readlines() in nearly all code, and combines well with enumerate(), generator expressions and sum().
with open("data.txt", encoding="utf-8") as f: for line in f: print(line.rstrip("\n"))
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.
Syntax
for line in file: ... next(file) next(file, default) iter(lambda: file.read(size), "")
| Name | Description |
|---|---|
file |
A file object in text mode (lines are str) or binary mode (lines are bytes, split on b"\n"). |
default |
Value next() returns at the end instead of raising StopIteration. |
Return value: each step yields the next line, including its line ending.
Raises: StopIteration from next(file) at the end of the file (the for loop handles this for you), ValueError if the file is closed, and UnicodeDecodeError for undecodable text.
Usage notes
- Memory friendly. The file is read in buffered blocks and split into lines lazily. A 10 GB log file loops fine;
f.readlines()would try to hold it all. - Newlines stay. Strip with
line.rstrip("\n")to keep other whitespace, orline.strip()to remove both ends. The final line may have no newline at all. - One pass only. Once the loop finishes, the file is at the end; a second loop does nothing unless you call
f.seek(0)first. - Shared position. The loop,
next(f),readline()andread()all use the same position. Mixing them works, so take a header withnext(f)first, then loop. One catch: in text modetell()raisesOSErrorafternext(f)or inside afor line in floop. - Use
enumerate(f, start=1)for line numbers in error messages. - The file must be open while you iterate. Returning a generator from a function that closes the file before the caller loops is a common bug; make the function a generator itself (
yieldinside thewith). - Universal newlines. In text mode,
\r\nand\rare read as\n, so line splitting works for files from any platform. - Binary files have no real lines. Iterate with
iter(lambda: f.read(size), b"")to process fixed-size chunks. - Very long lines are still read whole into memory; use
readline(limit)if you must bound them.