Skip to content
Tutorials Center
My learning

Topics

Lesson 75 of 135 ยท File Input and Output

Low-Level I/O with System Calls

Learn low-level file I/O in C with POSIX system calls - file descriptors, open flags and modes, read, write, lseek, errno, dup2 redirection, and stdio vs syscalls.

fopen, fprintf and fgets are library functions. Underneath them, the C library asks the operating system kernel to do the real work through a handful of system calls: open, read, write, lseek and close. On Linux, macOS and other POSIX systems you can call these directly. They're simpler than stdio in one way (they just move bytes, with no formatting and no buffer) and harder in another (you handle partial transfers, errors and efficiency yourself).

Why learn them when FILE * works? Because some things only exist at this level: pipes, sockets, devices, file permissions at creation time, exclusive creation, redirecting a program's output with dup2, and the file descriptors that sockets and signals code works with. Knowing the layer underneath also explains stdio's behavior, such as why output sometimes appears out of order. These functions are POSIX, not standard C; Windows has similar _open/_read functions in its C runtime and its own CreateFile API.

You should know file handling with FILE pointers and error handling with errno first.

File descriptors

The kernel identifies each open file of a process by a small non-negative integer called a file descriptor (fd). Every process starts with three already open:

fd Name in <unistd.h> stdio stream Usually connected to
0 STDIN_FILENO stdin the keyboard or a pipe
1 STDOUT_FILENO stdout the terminal, a pipe or a file
2 STDERR_FILENO stderr the terminal

open returns the lowest unused number, so a program's first opened file is normally fd 3. read(fd, buffer, count) and write(fd, buffer, count) work on any descriptor: a file, the terminal, a pipe or a network connection. Both return the number of bytes actually transferred, or -1 on error with errno set. A read that returns 0 means end of file.

C Reading and writing descriptors 0, 1 and 2

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

Notice what's missing: no format strings, no %s, no automatic '\0'. read stores exactly the bytes it received, including the newline from the Enter key and without any '\0', so the program shortens the count to drop the newline and passes n to write instead of relying on a terminator. The types are POSIX too: ssize_t is a signed size that can hold -1.

Opening files: flags and modes

open(path, flags) or open(path, flags, mode) comes from <fcntl.h>. The flags combine one access mode with optional behavior bits using |:

Flag Meaning
O_RDONLY, O_WRONLY, O_RDWR Read only, write only, or both (exactly one)
O_CREAT Create the file if it doesn't exist (requires the mode argument)
O_TRUNC Empty an existing file when opening it for writing
O_APPEND Every write goes to the end of the file, even with several writers
O_EXCL With O_CREAT: fail with EEXIST if the file already exists
O_CLOEXEC Close automatically if the program starts another program with exec

fopen's modes are shorthands for these: "r" is O_RDONLY, "w" is O_WRONLY | O_CREAT | O_TRUNC, and "a" is O_WRONLY | O_CREAT | O_APPEND. Only the system call gives you O_EXCL, which atomically creates a file only if it's new. That's how lock files and "don't overwrite" checks avoid race conditions.

The mode sets the new file's permission bits, written in octal: 0644 means the owner may read and write, everyone else may only read. The process's umask then removes bits from it (a typical umask of 022 removes write permission for group and others).

C Creating files with open

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

The new file gets descriptor 3, and stat confirms its size and permissions. The second open fails because O_EXCL refuses to touch an existing file, and opening a missing file for reading fails with ENOENT. As always with errno, read it right after the failing call, before anything else can change it.

Close every descriptor you open. A process can only have a limited number open at once (often 1024 by default on Linux), and a long-running server that leaks descriptors eventually fails with EMFILE, "Too many open files".

Partial reads and writes

read(fd, buf, 4096) doesn't promise 4096 bytes. It returns what's available: fewer near the end of a file, often much fewer from a pipe, terminal or socket, and 0 at end of file. write can also transfer fewer bytes than requested (for example on a pipe or socket, or when a signal interrupts it). Robust code therefore loops. A call interrupted by a signal before transferring anything fails with errno == EINTR and should simply be retried.

C Copying a file with read and write loops

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

The 43-byte poem arrives in chunks of 16, 16 and 11 bytes, and the next read returns 0, which ends the loop. Notice that close on the output file is checked too: some file systems report write errors only when the file is closed.

Moving around with lseek

Every open descriptor has a file offset, the position where the next read or write happens. lseek(fd, offset, whence) moves it and returns the new position, measured from the start (SEEK_SET), the current position (SEEK_CUR) or the end (SEEK_END). lseek(fd, 0, SEEK_END) therefore returns the file's size. It's the system-call version of fseek/ftell from Binary Files, and like them it's perfect for fixed-size records:

C Updating one record in place with lseek

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

After reading record 1, the offset sits at the start of record 2, so SEEK_CUR with a negative offset steps back one record before writing. POSIX also offers pread and pwrite, which take the offset as an argument and leave the file offset alone: handy when several threads share one descriptor. (Raw structs in files have portability limits; Binary Files explains padding and byte order.)

Buffering: why stdio exists

Each system call is a switch into the kernel, which costs far more than an ordinary function call. Writing a 1 MB file one byte at a time with write makes a million system calls; with fputc, stdio collects the bytes in a buffer (typically 4 KB or more) and calls write only a few hundred times. That buffer is the main reason the stdio layer exists, along with formatting and portability.

The flip side is that stdio's buffer and the raw descriptor are two separate paths to the same file. If you mix them, output can come out in a different order than you wrote it:

C Mixing printf and write

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

Line 2 appears first: when stdout is a pipe or a file (as it is here), stdio buffers it fully, so lines 1 and 3 sit in memory until the explicit fflush, while write goes straight out. In a terminal stdout is line-buffered and you'd see 1, 2, 3, which makes this bug hard to spot. Flushing before every write restores the order. The rule: don't mix stdio and raw calls on the same file, or flush at every switch. To move between the layers deliberately, fileno(stream) gives a stream's descriptor and fdopen(fd, mode) wraps a descriptor in a FILE *.

Redirecting with dup and dup2

dup(fd) creates a second descriptor for the same open file, using the lowest free number. dup2(old, new) makes new refer to the same file as old, closing new first if it was open. Since printf always writes to descriptor 1, pointing descriptor 1 at a file redirects everything the program prints. This is exactly what a shell does for program > out.txt before it starts the program:

C Redirecting stdout to a file with dup2

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

The two middle lines never reach the screen; they land in output.log. The fflush calls matter for the reason you just saw: stdio's buffer has to be emptied before the descriptor underneath changes, or buffered text ends up in the wrong place. Shells combine dup2 with fork, exec and pipe to build pipelines like ls | sort, and servers use it to send a daemon's output to a log file. For a pure-stdio alternative, freopen("file", "w", stdout) redirects the stream itself.

Which layer should you use?

Use stdio (FILE *) when Use system calls when
You read or write text, lines or formatted values You need flags stdio lacks: O_EXCL, O_APPEND with several writers, permissions, O_CLOEXEC
You do many small reads or writes (buffering pays off) You work with pipes, sockets or devices, often non-blocking
The code must be portable to any C compiler (stdio is standard C) You redirect descriptors with dup2, or pass them to child processes
You need precise control over when data reaches the kernel (fsync)

Most programs use stdio for ordinary files and system calls for everything else. Both report errors through errno, so the techniques from Error Handling apply unchanged.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A program has only the standard descriptors open and calls open() successfully. Which descriptor does it normally get?

    open returns the lowest unused descriptor, and 0, 1 and 2 are taken by stdin, stdout and stderr.
  2. 2. What does read(fd, buf, 100) returning 0 mean?

    0 means there is nothing more to read. Errors return -1, and a short positive count is a normal partial read.
  3. 3. Which flags give the same behavior as fopen with mode "w"?

    "w" opens for writing, creates the file if needed and empties an existing one. O_EXCL would make it fail when the file exists.
  4. 4. Output goes to a pipe. The program runs printf("A\n"); write(1, "B\n", 2); printf("C\n"); and exits normally. What is printed?

    stdout to a pipe is fully buffered, so A and C wait in stdio's buffer until exit, while write sends B immediately.
  5. 5. What does dup2(file_fd, STDOUT_FILENO) do?

    dup2 points descriptor 1 at the open file. Everything written to fd 1, including printf output once flushed, goes into that file.

Summary

  • System calls are the kernel's interface; stdio's FILE * functions are a buffered, portable library built on top of them. They're POSIX, not standard C.
  • A file descriptor is a small integer: 0, 1 and 2 are stdin, stdout and stderr, and open returns the lowest free number.
  • open(path, flags, mode) combines O_RDONLY/O_WRONLY/O_RDWR with O_CREAT, O_TRUNC, O_APPEND, O_EXCL; the octal mode is filtered by the umask.
  • read and write return the number of bytes transferred, 0 from read at end of file, and -1 with errno on error. Loop for partial transfers and retry on EINTR.
  • lseek moves the file offset (SEEK_SET, SEEK_CUR, SEEK_END); pread/pwrite read and write at an offset without moving it.
  • Each system call is expensive; stdio's buffer batches them. Don't mix stdio and raw calls on one file without fflush.
  • dup and dup2 duplicate descriptors, which is how output redirection works. Close every descriptor you open and check close's result.

Next: The Math Library