Skip to content
Tutorials Center
My learning

Topics

Lesson 103 of 135 ยท Advanced C

Memory-Mapped Files

Map files into memory with mmap, edit them through pointers, use MAP_PRIVATE copy-on-write, share memory between processes with MAP_SHARED and clean up with munmap.

Normally you read a file by asking the operating system to copy bytes into a buffer you provide (read, fread), and you write by copying bytes back out. Memory mapping takes a different approach: mmap asks the operating system to make a file's contents appear directly in your address space. After the call you get a pointer, and reading p[100] reads byte 100 of the file. Writing through the pointer can update the file. There's no explicit read loop and no buffer to manage.

The same mechanism also creates anonymous mappings (memory that isn't backed by any file), which malloc uses for large blocks and which can be shared between processes. This lesson covers mmap, munmap, msync and ftruncate from POSIX (<sys/mman.h>, Linux, macOS, BSDs). Windows has an equivalent API (CreateFileMapping) but not these functions. You should be comfortable with file descriptors from System Calls and File Descriptors and with the process model from Processes: fork, exec and Pipes.

How it works

A mapping connects a range of your virtual addresses to a file (or to nothing, for anonymous memory). The operating system loads pages from the file lazily: nothing is read until you touch a page, which triggers a page fault and the kernel fills the page from disk (or from its page cache, if the file was recently used).

flowchart LR
    F[("file on disk")] <-->|"page cache"| K["kernel"]
    K <-->|"page fault loads pages"| M["mapped region<br/>in your address space"]
    M --> P["char *data<br/>data[i] reads or writes the file"]

The mapped region is backed by the kernel's page cache: touching a byte loads its page, and for shared writable mappings changes are written back to the file.

The mmap call

C
void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);
Argument Meaning
addr Preferred address. Pass NULL and let the kernel choose
length Number of bytes to map (rounded up to whole pages internally)
prot Allowed access: PROT_READ, PROT_WRITE, PROT_EXEC or PROT_NONE, combined with |
flags MAP_SHARED or MAP_PRIVATE, plus options such as MAP_ANONYMOUS
fd An open file descriptor (ignored for anonymous mappings; use -1)
offset Where in the file the mapping starts. Must be a multiple of the page size

It returns the address of the mapping or the special value MAP_FAILED (not NULL) on error. Always compare against MAP_FAILED and print errno details with perror. When you're done, munmap(addr, length) removes the mapping. Closing the file descriptor does not unmap it: the mapping keeps the file alive, so you can close(fd) right after mmap.

For a file mapping you usually need the file size first, from fstat:

C
struct stat st;
fstat(fd, &st);
size_t size = (size_t)st.st_size;

Mapping an empty file (size 0) fails with EINVAL, and touching bytes beyond the end of the file inside the last page is fine but beyond that page raises SIGBUS.

Reading a file without read

In this example we create a small file, map it read-only and treat the mapped bytes like any array: counting lines with a plain loop.

C Reading a file through a pointer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

No read call and no buffer: data is the file. For a large file that you only inspect in a few places, the system loads just the pages you touch, which can be much cheaper than reading everything. For small files and sequential reads, plain read or fread is just as fast and simpler, so don't assume mapping is always quicker. Remember that data is a pointer into a file: functions like strlen or printf("%s") only work if the file content is zero-terminated, which a text file isn't.

MAP_SHARED versus MAP_PRIVATE

The two flags decide what happens to writes:

Flag Writes are... Visible to
MAP_SHARED Carried through to the file (eventually) Other processes mapping the same file, and anyone who reads the file
MAP_PRIVATE Copy-on-write: the first write to a page gives you a private copy Only your process; the file is never changed

To write to a file through a MAP_SHARED mapping, open it with O_RDWR and ask for PROT_READ | PROT_WRITE. The kernel writes modified pages back at some later point; to force it, call msync(addr, length, MS_SYNC). On munmap or process exit the changes are not lost.

C Editing a file by writing to memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

The private mapping changes the in-memory text to upper case, but the file is untouched. The shared mapping, by contrast, changes the first five bytes of the file itself, and a normal read sees the new content right away because both go through the same page cache.

Warning: If another process truncates a file you have mapped, touching a page past the new end of the file raises SIGBUS and kills your program (see Signals). Real programs that map shared files either lock them or handle that signal carefully.

Growing a file with ftruncate

A mapping can't extend a file by itself. To create a new file of a given size and map it, set the size first with ftruncate(fd, size), which fills the file with zero bytes, then map it. After that, writing into the mapping is how you fill the file.

C
int fd = open("data.bin", O_RDWR | O_CREAT, 0644);
ftruncate(fd, 4096);                                     /* now it is 4096 zero bytes */
unsigned char *p = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
                        MAP_SHARED, fd, 0);
p[0] = 0xCA;                                             /* writes the first byte of the file */

Databases use exactly this to keep indexes and storage files in memory-mapped regions: a pointer into the mapped file is as cheap to follow as any other pointer, and the kernel decides which pages stay in RAM.

Anonymous mappings and shared memory

If you pass MAP_ANONYMOUS (and fd = -1) there's no file at all: you get zero-filled memory straight from the kernel. This is how malloc obtains big blocks, and it lets you request page-aligned memory directly. Combined with MAP_SHARED, the memory is shared with child processes created after the mapping, because fork copies the mapping table but both processes still refer to the same physical pages. That makes it a fast form of inter-process communication, an alternative to the pipes of Processes: fork, exec and Pipes.

flowchart TD
    A["parent: mmap(MAP_SHARED, MAP_ANONYMOUS)"] --> B["fork()"]
    B --> C["child writes into the region"]
    B --> D["parent waitpid(child)"]
    C --> E["shared pages"]
    D --> E
    E --> F["parent reads the child's data"]

After fork both processes see the same physical pages, so the parent reads what the child wrote.

C Sharing memory between parent and child

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

With an ordinary malloced struct the parent would see zeros, because fork gives the child its own copy. With the shared mapping both see the same memory.

Note: Sharing memory means sharing data races. Here the parent waits for the child to finish, which orders the accesses. When both processes run at the same time you need synchronization, such as process-shared mutexes or semaphores, or atomics placed in the shared region (see volatile and C11 Atomics). Also keep pointers out of shared structs: the region can be mapped at different addresses in different processes, so store offsets and array indexes instead.

For sharing between unrelated processes, POSIX provides shm_open("/name", ...), which returns a descriptor you ftruncate and mmap. Two programs that open the same name see the same memory. It is the same pattern with a name instead of a fork; you may need -lrt on older systems, and you should shm_unlink the name when finished.

Alignment, page size and other details

  • Pages are typically 4 KiB. Get the real value with sysconf(_SC_PAGESIZE). A non-zero offset must be a multiple of it.
  • mmap returns a page-aligned address, so any type with an alignment requirement can live at its start.
  • madvise(addr, len, MADV_SEQUENTIAL) and similar hints tell the kernel your access pattern.
  • mlock pins pages in RAM, useful for secrets that must not be swapped out.
  • Mapping a file doesn't make I/O errors disappear; they surface as SIGBUS instead of an error return from read.
  • Windows programs use CreateFileMappingW and MapViewOfFile instead.

When to use mmap

Use mmap when Use read/fread when
Random access into a big file (databases, indexes) Reading a file once from start to end
Several processes need the same read-only data The file might shrink while you work
You want to treat file content as a struct or array in place You want portability to non-POSIX systems
You need fast shared memory between related processes You process streams, sockets or pipes (which can't be mapped)

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does mmap return when it fails?

    mmap returns MAP_FAILED on error and sets errno. NULL is not the error value, so comparing the result with NULL is a bug.
  2. 2. You map a file with MAP_PRIVATE and PROT_WRITE and change some bytes. What happens to the file?

    MAP_PRIVATE gives you your own copy of each page you modify. Use MAP_SHARED if the changes must reach the file.
  3. 3. Which flags create memory that a parent and a child created with fork can both read and write?

    A shared mapping keeps referring to the same physical pages after fork. malloc and private mappings give each process its own copy.
  4. 4. Why does mapping an empty file with mmap fail?

    mmap rejects a zero length with EINVAL. For a new file, first set its size with ftruncate, then map it.
  5. 5. Is it safe to close the file descriptor right after a successful mmap?

    The kernel holds its own reference to the file, so the descriptor is no longer needed. The mapping is removed only by munmap or when the process exits.

Summary

  • mmap maps a file (or anonymous memory) into your address space; you then use ordinary pointer reads and writes, and the kernel loads pages lazily.
  • It returns MAP_FAILED on error; release a mapping with munmap. Closing the descriptor doesn't unmap it.
  • MAP_SHARED writes reach the file and other mappers (flush with msync); MAP_PRIVATE writes are copy-on-write and never reach the file.
  • Set the size of a new file with ftruncate before mapping it; offsets must be page aligned.
  • MAP_SHARED | MAP_ANONYMOUS before fork gives parent and child fast shared memory; shm_open does the same for unrelated processes. Synchronize access.
  • Truncating a mapped file causes SIGBUS; prefer read/fread for simple sequential reads and non-POSIX portability.

Next: Project: Command Calculator