Skip to content
Tutorials Center
My learning

Topics

Lesson 46 of 66 ยท Files and Storage

File Concepts and Access Methods

Learn what a file is to an operating system, which attributes and operations it has, and how sequential, direct and indexed access methods work, with Python simulations.

Memory forgets everything when the power goes off. Programs, documents, photos and databases need a home that survives a reboot, and the operating system provides it with one simple idea: the file. A file is a named collection of related bytes kept on secondary storage, such as an SSD or a hard disk. Applications never see disk sectors or flash pages. They see names and bytes, and the file system does the translating.

This chapter follows that idea down through the layers. This first lesson covers what a file is, what the OS remembers about it, which operations you can perform, and the three classic ways to read its contents. The next lessons cover directories, allocation of disk blocks, free space and the Unix file system.

What the OS knows about a file

A file has contents (the bytes) and metadata (data about the data). The metadata is called the file's attributes:

Attribute Meaning
Name The human-readable label, such as report.txt
Identifier A number the file system uses internally (an inode number on Unix)
Type Regular file, directory, device, link, and so on
Location Where on the device the data blocks live
Size Current length in bytes, and sometimes the space reserved
Protection Who may read, write or execute it
Timestamps Created, last modified, last accessed
Owner The user (and group) responsible for it

The attributes are stored on disk too, in a structure that belongs to the file system. On Unix it is the inode; on FAT and NTFS it is a directory entry or MFT record. Notice that the name is just one attribute. On Unix the name is not even inside the inode: it is stored in the directory that contains the file.

The OS does not usually care what the bytes mean. A text file, a JPEG and a program are all "a sequence of bytes". The meaning comes from conventions such as the extension (.txt, .png), a magic number at the start of the file (a PNG begins with 89 50 4E 47), or the application that opens it.

Python Attributes and magic numbers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The "type" column came from the content, not the name. Rename logo.png to logo.txt and the bytes are still a PNG.

File operations

The OS exposes a small set of system calls that every program uses, whatever the language:

  • create: find space, add a directory entry, initialize the attributes.
  • open: look the name up once, check permissions, and hand back a small integer called a file descriptor (or handle). Later calls use the descriptor instead of repeating the lookup.
  • read / write: move bytes between the file and the program's memory, starting at the current file position kept by the OS for that open file.
  • seek (reposition): change the current position without reading or writing.
  • truncate: cut the file down to a given length, or to zero.
  • close: release the descriptor and flush pending data.
  • delete: remove the directory entry and free the data blocks (once nobody has the file open).

The OS keeps two tables to track open files. A system-wide open-file table holds one entry per open file with a count of how many processes use it, and a per-process table maps each descriptor to a system-wide entry plus the current position. That arrangement is why two programs can read the same file at different positions at the same time.

flowchart LR
    P1["Process A<br/>fd 3"] --> E1["Open entry<br/>position 120"]
    P2["Process B<br/>fd 5"] --> E2["Open entry<br/>position 0"]
    E1 --> I["Inode of notes.txt"]
    E2 --> I
    I --> D["Data blocks on disk"]

Two processes open the same file. Each has its own position, and both reach the same inode and blocks.

Access methods

How do programs reach the bytes inside a file? There are three classic patterns.

Sequential access

Bytes are read or written in order, one after another, like a tape. A read advances the position automatically; "rewind" moves it back to the start. Compilers, text editors and log writers use this almost exclusively, and it is the model behind standard input and output.

Direct (random) access

The file is treated as an array of fixed-size records or blocks. You can jump to record number n without reading the first n - 1 records. The position is simple arithmetic: offset = n * record_size. Databases and swap files rely on this.

Indexed access

An index (like the index at the back of a book) maps a key to the position of its record. To find the record for customer 4711, you search the small index, then jump straight to the data. Large files can have a multilevel index, with an index of indexes. This is how ISAM and database tables work.

Python Sequential, direct and indexed access

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Sequential search touched four records to find the last key. Direct and indexed access touched exactly one. Real indexes are kept sorted or built as B-trees so they stay small and fast even for millions of records.

Memory-mapped files

Another way to access a file is to map it into a process's address space with mmap. The file's bytes then look like an ordinary array in memory. The first touch of a page causes a page fault (see demand paging), and the OS loads that page from the file. Writes go to the page cache and are flushed to disk later. This removes copying between kernel and user buffers and makes random access natural.

Common mistakes

  • Thinking the name is the file. Several names (hard links) can point to one file, and a file can stay alive after its last name is deleted if a process still has it open.
  • Forgetting to close. Descriptors are a limited resource; leaking them eventually makes open fail with "too many open files".
  • Assuming a write is on disk when write returns. The data usually sits in the OS cache; fsync forces it to storage.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which item is part of a file's metadata (attributes) rather than its contents?

    Attributes such as owner, size, timestamps and protection describe the file; the bytes inside are its contents.
  2. 2. Why does open return a file descriptor?

    The name is resolved and permissions are checked once. After that the small descriptor identifies the open file.
  3. 3. Which access method computes a record's position as record_number * record_size?

    With fixed-size records the position is plain arithmetic, so you can jump straight to any record.
  4. 4. What does an index add to a data file?

    The index is a small structure that finds a key quickly and then points to the record in the data file.

Summary

  • A file is a named sequence of bytes on secondary storage; the OS tracks its attributes (name, type, size, owner, permissions, timestamps, location) separately from its contents.
  • Core operations are create, open, read, write, seek, truncate, close and delete. Open files are tracked in a per-process table and a system-wide table.
  • Sequential access reads in order, direct access jumps to a numbered record, and indexed access looks up a key in an index first.
  • Memory mapping lets a process treat a file as memory, with the OS paging it in on demand.

Next: Directory Structures and Path Names