Skip to content
Tutorials Center
My learning

Topics

Lesson 79 of 147 ยท Standard Library

Files, Directories and Paths

Learn to create, open, read and write files in Go, list and walk directories, build portable paths with path/filepath, use io/fs and embed, set permissions and confine access with os.Root.

Almost every real program touches the file system: it reads a configuration file, writes a log, stores uploads, scans a project folder or saves a result. Go's standard library covers this with three packages that work together: os to open, create and remove things, path/filepath to build and take apart file names correctly on every operating system, and io/fs, a small abstraction that lets the same code read from a real disk, an embedded archive or an in-memory test fixture.

You already know os.ReadFile and os.WriteFile from os, io and bufio. This lesson goes further: opening files with flags, reading and writing in pieces, safe patterns for saving data, directories and walking trees, paths, permissions, io/fs and embed, and the os.Root type (added in Go 1.24) that stops a program from escaping a folder. The sandbox that runs the examples allows temporary files, so every program here creates its files in a directory made by os.MkdirTemp and removes it again when it finishes.

Opening and creating files

There are three ways to get an *os.File:

Function What it does
os.Open(name) opens an existing file for reading only
os.Create(name) creates the file, or truncates it to zero length if it exists; opens for reading and writing
os.OpenFile(name, flags, perm) full control with flags and permissions

OpenFile takes flags that you combine with |:

Flag Meaning
os.O_RDONLY, os.O_WRONLY, os.O_RDWR exactly one of these access modes
os.O_CREATE create the file if it does not exist
os.O_EXCL together with O_CREATE: fail if the file already exists
os.O_APPEND every write goes to the end of the file
os.O_TRUNC truncate the file to zero length when opening

os.Create(name) is shorthand for OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0o666). Whenever you open a file, you must close it, and the idiom is to defer f.Close() right after checking the error. Here is a full tour of writing, reading, seeking and the error you get when you misuse a handle:

Go Create, write, read, seek and Stat

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Some details in that output:

  • The file holds 6 bytes (hello\n) plus 4 (n=5\n), so Size() is 10, and seeking to -4 from the end gives offset 6.
  • An *os.File keeps a position. Each Read and Write continues where the last one stopped, and Seek moves it. f.ReadAt and f.WriteAt take an explicit offset and leave the position alone.
  • info.Mode() prints like the Unix ls -l column: -rw-r--r-- is a regular file that the owner can read and write, and others can only read.
  • Errors from the os package are *fs.PathError values carrying the operation, the path and the underlying reason (here bad file descriptor). You can test them with errors.Is(err, fs.ErrNotExist), fs.ErrExist or fs.ErrPermission.

When you write a file, the error from Close matters. The operating system may delay a write failure (for example a full disk) until the file is closed. A deferred f.Close() that discards the error is fine for reading, but for writing you should return the Close error: if err := f.Close(); err != nil { return err }.

Appending, exclusive creation and safe saves

Three small variations come up constantly. Appending adds to the end of a log file, exclusive creation avoids overwriting somebody else's file, and the temp file plus rename pattern saves a file so that readers never see a half-written result:

Go Append, create exclusively, save atomically

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

os.CreateTemp(dir, "save-*.tmp") replaces the * with random digits so that concurrent programs do not collide. If the program crashes in the middle of saveAtomically, the original config.json is still intact; the reader sees either the old content or the new content, never a mixture. Passing an empty first argument (os.CreateTemp("", ...)) uses the system temp directory.

Directories

Directories have their own functions:

Function What it does
os.Mkdir(name, perm) creates one directory; parent must exist
os.MkdirAll(path, perm) creates the directory and any missing parents (no error if it exists)
os.ReadDir(name) lists a directory, sorted by file name
os.Remove(name) removes a file or an empty directory
os.RemoveAll(path) removes a path and everything below it (careful!)
os.Rename(old, new) moves or renames
os.Stat(name) / os.Lstat(name) information; Lstat does not follow symbolic links

To visit a whole tree, use filepath.WalkDir. It calls your function for every file and directory below a root, in lexical order, and you can return filepath.SkipDir to skip a directory or filepath.SkipAll to stop completely:

Go MkdirAll, ReadDir, WalkDir and Glob

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

ReadDir returns fs.DirEntry values, which already know the name and whether the entry is a directory without an extra system call, so it is cheaper than Stat on every file. Call e.Info() only when you need the size or modification time. Glob and Match use shell syntax: * matches anything except a path separator, ? one character and [a-z] a range, but there is no recursive **. For recursion use WalkDir.

Paths: path/filepath versus path

A path string is not just text. Windows separates folders with \ and has drive letters, while Linux and macOS use /. The path/filepath package handles whichever convention the program runs on, so always build file-system paths with filepath, never with + or fmt.Sprintf. The sibling package path always uses forward slashes and is for URLs and other slash-separated names that are not files on disk.

Go Joining, cleaning and splitting paths

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Facts you can read from the output:

  • filepath.Join("a", "b/../c", "/d.txt") gives a/c/d.txt: it joins, then cleans away .. and doubled slashes. An absolute-looking later element does not reset the path, unlike in Python.
  • filepath.Ext returns only the last extension (.gz, not .tar.gz).
  • filepath.Dir and filepath.Base never fail. Base of "/" is / and the directory of a bare file name is ..
  • filepath.IsLocal (added in Go 1.20) is a quick check that an untrusted relative path stays under the current directory: no .. escapes, not absolute, no reserved names on Windows.

Joining a user-supplied name onto a base directory is a classic security hole (a path traversal): filepath.Join("/srv/files", "../../etc/passwd") becomes /etc/passwd. Reject names where filepath.IsLocal is false, or better, use os.Root as shown at the end of this lesson.

Permissions

On Linux and macOS every file has a mode: three groups (owner, group, others) of three bits (read 4, write 2, execute 1). Octal notation makes this readable, and Go writes octal literals with the 0o prefix:

Octal Shown as Typical use
0o644 -rw-r--r-- normal file: owner writes, everyone reads
0o600 -rw------- private file such as a key or token
0o755 -rwxr-xr-x program or directory that everyone may enter
0o700 -rwx------ private directory

The permission you pass to Create, WriteFile and Mkdir is reduced by the process's umask (commonly 022, which removes write access for group and others), so the actual mode on disk may be tighter than what you asked for. Use os.Chmod to set a mode exactly. On Windows only the read-only bit is honored. In code, the type is fs.FileMode, and info.Mode().Perm() returns just the permission bits while Mode().IsDir() and IsRegular() test the kind of entry.

io/fs: one interface for many file systems

Reading a file from disk, from an archive and from a test fixture should not need three versions of your code. The io/fs package defines fs.FS, an interface with a single method (Open(name string) (fs.File, error)), plus helpers like fs.ReadFile, fs.ReadDir, fs.WalkDir, fs.Glob and fs.Sub. Names in an fs.FS always use forward slashes and may not start with / or contain ...

  • os.DirFS("dir") makes an fs.FS view of a real directory.
  • embed.FS holds files compiled into your program (below).
  • testing/fstest.MapFS is an in-memory file system, perfect for tests.
  • zip.Reader implements fs.FS as well, and http.FS serves any of them over HTTP.

A function that takes an fs.FS works with all of them:

Go The same code on a directory and on an in-memory file system

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The same two functions ran against memory and against disk. The test version needed no temporary files at all, which is why accepting an fs.FS instead of a directory path is a good habit for code you want to test.

Embedding files in the binary

The embed package puts files into the compiled program with a special comment, so the binary can ship templates, SQL scripts or static assets without any files next to it. This needs real files in your module, so the snippet below is read-only:

Go
package main

import (
	"embed"
	"fmt"
	"io/fs"
)

//go:embed templates/*.html
var templates embed.FS

//go:embed version.txt
var version string // a single file can be embedded as a string or []byte

func main() {
	fmt.Println("version:", version)
	entries, _ := fs.ReadDir(templates, "templates")
	for _, e := range entries {
		fmt.Println("template:", e.Name())
	}
}

The //go:embed line must be directly above a package-level variable, and the patterns are relative to the source file. Because an embed.FS is an fs.FS, everything in this section works on it too.

Confining access with os.Root

If a program opens files named by users, a name like ../../etc/passwd or a symbolic link pointing outside the folder can lead it to read files it should never touch. Go 1.24 added os.OpenRoot, which returns an *os.Root: a handle on a directory that refuses to leave it. Every operation on the root checks the path, including each symbolic link along the way:

Go os.Root blocks .. and symlink escapes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The two escape attempts fail with path escapes from parent, even though the symlink looks innocent. Plain os.ReadFile happily returns the secret. Go 1.24 gave Root the core methods (Open, Create, OpenFile, Stat, Lstat, OpenRoot, Remove, Mkdir, FS) and later releases added more. Use it whenever a path comes from a user, a network request or an archive you are extracting.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does os.Create("data.txt") do when data.txt already exists?

    Create is OpenFile with O_RDWR|O_CREATE|O_TRUNC. An existing file is emptied. Use O_EXCL to fail instead, or O_APPEND to add to the end.
  2. 2. Which is the correct way to build a file path that works on Windows and Linux?

    filepath.Join uses the separator of the running operating system and cleans the result. The path package always uses forward slashes and is meant for URLs and other non-file names.
  3. 3. What does filepath.Ext("archive.tar.gz") return?

    Ext returns the suffix starting at the final dot of the last path element, so only .gz.
  4. 4. What is the benefit of saving with a temporary file followed by os.Rename?

    If the program crashes while writing the temp file, the original is untouched. The rename on the same file system is atomic, so readers see either the old or the new content.
  5. 5. What happens when you call root.Open("../secret.txt") on an *os.Root?

    os.Root rejects any path that would leave its directory, including through .. elements and symbolic links, with a 'path escapes from parent' error.

Summary

  • os.Open reads, os.Create creates or truncates, and os.OpenFile takes flags (O_APPEND, O_EXCL, O_TRUNC, ...) and permissions. Close what you open; for writes, check the Close error.
  • Files have a position that Read, Write and Seek move. Errors are *fs.PathError; test them with errors.Is(err, fs.ErrNotExist) and friends.
  • Save important files by writing a temp file in the same directory and renaming it over the target.
  • os.MkdirAll, os.ReadDir (sorted, cheap), filepath.WalkDir (with SkipDir), filepath.Glob, os.Remove, os.RemoveAll and os.Rename manage directories.
  • Build file paths with path/filepath (Join, Clean, Base, Dir, Ext, Rel, IsLocal); the path package is for slash-separated names like URLs.
  • Permissions are octal modes such as 0o644 and 0o755, reduced by the umask.
  • io/fs lets one function read a directory (os.DirFS), embedded files (embed.FS) or an in-memory fstest.MapFS. Prefer accepting fs.FS.
  • os.Root (Go 1.24) confines file access to a directory and blocks .. and symlink escapes, which protects against path traversal.

Next: JSON shows how to turn Go values into JSON text and back.