Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 40 ยท Errors and I/O

Files and I/O

Read and write files in Julia: open with do blocks, read and readlines, append mode, IOBuffer, path helpers, and CSV files through packages.

Almost every real program reads something and writes something: a configuration file, a log, a table of measurements. Julia's input and output system is small and consistent. Everything that can be read from or written to, whether a file, the terminal, a network socket or a block of memory, is an IO object, and the same functions (print, println, write, read, readline) work on all of them.

In this lesson you will open files the safe way with a do block, read whole files and single lines, write and append, use IOBuffer as an in-memory file, build paths without string tricks, and read tables with the CSV package. Julia is not installed on this site, so the Julia code is read-only. The two runnable examples are Python stand-ins that use the same ideas, and each one says which Julia call it mirrors.

The shortest ways to read and write

For small files, two one-line calls cover most needs. Both open the file, do the job and close it for you:

JULIA
write("notes.txt", "first line\nsecond line\n")   # returns 23, the number of bytes written

text  = read("notes.txt", String)    # the whole file as one String
lines = readlines("notes.txt")       # a Vector{String}, one entry per line

println(lines)    # ["first line", "second line"]

write(path, data) creates the file, or replaces it if it already exists. read(path) without a type returns the raw bytes as a Vector{UInt8}, and read(path, String) decodes them as text. readlines removes the newline at the end of each line unless you pass keep=true.

Pick the right reader

read(path, String) The whole file as one string.
readlines(path) A vector with one string per line.
eachline(path) Lines one at a time, for huge files.
read(path) Raw bytes, for binary data.

All of them close the file for you.

1 / 2

Opening files with a do block

When you need more than one operation, or want to process a file line by line, use open. The do form is the idiom to learn first:

JULIA
open("log.txt", "w") do io
    println(io, "started")
    println(io, "working")
    print(io, "done at ", 42, "\n")
end

The block receives the open file as io. When the block ends, even because an error was thrown inside it, Julia closes the file. This matters because an unclosed file may keep unwritten data in a buffer and may stay locked on Windows. If you call open(path) without a block you get an IOStream back and you must call close(io) yourself, ideally inside a try/finally. The do block does that for you, which is why most Julia code uses it.

The second argument is the mode:

Mode Meaning
"r" Read only. This is the default. The file must exist.
"w" Write. Creates the file or truncates an existing one.
"a" Append. Creates the file if needed and writes at the end.
"r+" Read and write without truncating.
"w+" Read and write, truncating first.

Notice that you write to a file with the same println you use for the screen. println(io, x) prints to io, and println(x) is shorthand for println(stdout, x).

Reading line by line

readlines loads everything into memory. For a file of several gigabytes, loop over eachline instead. It reads one line at a time:

JULIA
open("measurements.txt") do io
    total = 0.0
    count = 0
    for line in eachline(io)
        isempty(strip(line)) && continue
        total += parse(Float64, line)
        count += 1
    end
    println("average: ", total / count)
end

The file is closed when the block ends, and eachline(io) yields each line without its newline. You can also call readline(io) yourself to get one line and eof(io) to ask whether you are at the end:

JULIA
open("measurements.txt") do io
    header = readline(io)          # first line only
    while !eof(io)
        value = parse(Float64, readline(io))
        println(value)
    end
end

Reading from the keyboard works the same way, because standard input is also an IO object: readline() reads one line from stdin, and parse(Int, readline()) turns it into a number.

Appending, and writing text versus bytes

Use mode "a" to add to a log without erasing what is already there:

JULIA
open("log.txt", "a") do io
    println(io, "restarted")
end

There is a small difference between print-style functions and write. print and println turn values into text, so print(io, 42) writes the two characters 4 and 2. write(io, 42) writes the integer in its binary form, eight bytes for an Int64. Use print for text files and write for binary files, where you control every byte. write returns the number of bytes written, which is why the REPL prints a number after it.

IOBuffer: a file in memory

An IOBuffer is an IO object that lives in memory instead of on disk. It is useful when you build a large string piece by piece, or when you want to test a function that takes an IO argument without touching the file system:

JULIA
io = IOBuffer()
for i in 1:3
    println(io, "row ", i)
end

s = String(take!(io))    # empties the buffer and returns its contents
print(s)
# row 1
# row 2
# row 3

take! returns the bytes collected so far and resets the buffer. Wrapping that in String(...) converts them to text. You can also go the other way and create a buffer that reads from a string, then use the normal reading functions:

JULIA
data = IOBuffer("10\n20\n30\n")
nums = [parse(Int, line) for line in eachline(data)]
sum(nums)    # 60

Writing functions that accept an io::IO argument instead of a file name makes them easy to test and reuse. The same function can write to a file, to stdout or to an IOBuffer.

Paths, directories and the working directory

Do not build paths by gluing strings with / or \. Use the helpers, which use the right separator on each operating system:

JULIA
path = joinpath("data", "2024", "sales.csv")   # "data/2024/sales.csv" on Linux and macOS

basename(path)        # "sales.csv"
dirname(path)         # "data/2024"
splitext("sales.csv") # ("sales", ".csv")

isfile(path)          # does a regular file exist there?
isdir("data")         # does a directory exist there?
mkpath("data/2024")   # create it and any missing parents
pwd()                 # current working directory
readdir("data")       # names of the entries in a directory
rm("old.txt"; force=true)    # delete a file, and no error if it is missing

Two more helpers are worth knowing. @__DIR__ expands to the folder of the file being run, so joinpath(@__DIR__, "data.csv") finds a data file next to your script no matter where you started Julia. And mktempdir() do dir ... end gives you a temporary folder that is deleted afterwards, which is ideal for tests.

Tables and CSV files

Julia's standard library includes DelimitedFiles for plain numeric tables, but most people use the CSV package, which is fast and understands headers, types and missing values. It works hand in hand with a table type such as a DataFrame (see DataFrames.jl):

JULIA
using CSV, DataFrames

df = CSV.read("sales.csv", DataFrame)      # columns get their types from the data
CSV.write("summary.csv", df)               # write a table back out

Without a table type, CSV.File("sales.csv") and CSV.Rows give lazy access, and each row can be read as a named tuple. Install packages with ] add CSV DataFrames in the package REPL (see Packages and Environments).

Text encoding is UTF-8 everywhere in Julia. readlines and read(path, String) assume it, and so do print and write for strings.

Runnable stand-ins in Python

Python's file API matches Julia's closely: a with block plays the role of do, and the modes "w", "a" and "r" are the same. This example writes a file, appends to it, then reads it back in three ways. The browser gives each page a small virtual file system, so it works here.

Python Write, append and read back (Julia: open ... do)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The next example mirrors IOBuffer. Python's io.StringIO is a text file that lives in memory, and a function that takes any file-like object works with both a real file and the buffer, exactly as an io::IO argument does in Julia.

Python An in-memory file (Julia: IOBuffer)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is open(path, "w") do io ... end preferred over calling open and close by hand?

    The do block closes the file when it ends, including when an exception is thrown, so buffered data is flushed and the file is released.
  2. 2. Which mode adds new lines to the end of an existing file without erasing it?

    Append mode "a" writes at the end. Mode "w" truncates the file first, which would erase the old contents.
  3. 3. What does String(take!(io)) do for an IOBuffer?

    take! returns the bytes written so far and resets the buffer; String converts them into text.
  4. 4. What is the best way to read a multi-gigabyte log file?

    eachline reads one line at a time, so memory use stays small. The other readers load the entire file.

Summary

  • Files, the terminal and memory buffers are all IO objects, so print, println, write and readline work on each of them.
  • read(path, String), readlines(path) and write(path, data) handle small files in one call.
  • open(path, mode) do io ... end closes the file for you; use "r", "w" and "a" for read, write and append.
  • eachline streams large files, and IOBuffer is an in-memory file for building or testing text.
  • Use joinpath, basename, isfile and mkpath for paths, and the CSV package with DataFrames for tables.

Next: turn those tables into analysis with DataFrames.jl.