Skip to content
Tutorials Center
My learning

Topics

Lesson 87 of 134 · Standard Library

StringIO and Streams

Learn StringIO in Ruby - read and write a string like a file, rewind and seek, capture $stdout and $stderr, fake STDIN for tests, inject input and output streams, and avoid EOFError and IOError.

A great deal of Ruby code reads from and writes to streams: File, $stdin, $stdout, sockets, pipes. A stream is a source or destination of data with a current position that moves as you read or write. The methods on streams (gets, read, each_line, puts, write, print, <<) are the same everywhere, which is a lovely property: code that works with "something that behaves like an IO" does not care what is behind it.

StringIO (from the standard library, require 'stringio') is an in-memory stream backed by a string. It looks and behaves like a file, but there is no disk, no file name and nothing to clean up. That makes it perfect for three jobs: testing code that reads or writes streams, capturing output that would otherwise go to the screen, and building text piece by piece. This lesson builds on Reading Input and Formatting Output and Files and IO.

Reading from a StringIO

Create one with the text it should contain. It starts at position 0, just like a file you have just opened.

Ruby gets, read, pos and rewind

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The important idea is the position. Every read moves it forward and the next read continues from there. Once you reach the end, read returns "", read(n) and gets return nil, and eof? is true. To read the same data again you must rewind or seek. Forgetting that is the number one StringIO mistake: you write something, read it back, and get nothing, because the position is at the end.

each_line, each_char, each_byte and readlines work as on files, and chomp: true removes the line endings. gets has variants too: gets(" ") reads up to a separator, gets(nil) reads everything.

The "read" family has a strict sibling, readline, which raises when there is nothing left instead of returning nil:

Ruby EOFError (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

After printing only line, the run ends with main.rb:5:in 'IO::generic_readable#readline': end of file reached (EOFError). Use gets in loops (while (line = io.gets)) because it returns nil at the end and readline when a missing line is a bug.

Writing to a StringIO

A StringIO with no argument starts empty and accepts the writing methods you know from $stdout (see Reading Input and Formatting Output). The accumulated text is available from string.

Ruby Building text with puts, print, printf and <<

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

out.string returns the real underlying string, not a copy, so you can inspect it at any time, even while you keep writing. After out.rewind, new writes overwrite from the start ("OVER" replaced the first four characters) instead of inserting. To start over, truncate(0) and rewind as above, or just create a new StringIO.

Closed and read-only streams

A StringIO built from a frozen string is read-only, and a closed one is unusable. Both raise IOError with a message that says why:

Ruby IOError from the wrong kind of stream

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

IOError is a StandardError and is what real files raise in the same situations (closed stream, not opened for writing), so code that handles closed streams can be tested without touching a disk.

Capturing output

puts, print and p write to $stdout, a global variable that holds the current output stream. If you replace it with a StringIO, everything printed lands in the string instead of the screen. The same holds for $stderr, which warn uses, and for $stdin, which gets reads from.

Ruby Capturing and restoring $stdout and $stderr

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

STDOUT (uppercase) is the constant for the real process output and never changes; $stdout (lowercase global) is the current output and can be reassigned. Always restore it, preferably with ensure, otherwise every later puts in the program silently vanishes. The helper capture_stdout shows the pattern, and Minitest ships a similar capture_io for tests (see Testing with Minitest).

Reassigning $stdout changes a process-wide global, which is risky with threads: output from other threads goes into your buffer too. Use it in tests and small tools, and prefer the injection style below in application code.

Faking input

The same trick works for input. Put the text a user would type into a StringIO and assign it to $stdin; then gets reads from it:

Ruby A fake $stdin

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

(Our runner can also preset real standard input, as other lessons do with input="...", but assigning $stdin works everywhere and lets one script feed several different inputs.)

Dependency injection: the clean way

Replacing globals is a workaround. A better design makes the streams parameters, with the real ones as defaults. The function then works with files, sockets, terminals and StringIOs alike, and needs no global tricks to test:

Ruby Injecting input and output streams

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

word_frequencies takes any object with each_line. In production you hand it File.open("book.txt") or $stdin; in a test you hand it a StringIO with a couple of lines, and the expected result is right there in the test. This duck typing (see Duck Typing and Polymorphism) is the entire reason StringIO exists. Notice input.gets&.chomp: gets returns nil at end of input, and the safe-navigation operator (see Ternary, Safe Navigation and Short-Circuit) avoids a NoMethodError on nil.

StringIO as a data source

Libraries that accept "a file or something like it" accept StringIO too: JSON.parse(io.read), YAML.load(io), CSV.new(io), loggers writing into a StringIO (see OptionParser and Logger). Here it plays a file of JSON lines:

Ruby JSON lines through a StringIO

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Bytes, characters and encodings

A StringIO can be read in characters or in bytes. getc and each_char give characters, while read(n) counts bytes and returns a binary string. If you read part of a multi-byte character, you get half a character, so use getc, gets and each_line for text:

Ruby read(n) counts bytes

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

read(2) returned "h\xC3": the letter h and the first of the two bytes of é, labelled as binary. This is the same byte-versus-character distinction as in Digest, Base64 and Encoding.

StringIO versus alternatives

  • versus String#<<. Building text by appending to a plain string works, and is simpler for small cases. StringIO wins when the code you call expects an IO (puts, printf, loggers, report methods) and when you want to read back what was written.
  • versus Tempfile. A Tempfile is a real file on disk with a path, needed when some other program or a library that requires a file name has to open it. StringIO has no path (fileno is nil).
  • limits. The whole content lives in memory, so StringIO suits small and medium data. Stream large inputs from a real file instead.
  • real IO details are missing. A StringIO is not a TTY (tty? is false).

Check your understanding

Quick check · 5 questions

of 5 answered You scored / 5 · perfect!

  1. 1. You write to a StringIO and then call io.read, but it returns an empty string. What is the most likely cause?

    Writing moves the position to the end, and read continues from the current position. rewind (or seek(0)) returns to the start.
  2. 2. What is the difference between gets and readline at the end of the stream?

    gets signals the end with nil, which suits while loops. readline treats a missing line as an error and raises EOFError (end of file reached).
  3. 3. Why should you restore $stdout inside an ensure clause after replacing it with a StringIO?

    If an exception skips the restoring line, every later puts keeps writing into the buffer. ensure runs on both normal and exceptional exits.
  4. 4. What is the main design advantage of a method like def report(records, io = $stdout)?

    Because the method only relies on io.puts, any IO-like object works. Tests pass a StringIO and compare its string.
  5. 5. What does StringIO.new("héllo").read(2) return?

    read(n) counts bytes. é uses two bytes in UTF-8, so two bytes cover h and half of é. Use getc or each_char to read whole characters.

Summary

  • StringIO is an in-memory stream with the same methods as files: gets, read, each_line, puts, print, write, <<, rewind, seek, eof?.
  • It has a position: reads and writes move it, so rewind before reading what you wrote; writes overwrite from the current position.
  • gets returns nil at the end, readline raises EOFError; closed and read-only streams raise IOError.
  • Capture output by assigning a StringIO to $stdout or $stderr, fake input with $stdin, and always restore the originals in ensure.
  • Prefer passing streams as parameters (input:, output:) over swapping globals; the code stays reusable and easy to test.
  • read(n) counts bytes; use getc, gets and each_line for text.

Next: Forwardable and Delegation