Skip to content
Tutorials Center
My learning

Topics

Lesson 89 of 147 ยท Standard Library

Running Commands with os/exec

Learn how to run external programs from Go with os/exec - capture output, check exit codes, feed stdin, build pipelines, stream output and stop commands with context timeouts.

Sometimes the quickest way to get something done is to call another program: compress a file with tar, ask git for the current commit, convert an image, or run a database dump. The os/exec package lets a Go program start a child process, give it input, collect its output and find out how it ended.

Go never runs your command through a shell unless you ask for one. You give exec.Command a program name and a list of arguments, and each argument reaches the program exactly as written. That makes os/exec much safer than building a command string by hand. The examples below use common Unix tools such as echo, sort and sh, so they run on the lesson runner (Linux) and on macOS; on Windows you would call different programs.

The lifecycle of a command

exec.Command(name, args...) only describes a command and returns a *exec.Cmd. Nothing runs until you call one of its run methods. After that, the process goes through a few stages:

stateDiagram-v2
    [*] --> Created: exec.Command
    Created --> Running: Start
    Running --> Exited: process ends
    Exited --> Finished: Wait
    Finished --> [*]

Run is Start followed by Wait; Output and CombinedOutput also wrap both steps.

The most common shortcuts are:

Method What it does
Run() Starts the command and waits for it to finish.
Output() Runs it and returns what it wrote to standard output.
CombinedOutput() Runs it and returns standard output and standard error mixed together.
Start() + Wait() Starts it, lets you do other things, then waits.

Capturing output

Output returns the bytes the program printed. If the program exits with a non-zero status, the returned error has the type *exec.ExitError:

Go Output, exit codes and missing programs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Notice the three different failure kinds. A program that is not found fails before it starts (exec.ErrNotFound, which you can test with errors.Is). A program that runs but fails gives an *exec.ExitError with the exit code. A program killed by a signal also gives an ExitError, with ExitCode() equal to -1. Learn more about matching errors in Wrapping Errors.

Standard output versus standard error

Output keeps standard error out of its result, but when the command fails it stores the first part of stderr in ExitError.Stderr. To capture both streams separately, point Stdout and Stderr at buffers (see Bytes and Buffers):

Go Separate stdout and stderr buffers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

After a command finishes, cmd.ProcessState holds details such as the exit code and CPU time. Setting cmd.Stdout = os.Stdout instead of a buffer connects the child directly to your terminal, which is handy for tools whose output you simply want the user to see.

Giving a command input

Set cmd.Stdin to any io.Reader. A strings.NewReader works well for short text:

Go Feeding standard input

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

If you are not sure that the program name or its arguments are trustworthy, never paste them into a sh -c string. Pass user input as a separate argument instead: exec.Command("grep", "-n", userText, "file.txt") cannot be tricked into running a second command, whereas sh -c "grep " + userText can.

Pipes: talking to a running process

For long-running programs you often want to write input and read output while the process is alive. StdinPipe, StdoutPipe and StderrPipe give you that. Call them before Start, and read all output before Wait:

Go Streaming output line by line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Two rules keep pipes from hanging. First, finish reading before calling Wait, because Wait closes the pipe. Second, if you use a stdin pipe, close it when you have sent everything, otherwise programs like sort wait for more input forever. See Reading and Writing with os, io and bufio for the scanner and reader tools.

Connecting two commands

A shell pipeline a | b is built by hand: connect the first command's StdoutPipe to the second command's Stdin.

Go A pipeline of two commands

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Timeouts and cancellation with context

A child process can hang. exec.CommandContext ties the process to a context: when the context is cancelled or its deadline passes, Go kills the process.

Go Killing a slow command after a timeout

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Check ctx.Err() to tell a timeout apart from a normal failure, because the error from Run just says signal: killed. By default Go sends SIGKILL, which gives the program no chance to clean up. Since Go 1.20 you can set cmd.Cancel to send a gentler signal first (such as SIGTERM) and cmd.WaitDelay to set how long Go waits before forcing the kill. WaitDelay also covers a common trap: a child that spawned its own children keeps the output pipe open, and without a delay Wait would block until those grandchildren exit.

Environment, working directory and safety

  • cmd.Dir sets the directory the command runs in.
  • cmd.Env is the full environment as KEY=value strings. If it is nil, the child inherits yours. To add one variable, use append(os.Environ(), "MODE=test").
  • exec.LookPath finds a program in PATH. Go deliberately refuses to run a program found in the current directory via PATH lookup (you get exec.ErrDot), which protects against planted binaries.
  • Always treat the exit code as part of the result. Many tools use non-zero codes for information, not only for crashes (grep returns 1 for "no match").
  • Prefer a Go library when one exists. Starting a process is slower, depends on what is installed on the machine and makes testing harder.

The sandbox that runs the examples on this site has a restricted set of programs. In your own environment, check that the tools you call are installed, and handle exec.ErrNotFound with a clear message.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does exec.Command("ls", "-l") do by itself?

    Command builds the description. The process starts only when you call a method such as Run, Start, Output or CombinedOutput. No shell is involved.
  2. 2. A command exits with status 1. What does cmd.Run() return?

    A non-zero exit status is reported as an *exec.ExitError. ErrNotFound is for programs that cannot be found at all.
  3. 3. Why must you read all output from StdoutPipe before calling Wait?

    The documentation requires reads to finish before Wait, because Wait closes the pipe once the process exits.
  4. 4. How can you stop a command that runs too long?

    CommandContext kills the process when the context is cancelled or its deadline passes. Check ctx.Err() to see why it stopped.
  5. 5. Which is the safer way to pass a user-supplied file name to grep?

    Separate arguments go straight to the program without shell parsing, so special characters in the name cannot start another command.

Summary

  • exec.Command describes a process; Run, Output, CombinedOutput or Start plus Wait actually run it.
  • Failures come in three kinds: program not found, non-zero exit (*exec.ExitError), and being killed by a signal.
  • Set Stdin, Stdout and Stderr to readers, writers or buffers, or use StdinPipe/StdoutPipe to stream while the process runs.
  • Read everything from a pipe before Wait, and close stdin pipes when you are done sending.
  • exec.CommandContext adds timeouts and cancellation; Cancel and WaitDelay fine-tune how the process is stopped.
  • Pass arguments separately instead of building shell strings, and use Dir and Env to control where and how the child runs.

Next: container/heap, list and ring covers priority queues and linked lists from the standard library.