/ Daniel Grady

Configuring your shell

Fiddling with shell dot files is a great way to spend a few hours and learn more about how these utilities work. One of the most confusing things for me was understanding exactly what happens when a shell starts up, what files it reads for startup settings and commands, and where it's appropriate to put different kinds of configuration, and these notes summarize what I learned while looking in to those questions.

Shells?

A shell is a command-line interface to a computer's operating system. Shells are different interfaces than the more familiar graphical interfaces; GUIs do not (necessarily) "run on top" of a shell, and a CLI and GUI can (and often are) both used on the same system for different kinds of tasks. Today it appears to be most common to run a shell within a graphical interface using a terminal emulation program.

I only discuss Bash and zsh here; these are the shells I've had experience with. Most operating systems today use Bash as the default shell, but come bundled with several alternatives that usually include sh, csh, ksh, tcsh, and zsh. The history of these shells is an intricate and undoubtedly fascinating subject. I am going to caricature this rich history with an inaccurate summarization:

There used to be lots of different shells in widespread use, but the Bourne shell (sh, released in 1977) achieved critical mass and became a de facto standard. The GNU Project needed an open-source shell for their operating system and wrote the Bourne-again shell (bash, released 1989), which took ideas not only from the Bourne shell but all its competitors as well. Bash has since become an even more firmly entrenched de facto standard than any other shell ever. It's the default shell on almost every POSIX system, and in fact many systems no longer include a stand-alone version of sh; running sh actually runs bash in Bourne-shell compatibility mode. The Z Shell (zsh, released 1990) is yet another shell that's slightly newer, with possibly slightly fancier features than bash, in some situations.

All of these shells (sh, csh, ksh, bash, zsh, and many others) are in the same family; it's the family of shells that led to the official standardization of a POSIX-compliant shell. Consequently, they're all pretty similar in many important ways. Bash and zsh have definitely subsumed all the good ideas from all the others, and definitely have the majority of mind share and development effort behind them today. There have been other efforts at designing different kinds of shells; the friendly interactive shell (fish, released 2005) is an example. It's hard for me to tell if fish is gaining popularity, and it's not included with an OS distributions I'm aware of at this time.

Shells have historically tried to solve two problems: providing an interface for interactive computer use, and also providing a programming language for writing scripts. Everything available today that calls itself a command shell has a syntax that's more or less inspired by the Bourne shell. Although Bash, zsh, and fish have over time done many things to improve the programming language part of these shells, many computing professionals (and I'm one of them) definitely recommend that you use Python, Ruby, etc for writing programs (even simple programs), and confine the shell to interactive work.

The two takeaway points, for me, are 1) pick one shell and stick with, and 2) that shell should be Bash or zsh.

What a shell does when it starts

Figuring out why your shell's environment looks the way it does is almost always a matter of figuring out what files your shell is sourcing at startup, and that's affected by whether you're using Bash or zsh and also by what mode your shell is invoked with.

A shell can be invoked in four different ways

Shells have several different modes of operation that affect what files they read on startup and also they way they behave. The two most important modes are interactive versus non-interactive, and login versus non-login. Both Bash and zsh allow you to invoke them in one of four different ways, depending on how those options are set. You're normally dealing with interactive shells; anytime you're typing commands in a shell running in a terminal, it's an interactive shell. You might also use non-interactive shells, for example to run a script; if you've ever run a script by typing bash ./my-script.sh, the shell that executed that script was a non-interactive, non-login shell. A login shell is created for you by the system every time you make a new ssh connection, or every time you log in through a terminal on some Linux distributions. Shells that you subsequently create, for example by typing bash inside a login shell, are non-login shells. One of the most important differences between these different operating modes is that the shell will source different configuration files on startup depending on what mode it's in.

Further details are provided in man bash:

A login shell is one whose first character of argument zero is a -, or one started with the --login option.

An interactive shell is one started without non-option arguments and without the -c option whose standard input and error are both connected to terminals (as determined by isatty(3)), or one started with the -i option. PS1 is set and $- includes i if bash is interactive, allowing a shell script or a startup file to test this state.

Some other helpful notes about these differences:

The files that a shell sources at startup

Bash and zsh use different startup files, and they also use different subsets of their startup files depending on how they're invoked.

For Bash, if it's a login shell, it will will first always source /etc/profile and then source the first (and only the first) of ~/.bash_profile, ~/.bash_login, or ~/.profile that it finds. If it's not a login shell but it is interactive, it sources ~/.bashrc only — Bash does not source any system-wide configuration in this case. If the shell it non-interactive and non-login, then Bash will try to expand the environment variable BASH_ENV and source the file it points to.

zsh's scheme appears more complicated, but is actually a little easier to manage in practice. zsh uses four pairs of configuration files (so eight files total); we can call the pairs environment, profile, resource, and login. Each pair of files includes a global, system-wide file and a user-specific file that lives in your home directory. The profile and login pairs are redundant, and the zsh documentation recommends that you only use one, and I'll assume that's profile. Each pair is intended to cover a particular case: the environment pair covers configuration settings that should apply to every single instance of zsh, no matter what mode it's operating in; the profile pair is supposed to include login-only configuration; and the resource pair is supposed to include only configuration that's used in interactive sessions. Every time zsh is invoked, it iterates through these eight configuration files (or a subset of them) in a predictable order.

This table summarizes the different startup configuration files that Bash and zsh use.

Interactive
login		 Non-interactive
login		 Interactive
non-login		 Non-interactive
non-login
Bash
/etc/profile x x
~/.bash_profile, ~/.bash_login, ~/.profile x x
~/.bashrc x
BASH_ENV x
zsh
/etc/zshenv x x x x
$ZDOTDIR/.zshenv x x x x
/etc/zprofile x x
$ZDOTDIR/.zprofile x x
/etc/zshrc x x
$ZDOTDIR/.zshrc x x
/etc/zlogin x x
$ZDOTDIR/.zlogin x x

Some other notes about this:

What should go where?

So, given this explanation of when and which files Bash and zsh use when they start up, what should we put in these files to maximally leverage our shells? Opinions differ, and here are mine.

The main guiding principle is to keep things simple. Beyond that, another good guiding principle is to try and separate environment configuration from interactive configuration. Things that affect your environment (for example, setting environment variables like PATH) should go into login-shell-specific configuration files, rather than interactive-shell-specific configuration files. The reason for this is that you would like to retain as much control as possible over the environment of the child processes you launch from your interactive login shell. For example, if you ensure that any commands which alter your PATH stay in login-specific config files, then you can experiment with other versions of programs or interpreters by launching a child shell with the path to those programs prepended, for example

PATH="./bin/fancy-new-thing:$PATH" bash

will launch a new shell that uses any binary in that folder by default. However, if ~/.bashrc contains commands that alter PATH, this won't work reliably. This sort of situation can affect shells started within other programs as well, like Emacs. In general, try to configure your environment a single time, once, for the login shell, and allow any further child shells to inherit whatever environment is current when they're invoked.

If you agree with this principle, these recommendations follow naturally from it:

Finally, I'll end with a suggestion for debugging your shell configuration. If you're trying to figure out why your PATH or some other part of your shell isn't set the way you expect, a good place to start are the system configuration files. It would be nice if distribution-provided configuration files were always useful examples of best practices, but unfortunately it's not the case.

An unstructured list of other useful references