Yaniv Aknin (whose blog I found through Planet Python) recently blogged about his ZSH and virtualenv setup. As I was reading it, I recognized that he's solving some of the same problems I only recently solved myself (though through a very different approach than he took), so I figured I would share it, too.

What is virtualenv?

virtualenv is, in my opinion, really the only way to do sane Python development. As its name implies, virtualenv creates a virtual Python environment, and in so doing solves a number of problems quite elegantly:

1. You are working on more than one project, each of which requires some of the same dependencies, but at different versions. You aren't using distribute's pkg_resources, so you can't know that the right version will be used by the right project (nor even which version will be used when imported).
2. Alternately, you are working on more than one project, each of which requires different dependencies, but your dependencies are poorly-behaved and trample one another's namespaces.
3. You want to be sure that your development and deployment environments are as identical as possible with regards to package makeup.
4. You have grown tired of typing sudo pip install ... to install packages.
5. You have grown tired of errors about PYTHON_EGG_CACHE when using zipped eggs (though I would be remiss if I didn't note: don't use zipped eggs, they are a stupid and awful idea, which could be the subject of a different blog post)

virtualenv solves all of these in one fell swoop by creating what is essentially a clone of your standard python installation into a new location, optionally including any system-installed site-packages, configures a few environment variables, and sets everything to be owned and usable by your unprivileged user.

After creating the virtual environment, you then activate it by sourcing the bin/activate script inside the virtual environment. Now when you type python at your command prompt, you are activating the virtualenv's Python executable rather than the system executable, and all Python scripts you run (importantly, pip) will use this executable as well. Your virtual environment has its own site-packages directory, and package installs done while the environment is activated will install into the virtual environment as well.

When you're done, simply type deactivate (or close your shell) to undo everything that the activate script did.

And what about virtualenvwrapper?

virtualenv isn't without its deficiencies, though. It's up to you to manage your own virtual environments and to remember where each one's activate script lies.

Many people adopt the convention that you create a directory named env or .env containing the virtual environment within the project root folder of the project for which the environment is meant. However, this clutters your checkout root, and you might mistakenly check your virtual environment in to source control.

Alternately you can have a centralized location for all your environments, but then you have to remember two paths for each project, rather than one, and you have to remember it each time you want to open a new shell or terminal tab for a given project.

virtualenvwrapper is a well-thought-out set of shell function wrappers for virtualenv which make life using it much better.

virtualenvwrapper stores all your environments in one place (by default in ~/.virtualenvs, or wherever the $WORKON_HOME environment variable points), but alleviates the pain of that approach by providing a suite of tools to manipulate and interrogate the virtual environments you have.

There is mkvirtualenv which supersedes the virtualenv command, and its new counterpart rmvirtualenv; lsvirtualenv which lists all the virtualenvwrapper-managed environments; cdvirtualenv and cdsitepackages which change directory to the virtual environment's root or site-packages directories, respectively; and my favorite, workon, which activates an environment by name.

But I want more!

virtualenvwrapper also has a full suite of hook scripts (both "global" hooks that are used regardless of which environment you are working on, and environment-specific versions) to customize aspects of its behavior. I'll mention two here which I've found incredibly useful:

First is the postmkvirtualenv hook, which runs after any invocation of mkvirtualenv. I have mine set up to create a directory named after the virtual envrionment as a subdirectory of the current working directory, if it doesn't already exist, and then cd into it:

# ~/.virtualenvs/postmkvirtualenv
parent="$PWD"
envname="$(basename $VIRTUAL_ENV)"
envdir="$parent/$envname"

if [ ! -e $envdir ]; then
    mkdir -p $envdir
fi

cd $envdir

The $VIRTUAL_ENV variable points to the directory of the newly-created virtual environment, something like ~/.virtualenvs/foobar, and is set because before the postmkvirtualenv hook is run, virtualenvwrapper has already activated the new environment for us.

Secondly, I use the postmkvirtualenv hook to write a postactivate hook into the newly created virtual environment, so that on subsequent activations with workon, I also get this "cd to the right place" behavior:

# also in ~/.virtualenvs/postmkvirtualenv
postactivate=$VIRTUAL_ENV/bin/postactivate

echo "cd $envdir"             >> $postactivate

With these two hooks configured, I can now forget about all paths, either to the virtual environment itself, or to my project's source root; all I need to remember is the name I used when I created it, and type workon that_name, and I'll be brought to the right place. And if I happen to forget the name, I've always got lsvirtualenv to help me remember.

My virtualenvwrapper hooks (as well as my ZSH setup, vim configuration, and a few other bits of random customization) are in my dotfiles repository -- feel free to fork it if you think it'll be useful for you!