Yaniv Aknin (whose blog I found through Planet Python) recently blogged about his ZSH and virtualenv setup. As I was reading it, I recognized that he's solving some of the same problems I only recently solved myself (though through a very different approach than he took), so I figured I would share it, too.
virtualenv is, in my opinion, really the only way to do sane Python development. As its name implies,
virtualenv creates a virtual Python environment, and in so doing solves a number of problems quite elegantly:
- You are working on more than one project, each of which requires some of the same dependencies, but at different versions. You aren't using distribute's
pkg_resources, so you can't know that the right version will be used by the right project (nor even which version will be used when imported).
- Alternately, you are working on more than one project, each of which requires different dependencies, but your dependencies are poorly-behaved and trample one another's namespaces.
- You want to be sure that your development and deployment environments are as identical as possible with regards to package makeup.
- You have grown tired of typing
sudo pip install ...to install packages.
- You have grown tired of errors about
PYTHON_EGG_CACHEwhen using zipped eggs (though I would be remiss if I didn't note: don't use zipped eggs, they are a stupid and awful idea, which could be the subject of a different blog post)
virtualenv solves all of these in one fell swoop by creating what is essentially a clone of your standard python installation into a new location, optionally including any system-installed site-packages, configures a few environment variables, and sets everything to be owned and usable by your unprivileged user.
After creating the virtual environment, you then activate it by sourcing the
bin/activate script inside the virtual environment. Now when you type
python at your command prompt, you are activating the
virtualenv's Python executable rather than the system executable, and all Python scripts you run (importantly,
pip) will use this executable as well. Your virtual environment has its own
site-packages directory, and package installs done while the environment is activated will install into the virtual environment as well.
When you're done, simply type
deactivate (or close your shell) to undo everything that the
activate script did.
And what about
virtualenv isn't without its deficiencies, though. It's up to you to manage your own virtual environments and to remember where each one's
activate script lies.
Many people adopt the convention that you create a directory named
.env containing the virtual environment within the project root folder of the project for which the environment is meant. However, this clutters your checkout root, and you might mistakenly check your virtual environment in to source control.
Alternately you can have a centralized location for all your environments, but then you have to remember two paths for each project, rather than one, and you have to remember it each time you want to open a new shell or terminal tab for a given project.
virtualenvwrapper is a well-thought-out set of shell function wrappers for
virtualenv which make life using it much better.
virtualenvwrapper stores all your environments in one place (by default in
~/.virtualenvs, or wherever the
$WORKON_HOME environment variable points), but alleviates the pain of that approach by providing a suite of tools to manipulate and interrogate the virtual environments you have.
mkvirtualenv which supersedes the
virtualenv command, and its new counterpart
lsvirtualenv which lists all the
cdsitepackages which change directory to the virtual environment's root or site-packages directories, respectively; and my favorite,
workon, which activates an environment by name.
But I want more!
virtualenvwrapper also has a full suite of hook scripts (both "global" hooks that are used regardless of which environment you are working on, and environment-specific versions) to customize aspects of its behavior. I'll mention two here which I've found incredibly useful:
First is the
postmkvirtualenv hook, which runs after any invocation of
mkvirtualenv. I have mine set up to create a directory named after the virtual envrionment as a subdirectory of the current working directory, if it doesn't already exist, and then
cd into it:
# ~/.virtualenvs/postmkvirtualenv parent="$PWD" envname="$(basename $VIRTUAL_ENV)" envdir="$parent/$envname" if [ ! -e $envdir ]; then mkdir -p $envdir fi cd $envdir
$VIRTUAL_ENV variable points to the directory of the newly-created virtual environment, something like
~/.virtualenvs/foobar, and is set because before the
postmkvirtualenv hook is run,
virtualenvwrapper has already activated the new environment for us.
Secondly, I use the
postmkvirtualenv hook to write a
postactivate hook into the newly created virtual environment, so that on subsequent activations with
workon, I also get this "
cd to the right place" behavior:
# also in ~/.virtualenvs/postmkvirtualenv postactivate=$VIRTUAL_ENV/bin/postactivate echo "cd $envdir" >> $postactivate
With these two hooks configured, I can now forget about all paths, either to the virtual environment itself, or to my project's source root; all I need to remember is the name I used when I created it, and type
workon that_name, and I'll be brought to the right place. And if I happen to forget the name, I've always got
lsvirtualenv to help me remember.
virtualenvwrapper hooks (as well as my ZSH setup, vim configuration, and a few other bits of random customization) are in my dotfiles repository -- feel free to fork it if you think it'll be useful for you!