Today I managed to get myself a little bit stuck with Heroku's otherwise seamless Python support. In an earlier revision of my app, I had used -e to install an editable version of a package (the version of the package I needed had not yet been released to PyPI, so I pointed pip at Github). Since then, the version I need has been released.

So, update requirements.txt, then git push heroku master, and all is well, right? Not so fast...

Here's what actually happened:

Traceback (most recent call last):
  File "/tmp/.../pip-1.1-py2.7.egg/pip/basecommand.py"
    status = self.run(options, args)
  File "/tmp/.../pip-1.1-py2.7.egg/pip/commands/install.py"
    requirement_set.install(install_options, global_options)
  File "/tmp/.../pip-1.1-py2.7.egg/pip/req.py"
    requirement.uninstall(auto_confirm=True)
  File "/tmp/.../pip-1.1-py2.7.egg/pip/req.py"
    assert (link_pointer == dist.location), ('Egg-link %s does '
        'not match installed location of %s (at %s)') %
        (link_pointer, self.name, dist.location)
AssertionError: Egg-link ../../../../src/package does not match
installed location of package (at /tmp/.../src/package)

It turns out this is a known bug in Pip, and will hopefully soon be fixed. Unfortunately, I don't think I can use pip to install an editable version of pip which fixes the bug in which pip can't uninstall editable versions of packages on Heroku. (Sorry for that sentence).

So I did what any good internet citizen would do: tweet first, google later. Google turned up a patch to the Heroku Python buildpack that was merged several months ago, and appeared to do the right thing -- sadly, there was no apparent way to get the CLEAN_VIRTUALENV environment variable to the compile phase of my app.

Fortunately, at this point my friend Andy saw the tweet, and provided the critical hint: the user-env-compile Heroku labs feature. This gave me a way to ask the buildpack compiler to create a new virtualenv for me. On my next git push heroku master, I got a nice message: "CLEAN_VIRTUALENV set, rebuilding virtualenv" -- however, the same bug as before when installing requirements :'(

I reviewed the patch more closely, then traced through the buildpack file, and found that the VIRTUALENV_DIRS variable that the compiler uses to clean up the virtualenv is never set, so nothing was being deleted, and despite the message, my virtualenv was not being cleared and recreated. However, since the compiler is just a shell script, the same trick that got CLEAN_VIRTUALENV to the compiler could also be used to set VIRTUALENV_DIRS. (And I've submitted a patch so that in the future this won't be necessary.)

So, to put it all together, here's what you need to do:

$ heroku labs:add user-env-compile
$ heroku config:set CLEAN_VIRTUALENV=yes
$ heroku config:set VIRTUALENV_DIRS=.heroku/venv
$ git push heroku master
$ heroku config:unset CLEAN_VIRTUALENV
$ heroku config:unset VIRTUALENV_DIRS
$ heroku labs:disable user-env-compile

Hope this saves someone some time!