(Hint: This article talks about the Pylons web framework which is essentially dead. But the warning about easy_install and pip is still valid.)
This article tries to make clear why blindly running setuptools/ez_setup on Debian/Ubuntu is dangerous and will happily break your operating system.
I loved the Pylons web framework. A significatnt number of developers was using it. It could be installed easily through a simple – though not sophisticated – tool called easy_install. Easy_install is a wrapper/bootstrapper around the Python setuptools.
Setuptools isn’t terribly bad per se
Pylons depends on a number of Python modules. It would be a real pain to download and install all the right versions of these modules from different web sites just to run Pylons. Fortunately there are the setuptools that download the dependencies and install them on your system. It uses the PyPi Python package index that lists information on available packages and where setuptools can get them from. If your operating system has no sane way of tracking the installed software (like Windows or UNIX flavors that do not have a package management) then this approach is fine.
Setuptools is not a decent software package management
Unfortunately most modern operating systems like Linux distributions have come with a sophisticated package management. Its job is to track what software is installed and what files belong to what package. You can install and uninstall packages (without leaving debris on your system), upgrade them all easily, install security patches, keep your configuration upon upgrades and track dependencies and conflicts. So it’s similar to setuptools but does a whole lot more than that. Setuptools does not even allow you to uninstall a module. After all we are not in the dark ages of operating systems any more where you just install software and the regular way of tidying up your operating system is by reinstalling it because it breaks after a few months anyway. I didn’t have to reinstall Debian for years on my system but installed tons of software for playing around with it and removed it later. Actually the only explanation I have why setuptools is so widespread is perhaps that many Python developers use operating systems without software management. And don’t get me started about its version numbers. Anyway setuptools is a quick’n’dirty way to install Python packages.
Apt versus Setuptools
Debian has a large number of Python modules available as Debian packages through the advanced package tool (apt). Installing Pylons is just a matter of running aptitude install python-pylons. Do you smell the trouble already? Apt does not care for Python modules installed via Setuptools and vice versa. So it’s possible to install a certain module twice in different locations. Depending on the order of your PYTHONPATH you may use different versions of the same module. Some people argue that they know which Python modules they install through “apt” and which are installed through “setuptools”. That’s a dangerous game to play. Since both know how to install dependent packages/modules you will quickly install software you are not aware of. Welcome to the mess.
Debian’s modules are always outdated
Yeah, right. That is the old rumor. People who say that have still not understood the difference of “stable” and “unstable” branches of Debian. Debian consists of pretty up-to-date packages. Whatever is available via setuptools is likely available as a Debian package within days, too. So you will not miss anything. Debian releases a “stable” version roughly every two years though. Of course there is a lot of change happening within two years. But many people favor stability over new features. They are happy that their desktops will work smoothly for two years. If you prefer the newest software then just use the “unstable” branch instead of the “stable” one. The name “unstable” is misleading. It doesn’t mean that your system will constantly be broken if you use it. It contains the newest software though that couldn’t be tested thoroughly. So my personal recommendation to Debian users is: use “unstable” if you like to stay up-to-date.
Jailed by virtualenv
If you do not like to bring your system to “unstable” and still want to run Pylons applications you can create a virtual environment (virtualenv). It will debootstrap a Python environment into a directory that does not conflict with the rest of your operating system. I wouldn’t recommend that for daily work. But it allows to use setuptools to install the newest software. And if you get fed up or do not need it any more then just remove that special directory and no harm is done. An additional advantage is that you can run different projects that each require different versions of software. Perhaps one project does only work with a package of version <= 0.3 while another project needs at least >= 0.4. In this case you just give each project what it needs.
The drawback is that you do not get security updated from “apt” so you are completely on your own. And it’s not really suitable for working on a daily basis because you hardly get access to anything outside of that jail.
So who’s fault is all this?
The actual problem is that the creator of the setuptools didn’t care for the needs of Linux distributions that come with some kind of package management. Redhat has just recently discovered they have the same problem. So it’s not just something a mad Debian developer has come up to confuse the world. Perhaps sometimes in the future we will have apt-aware setuptools or a way to automatically create Debian packages from ?PyPi modules.
What should you do then?
My personal recommendation:
- Use “unstable” for development
- Deploy your application in a virtual environment
Whatever you do – NEVER install a Python module system-wide with setuptools/ez_setup on your Debian system. And – no – /usr/local is not a safe place for Python modules either.
I told you so. 😉