[2019-11-15]

Debian packages are just so old

I feel obliged to briefly discuss the use of Debian packages in contrast to just pulling software from their website and installing it. Why is this important? Well…

“The roundcube software in this guide is totally outdated. Just go to roundcube.net and download a new version.”

(Frequent advice in the user comments of these guides.)

…or an even less friendly quote…

“I wish I could convince Debian maintainers to stop providing Rspamd in Debian repos. Probably, I should change a license for that, but I’m really tired of that crap…”

Vsevolod Stakhov – principal developer of rspamd

So what’s the deal? Shouldn’t we just use Debian packages where we find it comfortable and install any other software in a newer version? Sure, nobody keeps you from doing that. Consider these aspects though…

Advantage: cool new software

That advantage is pretty obvious. Why use the old dusty Roundcube version that comes with Debian Buster if you can get the sexy shiny new version directly from roundcube.net? It’s just a tarball that can be extracted somewhere and works easily. Some software even comes in handy APT repositories so it’s simple.

Disadvantage: conflicts with the APT package system

Debian arguably has the best software package management of all operating systems. It keeps track of all files that are installed on your disk. And it knows how to upgrade to newer versions without damaging your configuration.

There are defined locations on your disk (according to the FHS) like /opt or /usr/local where additional software can live. That is certainly true but then you need to make sure that you do not have the same APT package installed, too. Otherwise you would have to specify exactly which copy of the software you want to use. This can lead to annoying mixups.

Disadvantage: conflicts with other packages

Debian package maintainers take great care that packages in a stable Debian distribution work well together. Dovecot in Debian Buster comes with helpful instructions on how to use the Postfix version in Debian Buster. If you take packages from other sources then you may find that they use unusual paths in your file system. Or that they somehow do not work with the “old” version of other packages that you installed from the Debian sources. This can quickly lead to an update frenzy where you need to update several unrelated packages that in turn break other packages.

Disadvantage: no automatic integration

Debian packages are not just a different format of the original (“upstream”) software tarball. Debian developers make sure that every part of the software gets into the right spot. For example configuration data always ends up somewhere in /etc. Also it integrates nicely with other parts. Debian’s Roundcube package offers working configuration snippets that plug into your Apache web server. If you install the software on your own you will lack that.

Disadvantage: no security support

Especially in public-facing services like mail servers this is my most important issue. Surely you can install the newest version of Roundcube. But as a responsible system administrator you will need to make sure that you check for updates every day and install them when they pop up. Debian packages get updates automatically. Your manual installation will not.

And that is no matter of paranoid phantasy. I have lost several systems in my life because the update mechanism failed and script kiddies found open security issues and did bad things on my servers.

And I have also broken my servers because I gave in to the temptation to use newer software. Some developers provide their own APT repositories and actively discourage you to use the packages provided by Debian. In fact I was literally shocked to read that the rspamd principal developer Vsevolod Stakhov not only discourages but even threatens Debian users if they dare using Debian packages. He even considers changing his license to exclude Debian from using his software. Of course bug reports of Debian packages should be sent to the Debian project first. However developers must not force users to ignore basic principles of Debian. In fact this is the most unprofessional behavior of an open source developer I have seen in the last years. Yikes.

That aside, developers hardly ever fix issues in old software. That’s just not fun. Developers like to go forward. While they implement new features they also fix issues on their way. I totally understand that motivation. But that way they force users to accept their changes and new features even if the users just want to fix the security issues.

So that way you are forced to upgrade. In Debian’s security updates you will get patches for the existing software version. If you installed version 1.3.8 then you will stay on version 1.3.8 – just with security fixes applied. You can rely on your system that your configuration will work throughout the entire lifetime of that Debian release. However if you installed version 1.4.x manually then you will have to use every new version that the author publishes. Security issues are then fixed by updating. And if that changes the configuration or the general UI then you and your users will have to deal with that. Or you just ignore further updates and are likely to find your server part of a botnet that keeps computing Bitcoins.

So be aware that when you manage software yourself you will have to check for updates. And expect broken servers.

Conclusion

Throughout this guide I will always make you install Debian packages. Also please expect better support through the comments sections if you stay on the trail of what is good and righteous. 🙂

For more information see also the Debian FAQ on security.

Oh, and Roundcube was just a random example. There’s nothing wrong with Roundcube. 🙂

10 thoughts on “Debian packages are just so old

  • 2019-12-28 at 17:57
    Permalink

    This is totally fair. I remember in the last guide I posted a similar comment about going rogue to install RainLoop or something like that. I get your position, and respect it.

    Reply
    • 2019-12-29 at 11:28
      Permalink

      Thanks for the comment. I don’t mind installing newer packages. I do that, too, on internal systems. Just wanted to make sure what consequences it has.

      Reply
  • 2020-01-11 at 21:52
    Permalink

    What you’re describing, especially about rspamd, is typical of a new mindset. The confusion between open-source and free software is probably not unrelated to this. A man like RMS is an exceptional programmer, but above all a man of exceptional moral rigor. The developer of rspamd is undoubtedly a good programmer, but he lacks this moral rigor.

    For years I’ve been using Gentoo, which also has an excellent package versioning system, but it needs a lot of attention. A kind of linux from scratch that would handle dependencies.

    But if you want to become the maintainer of all the packages you use, then 24 hours a day won’t be enough, and you won’t have a second to do anything else with your computer than keeping it up to date…

    And I’ve been around Unix systems since 1980 or so, and I’m still the living proof that what’s old isn’t necessarily obsolete! 😉

    Reply
  • 2020-01-27 at 06:08
    Permalink

    I agree with all that you’re saying, Christoph.

    What I do on my own servers when I really, absolutely need the latest and greatest package version (e.g. Nextcloud) is I spin up a Docker container.

    Fortunately, most open-source server side projects do have an official Docker image these days, and you can set them up without the risk of screwing up your system’s package dependencies.

    Containers such as Watchtower ensure that the latest version is always installed, but of course when doing that it’s important to specify a version tag when initially building the Docker container, so that there won’t be unexpected upgrades to new major versions which could contain breaking changes.

    It does require some more attention to a project’s development roadmap, because you do have to keep track when a version of a project you’re using reaches end of life and upgrading to a new major version is advisable due to no patches being supplied for the older version.

    This may be too much effort for admins running huge server clusters, but for me (with only a handful of critical services running on a handful servers) it’s definitely manageable.

    BTW, I was *EXTREMELY* skeptical of the whole containerization movement when it first started, I used to think that it maybe could be useful for deploying services on a huge set of servers quickly for easy scalability, but my feeling was that the overhead of creating a whole container environment for just one service would not justify the benefits.

    Having used Docker for over a year at this point though, I must say that it has made my life so much easier, especially when installing various services with conflicting dependencies on the same machine, or in case I want to try out a project with lots of dependencies (GitLab and Logstash are great examples here) while being able to easily get rid of it if I want to.

    Ultimately it is up to each and every admin to decide what strategy is right for them, but in cases where installing the distro-supplied package is not feasible, I personally would always prefer setting up a container over installing the package directly onto the system by pulling from GIT or similar.

    Also thanks for writing up these wonderful tutorials and keeping it up for so long, you’re a legend Christoph!

    Reply
    • 2020-01-28 at 15:38
      Permalink

      Thanks for your thoughts, Robin. Even though I don’t consider myself a “legend”. Lol. 🙂

      In my professional life I work a lot with containers and Kubernetes. And I have already felt the urge to put up a mail server on Kubernetes despite all the hell that consists of YAML, Helm, CI etc. After all if done right you could just scale your mail server to the skies and back. But in reality that stuff keeps breaking for me.

      Looking at the Dockerfile that create the images found e.g. on hub.docker.com I’m scared of their quality. Last week it took me two days to debug a stupid timezone issue in the “mariadb” image. The next two days went down the drain for a weird design decision in the “ruby” image. The Dockerfiles are often just very very weird and remind me of my first BASIC projects on my trusted old C=64.

      And as you mentioned keeping the software up to date (e.g. security issues) is a problem, too.

      Reply
    • 2020-02-16 at 18:53
      Permalink

      He’s such a moron. I wonder why he keeps broadcasting that. I feel bad for endorsing the work of such a person. If it’s not too childish maybe we should use some other software for the next ISPmail guide.

      Reply
    • 2020-03-05 at 10:49
      Permalink

      LOL, he blocked me on Twitter.

      Reply
  • 2020-06-21 at 15:16
    Permalink

    What are your thoughts on using the buster-backports packages from Debian? In your example of Roundcube: buster-backports offers Roundcube 1.4.6 (at the moment)
    It tackles the problem of updates and dependencies. The only downside I see is that this might introduce dependencies to other packages in buster-backports, but my experience is that Debian tries to limit this effect by using sensible version dependencies (so packages in stable are compatible)

    PS: Thanks for your excellent work!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *