Debian packages are so old

Debian comes with tens of thousands of software packages that you can easily install on your system. But Debian only publishes a new “stable” release every 2-3 years. That creates the impression that Debian packages must always be up to 3 years old. And who wants to work with a three year old piece of software? Are the package maintainers lazy? Should I download my software from its own project website instead?

I feel obliged to briefly discuss this topic because it is a common source of trouble and surprises. And it may make you be

(more…)

Limiting your childrens’ internet access effectively using OpenDNS

There comes a time in a nerdy parent’s life when you happen to have kids that are old enough to get interested in using the internet. In my opinion there’s nothing wrong with digging through your pile of old hardware and assemble a reasonably good PC from it for your 10 year-old. But how do you avoid that your kids inadvertently find their ways into the less-desirable 90% of the internet? Read on about how to get the best results with adequate effort.


The Why of filtering

Recently my son (5th grade) claimed that other kids in his class were already allowed to use the internet. He would be the only one left out. Do the other parent really not care at all? And at the last parent teacher conference the class teacher told me that he announces everything class-related through his Facebook page. Yes – those are the people who are responsible for teaching our children media competency. He completely lost me when he claimed that getting a Facebook account is no big deal – it’s completely free. Excuse me? It appears that most people have no idea what’s going on on the internet. They occassionally get spam emails. They heard that there are porn sites. But what could possibly happen?

What to control?

There are essentially three different things that your kids may want from the internet: web surfing, email and installing software. Each of them pose different threats. Porn sites are not for 10 year-olds. Neither are social media sites that make children ruin their lives by posting inconsiderate content. Phishing emails may trick them to unwanted behavior. And installing random software thoughtlessly will easily get tons of adware, spyware and malware on their computers. So all three areas should be dealt with.

Web filtering

There is a reason why decent web filtering usually requires commercial solutions that are not affordable to anyone but large companies: it is hard. Why is it hard?

  • Filtering HTTP requires at least a piece of extra software that intercepts HTTP requests and checks them if they are desired (e.g. Squid)
  • Filtering HTTPS (which increasingly many web sites use) requires heavily invasive actions that are either unsuccessful or even harm the intended safety of the encrypted HTTP connection (deliberate man-in-the-middle-attack pattern with incomplete or no certificate validity checks)
  • Commercial products do that with a mixture of automated scans and manual moderation with multiple employees categorizing web sites. If you start managing a whitelist (block everything except a list of known web sites) then you will spend countless hours adding entries to that list because common web sites load their information from various other sources and CDNs (content delivery networks). If you instead try to manage a blacklist you would have to get a complete list of bad web sites – not just the dozen sites that come to your mind but the literally over a billion existing domains. Every second 1.5 new domains are registered. Don’t even try to catch up with that.

Fortunately there is a good approach for filtering web sites that is less intrusive and at the same time easier to implement: DNS filtering. Every time you run a request for a web site you need to find out the IP address for the host name that you are requesting. For example for http://workaround.org/ your operating system needs to get the IP address for the “workaround.org” host name. If you get the actual IP address of my web server then you can reach my web site. However if the response you get leads you somewhere else then that other server gets the HTTP request. And if that other server sends you a web page that contains an error message instead it gets you content filtering one step even before HTTP filtering can occur. And it doesn’t even matter if you intended to do an HTTP request. It will even work for any other protocol like HTTPS.

DNS-based filtering

My first idea was to use a local name server (I’m running bind here anyway for my local domain) and use a blacklist like MESD. But on second thoughts I didn’t trust such a list to be maintained well enough. Fortunately there are services that offer exactly such DNS servers that let you configure which kinds of sites you want to reach. One of them is OpenDNS. They do not publish their software nor their blacklists – so “open” is a bit misleading – but at least their service is free (beer) for personal use. Now if you can make your children’s PC use that DNS server then you wouldn’t need any other kind of URL filter.

Pros:

  • does not require maintaining any lists – you just have to make up your mind which categories of web sites you find inacceptable
  • easy to implement

Cons:

  • privacy: the external DNS filtering service gets to know which web sites you tried to access
  • tampering: this kind of filtering only works as long as the user can’t use another name server or manage their /etc/hosts entries

So in the simplest case you get a free (beer) account at OpenDNS, configure the forbidden categories of web sites in their dashboard and just use their DNS in your internet router’s settings to use the OpenDNS settings for everyone in your network from now on. But that way you would apply the same filtering policy for all users. If you don’t mind that you are done.

Filtering only on certain computers

However you may want to apply the filtering policy to only your childrens’ computers. So you need to change their network settings. On an Ubuntu workstation you can still use DHCP to get an IP address in your network but override the DNS servers like this:

Caveat: Your children may locate that setting and use the usual DNS server again but let’s not worry about that for the moment.

Optional: Assigning the OpenDNS name server from your DHCP server

If you are like me then you are running your own DHCP server in your network. I’m using the isc-dhcp-server package on a Debian server. So instead of changing the network settings on the PC you can assign the OpenDNS name servers by sending your children’s PCs a special DNS setting if they use DHCP. Edit your /etc/dhcp/dhcpd.conf file and add a section for each children’s host:

host kid1 {
    hardware ethernet 9a:d1:83:d2:50:e4;
    option domain-name-servers 208.67.222.222, 208.67.220.220;
    }

Of course you will have to find out the computer’s hardware address of the network card and use it instead of the one that I’m using. Now you won’t need any further settings at the PC.

Mandatory: Do you have a dynamic IP address?

Most internet providers just assign a dynamic IP address to you that is changing frequently. Unfortunately OpenDNS would not be able to identify who you are if your IP address changes. So you need to tell them your new IP address to make the old policy work for the new address. That requires two steps.

First you have to give your network (actually just your public IP address that you probably hide behind using network address translation (NAT)) a unique label. Log into your OpenDNS dashboard and locate the network settings. Then edit your network and give it a name. You will need that name again in a minute.

OpenDNS uses a DynDNS-style update mechanism that you can use to tell them your current IP address. The configuration is explained in one of their support articles. All you need to do is install the ddclient package and use the configuration as described in their configuration example. Now whenever your IP address changes OpenDNS will get to know that and apply your desired filtering policy to every request coming from your new IP address.

Make tampering harder by using a firewall

Your children may realize that you changed their DNS server to block web sites and try to change back the IP. To prevent that any other DNS server is used you would have to use appropriate rules in your firewall. For example if you are using Shorewall (which I can totally recommend on any Linux server) then these rules will do:

# Allow DNS requests to the OpenDNS servers
ACCEPT  lan  inet:208.67.222.222,208.67.220.220  tcp,udp 53
# Forbid using any other name servers for the chidren's PC having the IP address 10.7.0.42
REJECT  lan:10.7.0.42  inet   tcp,udp 53

# Otherwise allow any access to the internet...
ACCEPT  lan    inet
# ...and the server itself
ACCEPT  lan    $FW

Caveat: Your children could just disable DHCP and use a fixed IP address in your network that is not restricted. But unless you intend to introduce VLANs and IEEE 802.1x-based network authentication you may even live with that risk. A family therapist may be a more adequate solution if you are having parental problems that require such extreme security measures. If you just want to notice such attempts you may run arpwatch in your network.

Browser plugins

So far you have prevented access to unwanted web sites. But you may still want to keep the internet less distracting and confusing by using the mandatory set of browser plugins:

  • Adblock or Adblock Edge will get rid of any advertisements
    (I have no mercy with web sites that earn their money by showing so many ads that even an experienced internet user can hardly spot the actual content)
  • Ghostery will block all the trackers from commercial nosey big-data voyeurs

Filtering email

At the beginning of this article I also mentioned email as a potential threat. Why does spam, phishing and malware (viruses, trojans) work so well? Because even many adults can’t identify them and fall for it. If nobody would pay attention to it then the spam problem would be long gone. But if adults can’t tell good from bad then how are your kids expected to do that?

As a nerdy and worrying parent I strongly recommend that you don’t use a freemail service like Google, GMX, 1&1 or whatever. Those services are free (beer) for a reason – think about it. If you haven’t already then set up your own mail service by using my ISPmail tutorial. That way you have full control over incoming emails and can set server-based filtering rules. I have configured rules that only allow emails to my children from trusted sender addresses. All other emails end up in my inbox and I will forward it if appropriate. (Of course I won’t keep doing that indefinitely while my kids grow up.)

Installing software

As a sysadmin I find it unbelievable how many people fell for malware like browser bars. Of course I know how easy it is to get infected. You need to install Java… you forget to disable the checkbox that installs McAfee crap or an Ask.com browser disenhancement… there you go. (One more reason why I think people should stop using Oracle Java. A company that is in dire need of winding up their users like that should face a boycott.) And on a common PC magazine-style download web site I’d like to see you figure out which of the dozen download links actually gets you where you wanted (hint: it’s the tiny one two screens down that just consists of barely readable text – not the huge blinking button at the top).

What is my suggestion? Simple: use Linux – e.g. Ubuntu on your workstation. If you are using Windows you need to get all kinds of common applications from third-party sources. Your Firefox browser may be a hoax if you get it from the wrong source. Oracle pollutes your computer with browser bars. You inadvertently get dozens of pointless “download helpers”. On a modern Linux distribution you get most of your software packages from the built-in software manager. No need to dig through the internet to find trusted applications. You can’t really go wrong with Ubuntu’s “software center” application.

Conclusion

Now that we have dealt with technical measures it’s time to spend time with your children and explore the internet together. Don’t let them do that on their own. And don’t count on schools to teach them any kind of media competency because most of the teachers are as helpless as the pupils. But above all: let your kids explore IT. Compared to my generation today’s kids have become overly ignorant of the underlying technologies that power the internet. We really need to change that.

I hope you found this article helpful. Let me know what your experiences are with children and the internet in the comments below.

Reverse social engineering

In the past days I have watched a lot of 30C3 talks that I missed to attend in person. Getting a brain update of what is really going on in this world is what I enjoy. A well deserved break from politicians’ lies and mass media. One of the talks I came across was about social engineering. And just today yet another idiot in the world showed me that some people have become stupid enough to hack themselves. This is what I call reverse social engineering. (more…)

Moving contacts and calendar entries from Google to OwnCloud

As an Android user it’s hard to evade Google. You won’t be able to install apps without creating a GMail account and telling them your credit card number. But let’s at least not use GMail for your emails, contacts and calendar entries. In fact that is suprisingly simple. Let’s start with the…

Calendar

In OwnCloud enable the Calendar app if you haven’t already. Log in as “admin”, click on “Apps+” on the bottom of the left navigation area and activate the Calendar app.

Install CalDAV-Sync (yes, costs a bit of money) on your Android phone. Run the application and set up a CalDAV account. The CalDAV URL can be found in the OwnCloud web interface when you navigate to your calendar and click on the cog wheel icon in the upper right corner. It looks something like cloud.example.org/remote.php/caldav/

Log into your Google Calendar and go to the settings hidden behind the cog wheel icon in the upper right corner. Click on “Calendar” at the top. Look for a “export calendar” link. You will get a ZIP archive downloaded that contains an ICS file. Unpack that ZIP archive. Navigate to the files section on your OwnCloud in the web browser and upload the ICS file to any folder using drag-and-drop. Then click on the file in the browser and confirm to have that ICS file imported to your OwnCloud calendar. Now your calendar has been copied to OwnCloud.

Finally remove the appointments from your Google calendar and stop the sync from your smartphone. Neither can you remove the Google calendar from Google nor from your phone – it’s a pest. Make your OwnCloud calendar the new default on your phone. (If you own a Samsung S4 I suggest downloading the original “Google Calendar” app because the Samsung calendar app does not allow you to get rid of the Google calendar.)

Contacts

(Pretty similar as above.)

In OwnCloud enable the Calendar app if you haven’t already. Log in as “admin”, click on “Apps+” on the bottom of the left navigation area and activate the Contacts app.

Install the CardDAV-Sync app (still beta and thus free) on your Android device. Run the application and set up a CardDAV account. The CardDAV URL can be found in the OwnCloud web interface when you navigate to the contacts app and click on the cog wheel icon in the lower right corner and then on the globe icon. It looks something like cloud.example.org/remote.php/carddav/addressbooks/johndoe/contacts

Log into your GMail account and click on “More” and “Export”. Choose to export all contacts in vCard format. Navigate to the files section on your OwnCloud in the web browser and upload the VCF file to any folder using drag-and-drop. Then click on the file in the browser and confirm to have that contacts imported to your OwnCloud contacts.

And finally remove your contacts from GMail and stop the sync.

One step closer to Google-independence – yeah!

Trust over security in today’s internet

I assume that you are rightfully scared of public cloud services like Google and the legal terrorism done by insatiable secret services. If you are instead a happy GMail user and don’t know what I’m talking about – go watch cat pictures instead and enjoy your life. Beati pauperes spiritu. 🙂

For a while I followed the daily horror news after it became public what secret services are doing against the people that pay them. Then I started wondering what to do against it. A friend asked whether we should join public protest marches against surveillance. But let’s be honest – that had just wasted both our time and improved absolutely nothing. It’s probably human to “just do something about it” – no matter how pointless it actually is. Our politicians do not care for protesters. And what is worse: various backyard lobby meetings made their surveillance legal. That’s why I speak of legal terrorism against us – the people. Of course what they are doing it completely wrong in every aspect. But protesting does not help. Voting does not help either because our politicians are bound by their own laws anyway. It’s about too powerful individuals, lobbyism and money. Everyone with at least an average informedness should know from public information sources that it’s not about terrorism but industrial and economical espionage and keeping power over the people. It is very unlikely that you have any power to change that situation alone. Protesting? Screw that.

The next logical idea is increasing technical security. After all we are geeks so let’s use our knowledge and improve the ways we communicate. I switched all my own web sites to HTTPS (of course I used self-signed certificates), switched to SSH public key authentication and replaced all passwords that I might ever have used over an unencrypted channel like HTTP. I enabled the OTR encryption in Pidgin to communicate with my friends in a safer manner. That made me feel very relieved regarding the services I was personally responsible of. The only lacking part was email. I could just convince two relatives to switch to GnuPG for personal email – but most people don’t care enough or claim that GnuPG makes their lives too complicated. So I consider email a public medium and educate people to use it like that. Okay, technical security increased – check.

So the communication channels were secured. But what information did I store on the internet? Well, unless I stopped shopping on the internet there would be no way I could avoid leaving footprints. Strangely it scared me way more to save my contacts and calendar entries at Google to sync them with my Android smartphone than having them know my credit card number. Think about it: what is the worst thing that could happen if someone steals my credit card number? I would call my bank, cancel the card and get a new one. But what if someone steals the full contact information of most people I deal with and knows from my calendar where I will be at a certain time? Why did I worry about encryption and on the other hand fed Google all that information? Just because the Android installation wizard tricked me into it and made it easy. Oh, you evil laziness. So I moved my calendar entries and contact information into my personal OwnCloud instance. Except for the “Play Store” (still the most ridiculous name for an appstore anyway) I did not need anything from Google. And while I was at it I stopped using my Google account as an OpenID provider and disconnected all trust relationships to other websites and instead gave every website a distinct long password saved in my encrypted personal password file. I removed the last instance of Google Analytics and went through the ridiculously simple steps to set up Piwik instead.

Then at 3 a.m. one day I was on the phone with a friend when I suddenly realized what the actual problem is: trust. What is the good of having a smartphone calendar if I have a bad feeling storing personal information at an organisation that I don’t trust. To hell with Google’s “don’t be evil” slogan – it died long ago. I don’t trust politicians to decide in my interest. I don’t trust my bank to have any interest in anything but my money. I don’t trust the press to provide me with well-balanced information about the state of the world. I used to be called paranoid. Now I feel like the only non-naive running around. I find it alarming that the majority of people seem to have stopped thinking about trust. Face it: what is the motivation of a newspaper to deliver news to you? Their interest in being honest to you? Or their interest in making money? I don’t mean to say that you should distrust humanity entirely. Many people do great things. Just be careful who you trust – and that’s way more true than decades ago when your mom told you so.

You think you control who you trust? Take a look at the trusted certification authorities in the browser that you use to read this article. What made you trust these organisations? My next article will probably about the absurdity of trust centers who trade money for fake trust. Stay tuned.

Your privacy checklist for the post-cloud era

Secret services are spying on us. Companies call us consumers and consider us slaughter cattle. Governments invent tales of terrorists to ensure they keep the powers that they don’t deserve. I used to be reviled as paranoid – but nothing is for free. Face it: the golden age of cloud services over where we deliver our intimate data to untrustworthy organisations “just because it’s free and easy” or because we have legal agreements that we rely on.

So why not increase your privacy bit by bit? Here is your checklist for the post-cloud era.

ProblemSeveritySolution
Contacts and calendar entries in AndroidHighSet up OwnCloud and use CardDAV and CalDAV to store your data in your own private cloud. Works reliably. Or maybe your NAS at home (like Synology) has such DAV services built-in.
Social networking (Google+, Facebook, Twitter)HighJust cancel your accounts. You won’t miss them after a while. They are a waste of time.
Email account at a public webmail provider (Google Mail, GMX, Freenet)HighRent your own Linux server. Starts at less than 8€ per month. Set up your own mail server.
DropboxHighRemove your files and cancel your account. You don’t really need your files in several places. Carry an encrypted external USB drive with you if your computer does not provide enough room. It is much faster anyway. If you totally need a cloud storage then install OwnCloud on your own Linux server.
Google AnalyticsHighIf you want to get beautiful statistics about your web site then use Piwik. But often it’s enough to just analyse your access log.
Ads and tracking in browserHighAds are annoying. They steal your time. And sometimes they even carry malware. Install the Adblock Plus uBlock-Origin extension in your browser. To gain extra points also install uMatrix and get complete control about scripts, stylesheets, Javascripts, cookies and frames loading from each site.
User-agent tracking in browserHighThe identification of your browser is so unique nowadays (try it) that analytics companies trust them more than putting cookies on your disk. Hide yourself by randomizing your user-agent string. There are add-ons for Firefox but I didn’t find anything decent for Chrome.
Untrusted browser add-onsHighConfigure your browser for click-to-play plugins.
Google searchMediumUse startpage.com as your search engine. You will still get Google’s good results but with less tracking.
SkypeHighUninstall it. Install Pidgin and sign up for an XMPP (aka: Jabber) account at any server – e.g. jabber.workaround.org. Enable the Off-The-Record (OTR) plugin to encrypt your chats.
WhatsappHighUninstall WhatApp. Get Threema (proprietary encrypted client) or Xabber (XMPP-based) on your smartphone. Signal (formerly known as TextSecure) is a free and pretty good alternative that also encrypts voice calls. Never allow any messenging program to transfer your address book.
Closed-source softwareMediumGet rid of as much of it as possible. You hardly need Adobe Photoshop and Microsoft office – I bet you can live with Gimp and LibreOffice.
WindowsMediumThrow away Windows and use a free operating system. Unfortunately Ubuntu has too much spyware nowadays – I recommend plain Debian for the nerd and Linux Mint for the casual users. If you want to use Windows for playing games then dual-boot and don’t use Windows for anything relevant.
Online BankingHighNever do banking with your smartphone and avoid closed-source operating systems like MacOS and Windows. Avoid software like StarMoney that only runs on Windows and is famous for its bad user interface. Try Jameica.
WiFiMediumChange your WiFi password now. I bet you haven’t done that in years. Use a long password.
RouterHighUpdate your router firmware now.
Unencrypted mediaHighIf you are carrying your computer (e.g. laptop): Encrypt your hard drive and any external disks and USB sticks that contain your data. Decent Linux distributions offer effortless full-disk encryption. Encrypt external media using cryptfs. If you find those solutions too complex then use TrueCrypt VeraCrypt with hidden volumes. See Paul’s article on further alternatives to TrueCrypt.
OpenID authentication (Google, Yahoo)MediumDon’t use OpenID authentication providers. Get your own account on the web sites you use and use different passwords everywhere.
Password uniquenessHighNever use the same password for multiple different purposes. Web sites get hacked pretty frequently and you don’t want anyone to try your password in other places.
Password complexityHighAlways make passwords as long as possible and save it on your computer (e.g. encrypted in your browser). The length of your password matters more than the complexity. “s3cr3t” is a ridiculously simple password. A good password is “l(S.e/>oOVE~N>c+BO>P~(“. The more is looks like garbage the better. (The previous password was made using “apg -m 22 -a 1”. See also XKCD. Let your browser save the password if you want.
Keeping passwordsHighDon’t put your passwords in an unencrypted text file or on paper. Console nerds use VIM and a script. Graphical users use KeypassX.
Your own websiteMediumConfigure your own websites to use HTTPS. Forward any HTTP requests to HTTPS. Either create a self-signed certificate or get a free certificate at StartSSL if you want your certificate to be accepted by other visitors. But always create your own key and certificate request – don’t let the certification authority create your private key.
Unencrypted emailHighCreate a PGP keypair (4096 bit) and start sending encrypted email. Persuade your family and friends to switch to use PGP. A proven and easy solution is Thunderbird with the Enigmail add-on. Use obscure subjects because they are not encrypted. If you don’t like Thunderbird then on Linux you may like Evolution which is way more stable and performant and supports PGP.
Google MapsLowIt’s great for navigation. But if you just need a map or browse around then use Openstreetmap.
IP addressMediumYour IP address can easily be tracked down to your home and your person. If you are serious about security consider using Tor.
European ISPsMediumTry to avoid US-american ISPs. Of course other countries like the UK are not that much better. But apparently the US are monitoring most parts of the internet so your chance to accidentally get under suspicion is the greatest there. Unfortunately an increasing number of european countries is stacking up their surveillance of the people. At least Germany and France cannot be considered good hosting countries either.
Browser fingerprintingLowSome browsers like Firefox already warn you from fingerprinting attempts. Basically they try to track you even though you have cookies switched off. Not many sites do this. See Panopticlick and this article for more details.

Further ideas? Comment!

See also: https://www.privacytools.io/

Desktop despair – Gnome 3 risked too much

I've always been enthusiastic to show people the advantages of using a Linux desktop. KDE 3.5 accompanied me for many years until some ill-conceived KDE 4 ate my spare time and finally made me switch to Gnome 2 – after all we open-source people have a choice. Gnome 2 became more featureful and Ubuntu perfected the installation. There are times I deliberately toy around with my operating system. But when I needed to get work done it was Ubuntu. I could rely on it.

Then came 2011. Okay, Ubuntu had moved the window buttons to the left side already for no reason. Matter of taste. A little "gconf" magic and the buttoms were back where they belong. The came Unity with its huge buttons and turned my power-user desktop setup into something as sexy a ticket vending machine. And it killed the scrollbar in favor of some fancy user-unfriendly gadget. I finally gave up on Ubuntu because it needed too many fixes to be "my desktop" and so I was happily back to Debian with Gnome 2 on my desktop. And while I still wondered what Gnome 3 will be about it suddenly poured onto my desktop with a package upgrade (I'm not running "stable"). There I was on Gnome 3 and couldn't believe it. A useless panel that couldn't be customized any more. The right mouse button was dead. My virtual desktops were on drugs. No more icons on the desktop. My windows just had a close button. I was sure that I had made a mistake when I upgraded my system but reading other blog posts I did it all right. Who did it all wrong was the UI shamans at the Gnome project.

Don't get me wrong. I'm all in favor of trying something new every now and then. But the Gnome developers "fixed" something that was not broken. Of course they will argue that this is just a "developer release" and "not yet suited for the common user". But face it: your favorite Linux distribution will not give you Gnome 2 any more. This is not a design study – it is the future of Gnome. I don't know just one Gnome user who actually uses Gnome 3. They have either downgraded or switched to some entirely different window manager.

So what is the alternative for people feeling let down by Gnome? For me the refuge is XFCE. But XFCE is still behind Gnome 2. Docking/undocking with my laptop and two monitors does not work with XFCE (which worked perfectly with Gnome). The window manager (xfwm4) uses XRender instead of OpenGL which means that I have ugly effects when moving windows or watching movies because it doesn't sync with the vertical blank of my monitor. (Workaround: use compiz with xfwm4.) And Firefox doesn't work well with XRender either – scrolling on some pages burns my CPU. I also had to pull in parts from Gnome like the keyring manager. I'm really thankful that XFCE has evolved that much and I'm sure that it will get a lot of interest because it's the closest to Gnome 2.4. I second what Linus Torvalds said: I really hope that someone forks Gnome 2.4. But that hasn't happened with KDE 3.5 and I don't really expect that to happen with Gnome 2.4. So both KDE and Gnome are losing a relevant amount of fans.

These are wild times. Android is outgrowing IOS. Macs have become computers with a serious market share. Micro$oft is on its slow journey to insignificance. Killing the two major Linux desktops (KDE and Gnome) and seeing Ubuntu introduce changes that increasingly annoy users is a dangerous evolution. Two die-hard Linux coworkers have just recently switched to Windows 7. I'm scared. Plenty. What will I blog in two years? I'll probably be either on fvwm2 (which I used in the 1990s) or have given up and will be a happy Mac user. Who had expected that the Linux desktop would evolve backwards one day.

Getting help on IRC

IRC (Internet Relay Chat) is a real-time chat where you can talk to other people from around the planet. Although many channels deal with smalltalk about the weather, girl-friends and politics there are IRC networks and channels that are dedicated to a piece of software or an operating system. irc.freenode.net for example exists to support open-source projects. In case you need urgent help or just don’t want to use a mailing list then you are welcome on IRC. IRC has been there for ages and has evolved with the time. So it may appear like a hot tub of completely mad people at first. This article is meant to help you understand how to IRC works socially and how to get the most out of it.

#1 – Don’t ask to ask

Do not start your conversion with “May I ask a question?” or “Can anyone help me?“. Although that may be polite in normal human interaction it does not work that way on IRC. You are supposed to just ask your question right away.

#2 – Be precise

There is nothing worse than “My printer doesn’t work.” or “My mail server doesn’t work.“. How are others supposed to help you if you don’t give them any details? Make sure you include at least this information:

  • What did you try to accomplish? (I wanted to fetch my mails via POP3.)
  • What has happened (wrongly) instead? (But my inbox remains empty.)
  • What environment are you using? (I am using kmail 1.8 in KDE 3.4.)
  • What did you try to find the cause? (I already checked my POP3 account setup.)
  • Are there error messages or log files? (The error message reads: “SSL negotiation failed.”)
  • Has it ever worked? What did you change? (I switched accounts to a new provider. It worked before with another provider.)

If all this information is included then you will have a high chance of getting a useful reply.

#3 – Tell what you are doing

Others only know what you tell them. If you change things silently then nobody will know about it. The worst thing you can do is say: “I changed something. Now I have another problem.” It will make others give up instantly. If you expect help you need to work with the supporters.

#4 – Read the /topic

Your IRC client will likely show you the “topic” of a channel when you enter it. You should find some useful information about how this channel is supposed to work. Often the topic even lists URLs for FAQs and documentation. Asking something that is clearly answered in the topic is considered stupid. If there are FAQ URLs listed in the topic first check if you find your question answered there.

#5 – Do not be demanding

Do not expect a single person to stay online until your problem is fixed. IRC is a stateless medium. Anyone can quit at any time. People stay there because they like staying there. Nobody is paid for helping you.

#6 – Do not /msg people without asking

You can either talk to others in a channel publicly or /msg them privately. However the latter is considered rude if done without being asked. Don’t occupy a single person for your purpose. Others may as well be interested in a solution to your problem, too.

#7 – Unintended rudeness

Sometimes you may feel that others are rude. You may get a response like “Reinstall the package. Restart the service. Read the /usr/share/doc/mysoftware/README.” Although this is a very brief reply it is likely not meant to be rude. IRC is often like human interaction without all the friendly bits. Other people have probably answered your question a dozen times today and they just want to help you solve the problem – not become close friends with you. Don’t be offended by it.

#8 – Do not repeat yourself

Asking the same question every minute is annoying. If anyone in the channel isn’t paying attention then they will neither read you the second and third time. And those who came back from lunch or sleep will likely see in their “lastlog” what has happened lately.

#9 – Speak English

In most channels the only language spoken is English. There are certain local channels like #debian-fr which are specifically for frenchmen. But everywhere else you must not write anything but English. It doesn’t matter if you consider your English language skills bad. Most people in the world speak something different than English, too. And if you find another person that speaks your language then do not start to talk in your local language. Nobody else can follow you and it’s considered rude.

#10 – Don’t be lazy – read the documentation

If people tell you to read the documentation then you should do so. Never say: “I’m too lazy. Come on. You all know what I need to do. Just tell me the solution.” This will get you ignored for all eternity. However you can expect to be told where to find the documentation. If you get a URL then follow it and read it. If the documentation is too technical or you don’t understand certain sections then be more specific like: “I have now read chapter 3.1 of the URL you gave me and I understand how virtual domains work in general. But how would I use both virtual and non-virtual domains together?”

#11 – Give others a chance to answer

Even if you are in a hurry you should allow other participants some time to answer. Sometimes you are lucky and a helpful reply appears within a few seconds. But it is not uncommon that you will have to wait 10-20 minutes until the right person who is familiar with your problem talks to you. The pattern looks something like this:

 10:07 Foo joins
 10:07 Foo> How can I install the shripdibble module?
 10:09 Foo quit

Then at 10:14 an expert on shripdibble joins but can’t help you any more because you gave up too quickly.

#12 – Stay if you can

Using IRC is a matter of give and take. At the beginning you will surely have more questions than answers. Just ask them and be grateful if you get attention and replies. Others do not expect more than a “Thank you.” for their help.

If you can, please consider staying online in the channel. The more knowledge you gain the more you can help other people. And you probably get to know people you like to hang out with – even if it’s just virtually.

#13 – Answer the questions that you get asked?

It may sometimes be hard to follow all the conversation on IRC. But if people are trying to help you and need to ask you a few questions please answer them all. If you get asked three questions and only answer one or keep asking the same question time and again (showing that you don’t read what you get asked) you risk to be ignored. Show some initiative and stay focused. Then your problem will likely be fixed within a few minutes.

#14 – Tell others about the solution

It doesn’t help anyone if you just say “Nevermind, found it.” and quit the channel. Please tell the others what the solution to your problem was so everybody can learn from it. You will not look stupid – trust me.

#15 – Careful who you trust

When getting responses be careful who you trust. Some people just want to feel important and reply to you although they have no more clue than you do. Perhaps even an “educated guess” might help – just make clear if it’s something that was tried and worked. Some people are even jokers who want to be funny by telling you how to erase your harddisk. So verify others’ proposals instead of applying them blindly.

#16 – Do not flood the channel – use a pastebin

Sometimes you need to show others more than one line. Perhaps parts of your configuration files or a log file. Never just copy and paste larger amounts of text right into the channel. Always use a pastebin. Pastebins are public services (web sites) where you can paste your text and everybody can access them as a URL. Just paste your contents there and send the URL into the channel. That should allow everybody who’s interested to take a look. Just don’t paste it without any comment. Rather say: “Emails seem to get lost since I installed program foobar. Please take a look at my logs: http://…”

See also:

MySQL to PostgreSQL – a Bacula conversion odyssey

Why is it that always the seemingly most simple things turn out to be the most annoying? This time I “just” wanted to get rid of one of my last MySQL databases and move it over to PostgreSQL. The Bacula catalog that saves which file I backed up when and to which storage medium. I tried with MySQL’s “mysqldump” and it’s PostgreSQL compatibility option – but apparently MySQL developers know nothing about PostgreSQL. Then I tried with “sqlfairy” – and found myself booting my system hard after it ate 2 GB of swap and died while converting 500 MB of data.

So finally I asked in #bacula and was told to try CSV (comma-seperated values) as an intermediate format. Yuck… that satan-spawned format that reminds me of my dark past? Okay. First dump the catalog from MySQL:

for table in BaseFiles CDImages Client Counters Device File Filename FileSet Job JobMedia Location LocationLog Log Media MediaType Path Pool Status Storage UnsavedFiles Version ; do mysqldump -u root -pmypassword -T. bacula $table; done

(Okay, okay, this is not comma- but tab-seperated. But that’s even better for running the COPY-FROM command later.)

This creates an ‘.sql’ (the schema) and a ‘.txt’ (the rows/records) file for each table in the current directory. Just don’t try to apply the schema to PostgreSQL. Instead better create a new schema. Bacula ships with a script for that purpose.

Unless you have the PostgreSQL database for Bacula ready you should run something like…

/usr/share/bacula-director/make_postgresql_tables -h localhost -U bacula

…and…

/usr/share/bacula-director/grant_postgresql_privileges -h localhost -U bacula

Now on to read the tab-delimited data into PostgreSQL. The importing via the COPY command must be done with administrator privileges! And it’s important to explicitly state which columns correspond to which table columns (see the respective ‘.sql’ files) or otherwise you’ll get chaos. Of course this only has to be done for .txt files larger than 0 bytes. Oh, and the filename has to be absolute. Example:

  • psql -h localhost bacula admin -c “COPY basefiles (baseid,basejobid,jobid,fileid,fileindex) FROM ‘/var/tmp/BaseFiles.txt'”
  • psql -h localhost bacula admin -c “COPY cdimages (mediaid,lastburn) FROM ‘/var/tmp/CDImages.txt'”
  • psql -h localhost bacula admin -c “COPY client (ClientId,Name,Uname,AutoPrune,FileRetention,JobRetention) FROM ‘/var/tmp/Client.txt'”
  • psql -h localhost bacula admin -c “COPY counters (Counter,MinValue,MaxValue,CurrentValue,WrapCounter) FROM ‘/var/tmp/Counters.txt'”
  • psql -h localhost bacula admin -c “COPY device (DeviceId, Name, MediaTypeId, StorageId, DevMounts, DevReadBytes, DevWriteBytes, DevReadBytesSinceCleaning ,DevWriteBytesSinceCleaning, DevReadTime,DevWriteTime, DevReadTimeSinceCleaning, DevWriteTimeSinceCleaning, CleaningDate, CleaningPeriod) FROM ‘/var/tmp/Device.txt'”
  • psql -h localhost bacula admin -c “COPY file (FileId, FileIndex, JobId, PathId, FilenameId, MarkId, LStat, MD5) FROM ‘/var/tmp/File.txt'”
    psql -h localhost bacula admin -c “COPY fileset (FileSetId, FileSet, MD5, CreateTime) FROM ‘/var/tmp/FileSet.txt'”
  • psql -h localhost bacula admin -c “COPY filename (FilenameId, Name) FROM ‘/var/tmp/Filename.txt'”
  • psql -h localhost bacula admin -c “COPY jobmedia (JobMediaId, JobId, MediaId, FirstIndex, LastIndex, StartFile, EndFile, StartBlock, EndBlock, VolIndex, Copy) FROM ‘/var/tmp/JobMedia.txt'”
  • psql -h localhost bacula admin -c “COPY location (LocationId, Location, Cost, Enabled) FROM ‘/var/tmp/Location.txt'”
  • psql -h localhost bacula admin -c “COPY locationlog (LocLogId, Date, Comment, MediaId, LocationId, NewVolStatus, NewEnabled) FROM ‘/var/tmp/LocationLog.txt'”
  • psql -h localhost bacula admin -c “COPY log (LogId,JobId,Time,LogText) FROM ‘/var/tmp/Log.txt'”
  • psql -h localhost bacula admin -c “COPY media (MediaId, VolumeName, Slot, PoolId, MediaType, FirstWritten, LastWritten, LabelDate, VolJobs, VolFiles, VolBlocks, VolMounts, VolBytes, VolErrors, VolWrites, VolCapacityBytes, VolStatus, Recycle, VolRetention, VolUseDuration, MaxVolJobs, MaxVolFiles, MaxVolBytes, InChanger, MediaAddressing, VolReadTime, VolWriteTime, EndFile, EndBlock, LabelType, StorageId, VolParts, MediaTypeId, DeviceId, LocationId, RecycleCount, InitialWrite, ScratchPoolId, RecyclePoolId, Enabled, Comment) FROM ‘/var/tmp/Media.txt'”
  • psql -h localhost bacula admin -c “COPY mediatype (MediaTypeId, MediaType, ReadOnly) FROM ‘/var/tmp/MediaType.txt'”
  • psql -h localhost bacula admin -c “COPY path (PathId, Path) FROM ‘/var/tmp/Path.txt'”
  • psql -h localhost bacula admin -c “COPY pool (PoolId, Name, NumVols, MaxVols, UseOnce, UseCatalog, AcceptAnyVolume, VolRetention, VolUseDuration, MaxVolJobs, MaxVolFiles, MaxVolBytes, AutoPrune, Recycle, PoolType, LabelFormat, Enabled, ScratchPoolId, RecyclePoolId, LabelType, NextPoolId, MigrationHighBytes, MigrationLowBytes, MigrationTime) FROM ‘/var/tmp/Pool.txt'”
  • psql -h localhost bacula admin -c “COPY storage (StorageId, Name, AutoChanger) FROM ‘/var/tmp/Storage.txt'”
  • psql -h localhost bacula admin -c “COPY unsavedfiles (UnsavedId, JobId, PathId, FilenameId) FROM ‘/var/tmp/UnsavedFiles.txt'”

Unfortunately my “Job.txt” and “Media.txt” contained datestamp entries like “0000-00-00 00:00:00” which are not valid for PostgreSQL. So I went into Vim and replaced it: s/0000-00-00 00:00:00/1970-01-01 00:00:00/g. Clear the table (DELETE FROM job) and import again.

And finally it’s important to get the sequence numbers right as described in the Bacula manual.

Morale: spend two extra-minutes to start with PostgreSQL right away instead of bothering about conversions later. And never assume converting from one database to another would work – just because both have “SQL” in their names.

setuptools versioning – wtf?

Nowadays I'm taking Debian's APT software management system pretty much for granted. It probably represents the most advanced and powerful system. So I'm not instantly turned away if I find other systems that don't work as well. Observing how Windows users struggle with getting the 50 different installed pieces of software to update that all use their own mechanisms thus wasting computing time and hard disk space while often getting annoyed by "Do you want to update now?" requests or sudden reboots is fun. Anyway, as an addicted Python programmer I obviously had to deal with the "setuptools" already. It's Python's pendant to Perl's CPAN. It can download Python modules with certain versions, satisfy dependencies automatically etc. The central place for Python modules is the Python Package Index (PyPi). Obviously I usually wouldn't need to deal with the setuptools often because Debian already provides properly packages Python modules but sometimes it's nice to deploy applications within a virtualenv environment (similar to a chroot with its own set of module versions not conflicting with other versions installed on the system). Still I think that setuptools is pretty pathetic.

  • You can't uninstall modules properly. This is a no-go – we aren't in the Windows 95 ages any more.
  • PyPi often just links to the developers' home pages which tend to go offline when you urgently need to deploy an application and can't get a certain version downloaded. (Pylons has kindly collected the needed eggs on their download page for that case.)
  • Sometimes certain versions are requested that are no longer available on PyPi that have been released just weeks ago.
  • Outside of the virtualenv you'd have to install modules system-wide which collides with your software management like APT (unless you have an operating system like Windows that doesn't care anyway).

But the single most pathetic detail about setuptools is their versioning. Gustavo Noronha Silva pointed me to /usr/share/pyshared/pkg_resources.py which contained an explanation of how version numbers are compared:

Strings like "a", "b", "c", "alpha", "beta", "candidate" and so on (that come before "final" alphabetically) are assumed to be pre-release versions, so that the version "2.4" is considered newer than "2.4a1". Finally, to handle miscellaneous cases, the strings "pre", "preview", and "rc" are treated as if they were "c", i.e. as though they were release candidates, and therefore are not as new as a version string that does not contain them.

So the sorting goes something like a, b, c, pre, preview, rc, alpha, beta, candidate, final. And "2.4" is newer than "2.4a1" but not newer than "2.4f1"? Please people, lay off the drugs. I feel tempted to send that to The daily wtf. I thought that setuptools and virtualenv is a good combination if a certain Python module is not yet available in Debian. But I think I'd rather spend 30 minutes to create a proper Debian package with a real version numbering system instead of wasting my time with these antics. Don't get me wrong. I appreciate the intention of setuptools and easy_install. But the world needs a 2.0 of this work. From what I heard hardly anyone understands setuptools well enough to provide something better. Bummer.

© 2021 workaround.org - Proudly powered by theme Octo