Apt is able to handle that but it’s not really easy to do (and to remember). There is a utility called wajig that simplify that a lot (wajig aim is to do all the package handling stuff simpler than apt and dpkg).

Install wajig as usual:

Code block

$ sudo apt-get install wajig

And here is how to use it for holding a package, unholding it or list all packages that are on hold:

Code block

$ sudo wajig hold
$ sudo wajig unhold
$ sudo wajig list-hold

Since I have the problem from time to time, let’s write a reminder on how to fix this.

Sometime there are grub updates, and on the next reboot, well, nothing happens. Hopefully, my hosting provider provides an option to boot my server on a Ubuntu live CD (my server is running Debian but that’s fine).

When Ubuntu has finished booting, I can ssh on it and list my hard drive partitions:

Code block

# fdisk -l /dev/sda
(…)
 Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          12       96389+  83  Linux
/dev/sda2              13       19324   155123640   83  Linux
/dev/sda3           19325       19454     1044225   83  Linux

Here sda1 is my boot partition, sda2 is my root partition ans sda3 is swap. Let’s mount that:

Code block

# mount /dev/sda2 /mnt
# mount /dev/sda1 /mnt/boot

Now we have to change the “root” directory from the live CD to my server’s disk so we can simulate being running on my server’s disk:

Code block

# chroot /mnt

And now we can reinstall grub:

Code block

# grub-install /dev/sda

In my case I have to give the --force option to install grub. That’s all, I can now restart my Debian server.

Someone started a thread about that on Apple’s forum: http://discussions.apple.com/thread.jspa?threadID=2564925

One guy says to have a look to forked-daapd which is a full rewriting of Firefly. So I did. On my Debian box, a package is available in “testing”:

Code block

$ sudo apt-get install forked-daapd

Change the directory where the music is in /etc/forked-daapd.conf, restart and that’s all. Then go to iTunes, the shared library should be visible and filled with your music!

Since Firefly is a dead project (no activity for a while), everybody using it should migrate to forked-daapd.

Note: this is my 100th post on this blog!

Why moving from dotclear to WordPress? First of all, my linux server moved to PHP 5.3… with is not supported by dotclear yet. So I had to install an alpha version of dotclear that supported it… kind of. It was a true alpha version (meaning lots of stuff were not working anymore, if you used to navigate on my blog you may have noticed that). And I feel a bit like dotclear is on the dying side (maybe it’s not true, but that’s my feeling). I don’t know if it’s related by the departure of the creator of dotclear (Olivier Meunier) last year, maybe they are adjusting from that change, but in the mean time, dotclear is not moving fast enough and I had to do something about getting my blog working.

I know, since I’m a developer, I could have helped dotclear guys supporting PHP 5.3, by fixing some bugs… I don’t remember PHP a lot but it’s a quite simple language, I should have been able to help quite quickly I think. But I’m also lazy and I did wanted to try WordPress since some people recommended it to me.

So, this is my first post written with WordPress, and it’s quite nice, the interface is quite good but I have to admit that it’s far from being perfect too.

Installation

As usual I used the debian package, which is already an improvement over dotclear since there is no dotclear debian package. On the downside, I encountered several bugs that seems to stupid to be there. And looking on the internet, they are known bugs, but not fixed. For instance:

• You can’t specify an absolute path for your upload folder (which is what is done by default by the debian package… so uploading anything was not possible until I changed the path)
• When you try to install a theme or a plugin using the admin interface, WordPress checks if it’s authorized to write files first, if not WordPress allows you to upload it through FTP. That’s nice except that the script checking authorization is bugged (and that FTP is a protocol that should died but that’s an other story ). The script checks that the user executing it is the owner of… the script file! instead of checking that it’s the owner of the theme/plugin directory. Since they do the things right in debian, the script is owned by root but executed by www-data (apache’s user). So I had to change the owner of the script to make it work.

Migration

In WordPress tools, there is a script to import data from dotclear… which does not really work. Maybe it was done for dotclear 1, or my alpha version did have a lot of changes but obviously there was a lot of differences between what the import script was excepted and what my dotclear database looked like. I customized it a bit so it was able to import my posts and comments but I had some encoding issues (mostly solved), and of course I had to upload again all the pictures, audio files and videos… It took me several evenings to do the migration. Maybe I should have done it all by myself without using the original script.

After the import, the comment count was not set on the blog posts and I saw that other people had the problem (comment count not matching the actual number of comments) at some point for a reason or an other and the solution I saw was usually a small PHP script looping on each post and executing the same SQL query. So here is the same solution but with only one SQL query, no PHP involved:

Code block

UPDATE wp_posts SET comment_count = (SELECT count(*) FROM wp_comments WHERE wp_comments.comment_post_id = wp_posts.id);

Usage

I just started using it. For the people reading my blog from the RSS feed, check out the new URL. On the overall it looks quite simple, but there are some stuff I don’t really like. For instance I have to use categories. So I put everything in the “Uncategorized” category. On dotclear I was able to say that I don’t want to use categories. Tags are way much better than categories… and they are optional.

The visual editor is quite good but the HTML editor is a “simple HTML” editor, not showing the full HTML, and switching from one editor to the other change the HTML code you may have write, and sometimes you don’t want to. For instance putting audio and video tags (HTML 5 tags) in an XHTML 1 page… is not really allowed, and switching from the HTML editor to the Visual editor simply remove them . There is no wiki syntax, I can understand that choice but I was a bit used to it.

The plugin and theme system work quite well (except for the bug I talked about earlier). For the theme I chose Garland-revisited, which is a copy of Drupal‘s default theme (I used Drupal once and I had to customized the theme for the website but I love their Garland theme, so here is a way to have it ).

I also like the idea of having fixed pages. And I should start by filling the “about” page. That’s all for today, I still have a lot of stuff to configure.

In fact, my failure today was due to some electrical problems. If you are following this blog you know that my RAID drives are plugged to the server (Debian GNU/Linux) using USB, which is an extremely bad idea (don’t do that at home ). And to add more on my stupidity, in order to reduce power consumption I changed my hard drives to laptop hard drives and have them powered through the USB hub… which was not plugged to the UPS. So today there was a power failure at home and since the server’s USB was not providing enough power, two drives went off.

Since nothing was being written when it occurred, I know that the content on every drive was still good, but mdadm reported the array as degraded and reading was not really possible anymore.

So, what to do in that case? From what I have seen, the first thing is to stop the array, then to try to reassemble it with various options (but do not try to re-add the “failing” drives). Obviously I did some mistake… So, if at some point mdadm --assemble with any kind of options does not work, re-creating the array might be your last solution. At least it worked for me.

But be careful, when creating the array, you have to provide the same options (chunk size…) as it was before, and you have to keep the drives in the SAME ORDER. And when you have drives on USB, the order is a bit random (maybe I should have looked at each disk’s UUID and write the order somewhere).

So I re created the array with the following command (DON’T FORGET “—assume-clean” otherwise mdadm will start re-synchronizing your disks and it’s something you may not want):

Code block

mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 --assume-clean /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

After doing so… it was still not working. Why? Because I didn’t gave the right disk order! With 4 drives I have something like 24 different possibilities. How to find out if it’s the right one? Well, that’s quite easy, you should be able to mount the disk . Doing a fsck might also be a good idea (don’t forget the -n option as you don’t want to write on the drive until you are sure that it’s the correct order).

I was quite lucky since I found the right one on the second try.

Related posts:

• RAID 5 software sous GNU/Linux
• La panne
• Moving a RAID array
• Restoring your RAID array

Requirements

This time my needs were a bit different than the ones for my router. I was looking for a low power consumption device with at least 512 MB of RAM, fast enough to read videos (I may use it as a HTPC in the future), fan-less and of course, it must work with Linux.

fit-PC2 specifications

So I end up with Compulab’s fit-PC2. Here are the official specs:

• CPU: Intel Atom Z530 1.6 GHz
• RAM: 1 GB DDR2-533
• HDD: 160 GB SATA 2.5”
• 1 Gigabit ethernet port
• 4 USB 2.0 ports
• 2 mini-USB 2.0 ports
• 1 mini-SD socket
• 1 IR receiver (infra-red, for remote control)
• DVI output (using a HDMI connector, HDMI to DVI cable provided)
• Power consumption: 6 W idle, 7 W reading H.264 videos, 8 W at full load
• Shipped with Ubuntu

Several flavors are available (with WiFi, without hard drive, with an Atom Z510 CPU…).

Pictures

Usage

I tried the pre-installed Ubuntu but it’s an old version (8.04). Anyway, it was working quite fine. For video playback only MPlayer was using hardware acceleration provided in fit-PC2. I played Sita Sings The Blues on it, it works wonderfully (the movie is a Full HD (1920*1080), 4.12 GB file (I was hardly able to read it on my MacBook)).

But I don’t really want to use Ubuntu. If I want a *buntu linux I would install Kubuntu instead, anyway I installed Debian. Here is the beginning of my troubles.

First, I’m using Debian’s “testing” version, which proved to be quite stable so far. Except with the new 2.6.30 Linux kernel which froze my VIA C7 powered Dedibox and is behaving weirdly on the fit-pc2. I don’t know if the problem is in linux kernel, gcc, debian… but installing 2.6.30 kernel was not a good thing on my debian boxes. I’m using the fit-pc2 remotely (ssh, music streaming, time machine backups…) and from time to time it stops to respond, but if I hit a key on the keyboard, network operations resume… very weird. A problem in power saving code? a deadlock which resolved itself when a new interruption is raised? No idea, I’m clearly not competent in that domain. Easy solution: revert to 2.6.26 kernel.

Second problem: drivers. The graphic chipset used is an Intel GMA500. Even if Intel recently helped the Open Source community by giving specifications of its chipsets, they didn’t for that one (because it’s not really an Intel one, it was developed by PowerVR (and they are much less cooperative)). Somehow Compulab/DeLL/Ubuntu managed to get a partly open source driver with hardware acceleration (that part is still proprietary), but according to various websites, the driver is crappy and not well maintained (but it looks like some people are trying to make that better). So, for now, hardware acceleration does not work easily on Debian. I hope it will work soon.

Power consumption

• Off: 0.3 W
• Idle: 9 W
• H.264 playback: <10 W
• Full load: <11 W

It seems that it consume a bit more than expected (my watt-meter is a cheap one, I’m not sure how reliable it is), or maybe figures on fit-pc2 website were given for an Atom Z510.

Note: the case can be very hot.

Conclusion

fit-PC2 is a nice bit of hardware, but if you are going to install an alternative BSD/Linux operating system, you have to know that you may get some troubles with video drivers (if you need nice graphical interface).

Few month ago I moved my RAID array from SATA to USB but USB components seems to be of lower quality than SATA ones. So from time to time I see some “USB reset” when reading/writing a lot of data, fortunately it only stop transfers for a few seconds before resuming.

Today my server was fsck‘ing the array when a disk failed badly. It looks like the external drive’s USB controller crashed. The disk was not visible anymore from my server (I had to manually restart the external drive).

Since it’s a RAID 5 array, everything was still working fine, in degraded mode:

Code block

$ sudo mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Sat Jan 19 17:03:46 2008
     Raid Level : raid5
     Array Size : 1465151808 (1397.28 GiB 1500.32 GB)
  Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent
 
    Update Time : Wed Jul 22 22:16:04 2009
          State : clean, degraded
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0
 
         Layout : left-symmetric
     Chunk Size : 64K
 
           UUID : 02918692:35547ed8:ccb6a325:e4cda885 (local to host seth)
         Events : 0.3774
 
    Number   Major   Minor   RaidDevice State
       0       8       65        0      active sync   /dev/sde1
       1       0        0        1      removed
       2       8       81        2      active sync   /dev/sdf1
       3       8       33        3      active sync   /dev/sdc1

Then adding back the missing drive:

Code block

$ sudo mdadm --manage /dev/md0 --add /dev/sdd1
mdadm: re-added /dev/sdd1
$ sudo mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Sat Jan 19 17:03:46 2008
     Raid Level : raid5
     Array Size : 1465151808 (1397.28 GiB 1500.32 GB)
  Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0
    Persistence : Superblock is persistent
 
    Update Time : Wed Jul 22 22:19:23 2009
          State : clean, degraded, recovering
 Active Devices : 3
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 1
 
         Layout : left-symmetric
     Chunk Size : 64K
 
 Rebuild Status : 0% complete
 
           UUID : 02918692:35547ed8:ccb6a325:e4cda885 (local to host seth)
         Events : 0.3780
 
    Number   Major   Minor   RaidDevice State
       0       8       65        0      active sync   /dev/sde1
       4       8       49        1      spare rebuilding   /dev/sdd1
       2       8       81        2      active sync   /dev/sdf1
       3       8       33        3      active sync   /dev/sdc1

Now I can monitor the slow process of rebuilding the array:

Code block

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd1[4] sde1[0] sdc1[3] sdf1[2]
      1465151808 blocks level 5, 64k chunk, algorithm 2 [4/3] [U_UU]
      [>....................]  recovery =  2.2% (11184896/488383936) finish=661.5min speed=12019K/sec
 
unused devices: <none>

At least 11 hours to rebuild!!! So, for people as stupid as me: don’t use USB for a RAID array if you are not obliged to, it’s not very safe and it’s a bit slow.

The purpose of Time Machine is to backup your Mac and to allow you to retrieve some files you deleted or to restore your system. Last week my MacBook’s hard drive suddenly died without a warning. Since I was moving in my new apartment, the last backup was few days before, but at least I had a quite recent backup.

The question was: does Time Machine will allow me to restore my backup as expected?

I bought a new hard drive, plugged it, boot on Mac OS X install DVD, selected Utilities/Restore system from backup… then nothing. No backup listed and the “Connect to remote disk” is grayed/disabled .

I was a bit disappointed. I launched the terminal and tried to manually mount the remote backup but I failed. This was the good solution, I just didn’t know the right command to mount a remote apple volume. After googling a bit (thanks to my iPhone) I found the right instructions:

• Create a directory where to mount the remote backup

# mkdir /Volumes/backup

• Mount remote volume

# mount_afp afp://login:password@hostname/volumename /Volumes/backup

Then relaunch the “Restore from backup” utility, the remote volume was now listed. Just had to select and start waiting (restoring the system may be quite long).

A long time ago there was several protocols to send a message from a computer to an other one. There was a protocol per network (the internet was not really born at that time). Those protocols were not compatible between each others. So for instance, if you were using FidoNet, you were not able to send emails to people using BITNET. Hopefully some people created some gateways to transfer emails from a network to an other one (but it looks like it was quite a nightmare). At some point in time, ARPANET and its email protocol became the standard and other protocols started vanishing. People were now able to communicate with each other easily.

Instant messaging

Instant messaging (IM) appeared much more recently (email started in the late 60s, late 80s for IM). Personal IM became very well known in 1996 with ICQ (I still have my ICQ account! (but nobody to talk to me on it anymore)). Then several other protocols appeared:

• Jabber (1998, free and open protocol)
• QQ (1998, closed and proprietary)
• Yahoo! Messenger (1998, closed and proprietary)
• MSN Messenger (1999, closed and proprietary)
• …

Of course you can’t chat with somebody using MSN Messenger if you use Yahoo! Messenger. ICQ was bought by AOL, AOL created AIM (AOL Instant Messenger) based on ICQ protocol. At some point people using ICQ were able to speak with people using AIM.

In 2004, XMPP, the protocol used by Jabber, became the official IETF instant messaging standard (IETF is the group defining the internet standards (like HTTP for the web, SMTP for emails, FTP for file transfer…)).

Jabber/XMPP

XMPP as been the standard for 5 years now, but still, most of my contacts are using MSN Messenger. Why? Because they also use a Microsoft Windows based computer and Microsoft MSN Messenger is the default instant messaging software installed on those computers.

XMPP allows creating gateways to other protocols (like MSN, Yahoo!, ICQ…). The situation looks a bit like the email status in the old days. But the email standard imposed itself as the killer application of ARPANET. So, what will be the thing that will wipe out all proprietary protocols and impose the XMPP open standard?

The problem with standards is that it take time to be developed. History as shown that new versions of a standard are not implemented by all software vendors quickly, nor they are deployed as fast as possible. So changes to the standard must not happen everyday and having it quite right at version 1.0 take time.

In the meantime, other protocols evolve faster because vendors have a captive market and a better control on how the software is distributed/used (and they don’t have to wait for others to implement changes).

While XMPP was on the way to be standardized, other protocols got voice then video functionalities. I had a lot of hope in 1995 when Google released it’s own IM software Google Talk. GTalk is based on XMPP and add some voice extension and video extension later.

Then AOL started an experiment to allow XMPP connections to its network, Yahoo announced that they were interested by XMPP too. The chat in Facebook uses XMPP (but the network is closed, you can only talk to Facebook users), several other community websites do the same.

Nowadays

Last week, the specifications for voice/video in XMPP were released. The biggest missing features making people stay with their proprietary IM is going to be old story soon. But I’m not sure it will be enough to see a big migration to XMPP.

Those last few years, XMPP interest increased a lot and nearly all IM vendors are now looking at XMPP… except Microsoft. It looks like you will soon have the choice between speaking to nearly everybody except MSN Messenger users or speaking only to MSN Messenger users.

Of course there are lots of softwares allowing you to use several accounts at the same time (so being connected to MSN, Yahoo! and Jabber at the same time). I also have several emails account. The difference is that from my professional email account I can send emails to everybody, same for my personal email. I’m choosing the email account I’m using depending on what my “role” is. If I want to send a message to a colleague, I will use my professional email address.

In IM, you can’t do that, except if you open several accounts on each protocol you use. Having a professional and a personal account on MSN, on Yahoo!, on AIM, on Jabber… With all those protocols I currently have 9 accounts. And you know what? I like keeping things simple. For my email addresses I have started closing several accounts, keeping only the mandatory ones (my personal email address and the ones I have to use for my job). And I would really like doing the same for IM but you know what? Here I can’t do what I want. Why? Because if I close my MSN account, I will lose contact with a lot of people.

I feel a bit like in jail. Worse, I feel like my friends are in jail too but they are saying “Where do you see a jail? There’s only walls and fences”.

Anyway, lots of people are using GMail now, and there is a chat embedded in GMail. Of course this chat uses Google Talk so it uses XMPP. Even if I don’t really like GMail, I prefer having my friends using GMail/Google Talk than Hotmail/MSN Messenger (or whatever the name of those services are this week).

ejabberd

Since XMPP is an open protocol, anybody can implement it. There are several XMPP clients (Pidgin, Adium, Kopete, Trillian, iChat…) and there are also several servers.

In XMPP servers there is well-known one: ejabberd. This server is open source and written in Erlang. Ejabberd use the power of Erlang to be fault-tolerant, redundant, scalable, <add here any cool property a server should have>.

And since XMPP is a decentralized system, I can install my own server (as I did for my email server for instance).

Installing ejabberd on debian is as easy as usual:

$ sudo apt-get install ejabberd

To configure it, you just need to change the domain name to serve in /etc/ejabberd/ejabberd.cfg. If your domain name is example.org change the following:

%% Hostname {hosts, ["example.org"]}.

And set the admin user:

%% Admin user
{acl, admin, {user, "admin_user_name", "example.org"}}.

Add a user with the following command:

$ sudo ejabberdctl register user_name example.org password

Restart the server. Done.

Of course there are a lot more parameters to change if you want to fine tune it. You may also need to create a SRV entry in your DNS if your server is not the one serving “example.org” (but “im.example.org” for example).

So my home server is running a 4 disks array (about 500 GB each, for a total of 1.5 TB available). I used SATA disks plugged in the server. For some silly reasons I wanted to move the disks out of the server and plug them using a USB interface (no, not a eSATA one, USB (I told you, it’s silly, but it could have been worse, like creating a floppy RAID array)).

I bought 4 Icy Box enclosures and a USB hub. Shut down the server, move the disks into the external enclosures, plug everything (lots of wires), switch on the server, cry.

As quite expected, it didn’t work right away. My Debian server stopped on a maintenance shell, complaining that it was not able to check /dev/md0 (the array).

No problem, I tried a simple command:

mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

Array assembled, exit the shell, Debian finished booting, everything works. But, in doubt, I did a reboot. Again, array not recognized. After a bit of googling and man reading, I tried the same command with a little option added:

mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 --update=homehost

Not really sure about what it did, but after rebooting, the array was recognized and assembled automatically.

So YES, you can change the controller interface used to plug your disks in an RAID array.

What about speed? Well… high speed is obviously not the purpose of this experiment (but at least it’s > 30 MB/s in continuous read).

What about reliability? Well… it’s Raid-5 so I might have to rebuild from time to time. So far I only get some USB reset events so transfer stall during ~20 seconds then resume. Of course I get more resets when doing a big transfer.

There are a lot of ways of fighting spam with various degrees of effectiveness, simplicity, standards respectful, etc.

For instance, there are some stupid systems like SpamCop based on user submissions. If a user tell that he received a spam from a given IP, then the IP is temporarily marked as a SPAM sender and will be blocked by mail servers using SpamCop black list. So, when your email address is hosted by an ISP used by a lot of people, of course there are emails reported as junk, and your ISP is black listed. Several mails I sent were blocked by SpamCop. Junk emails are a pain, but being blocked is way much worse. So please, don’t use system generating to many false positives.

Greylisting

Greylisting consist of sending a temporary error when an email is received, store information about this email (server IP, sender, recipient) and wait for a second attempt after a given amount of time. If the email is received again, it’s allowed.

Usually spam senders send emails to a huge list of addresses, so when they received an error, they just try with the next email address. This is very simple and effective against that kind of spam. The bad news is that emails are often delayed (the sender server may try again 5, 10, 30… minutes later). By definition emails are not instantaneous, but usually it’s quite fast, so it can be a bit slower with this solution.

SPF

SPF stands for Sender Policy Framework. It’s a way to check if the sender if allowed to send mails. For instance, a domain owner can specify in its DNS a TXT record containing IPs allowed to send emails from that domain.

I strongly suggest that you add this record if you have a mail server. For example, if you dig my domain name for TXT records you will get:

Code block

$ dig desgrange.net TXT
 
; <<>> DiG 9.5.0-P2 <<>> desgrange.net TXT
(…)
;; ANSWER SECTION:
desgrange.net.		600	IN	TXT	"v=spf1 a mx ~all"
(…)

The value v=spf1 a mx ~all means that A and MX IPs are allowed to send emails and that all other IPs should not.

tumgreyspf

In order to add greylisting and SPF to postfix, I decided to install tumgreyspf. The name is quite explicit, it does both greylisting and SPF.

Installation:

$ sudo apt-get install tumgreyspf

Tell postfix how to start tumgreyspf in /etc/postfix/master.cf by adding:

/etc/postfix/master.cf

(…)
tumgreyspf unix -       n       n       -       -       spawn
  user=tumgreyspf argv=/usr/bin/tumgreyspf

Tell postfix when to ask tumgreyspf to perform a check, set smtpd_sender_restrictions in /etc/postfix/main.cf to something looking like that:

/etc/postfix/main.cf

(…)
smtpd_sender_restrictions =
    permit_sasl_authenticated,
    reject_unauth_destination,
    check_policy_service unix:private/tumgreyspf
(…)

And now restart postfix (reload should be enough):

$ sudo /etc/init.d/postfix restart

Misc

After some tests, I found that tumgreyspf does not reject emails when SPF returns softfail but only fail and permerror.

If you configure your DNS like I did, which seems to be a quite common configuration, you may want to consider softfail has a reason to reject an email.

To change that, change line 122 of /usr/bin/tumgreyspf to the following (it’s written in python, don’t change indentation):

/usr/bin/tumgreyspf

if spfResult == 'Fail' or spfResult == 'Permerror' or spfResult == 'Softfail':

Mail Stories posts

• Installing a mail server
• Installing a webmail
• Spam reduction (this post)

Sometimes I want to check my emails but I don’t have my own computer with me, only an access to an other computer connected to the internet. So I need a webmail. For that purpose I use RoundCube, it’s an AJAX webmail written in PHP, using IMAP to retrieve emails and SMTP to send them.

Installation

In Debian, RoundCube seems not to be a very popular package, or maybe Debian people thinks the project is still too young. In fact, there is no RoundCube package for the current stable release of Debian. In “testing” there is RoundCube 0.1.1, and 0.2~stable in “unstable”. Current official release is 0.2.1.

So lets install the “unstable” package (which is quite stable as far as I know ).

First you need to add unstable repositories to APT if it’s not done already. Edit /etc/apt/sources.list and add the following lines (replace ftp.fr.debian.org with your closest mirror):

/etc/apt/sources.list

deb http://ftp.fr.debian.org/debian/ unstable main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ unstable main contrib non-free

Then tell APT to keep using stable packages by default (currently the stable release is called lenny, so set the value accordingly with your system), edit /etc/apt/apt.conf.d/70debconf and add:

/etc/apt/apt.conf.d/70debconf

(…)
APT::Default-Release "lenny";

It’s time to retrieve the list of unstable packages:

$ sudo apt-get update

Install RoundCube (RoundCube needs a database, it works with MySQL, PostgreSQL and SQLite, select the one you prefer, here I’m going to use SQLite):

Code block

$ sudo apt-get install roundcube
-> Configure database for roundcube with dbconfig-common? Yes
-> Database type to be used by roundcube: sqlite

Configuration

RoundCube should be accessible at http://mailserver.example.com/roundcube/. If it’s not the case, you may need to uncomment the two Alias directives at the beginning of /etc/roundcube/apache.conf (don’t forget to reload apache configuration before checking if it works).

Now edit /etc/roundcube/main.inc.php. Each parameter has a quite explicit comment, read them. Things you may want to change:

/etc/roundcube/main.inc.php

(…)
$rcmail_config['enable_caching'] = FALSE;
$rcmail_config['default_host'] = 'ssl://localhost:993';
$rcmail_config['default_port'] = 993;
$rcmail_config['smtp_server'] = 'localhost';
$rcmail_config['smtp_port'] = 587;
$rcmail_config['smtp_user'] = '%u';
$rcmail_config['smtp_pass'] = '%p';
$rcmail_config['create_default_folders'] = TRUE;
(…)

You should now be able to login, read and send emails.

Misc

In the parameters I changed, there is create_default_folder set to TRUE. This in order to have the default IMAP folders always here (INBOX, Drafts, Sent, Junk and Trash). By default they are all protected, so they can’t be removed.

But there is no real convention on those folder names. For instance, by default Apple Mail use some different directory names. But you can tell Apple Mail to use the ones you want. First you need to select the directory ({Mailbox} in Apple Mail):

And then associate it with the right meaning:

For other IMAP clients, see the respective documentation. For instance, here is how to change the trash folder on Thunderbird.

If you have an iPhone, you only need to synchronize it again, the iPhone uses Mail.app configuration (don’t forget to check Mail Accounts in Info/Advanced/Replace information on this iPhone: in iTunes before synchronizing).

Note: since I wrote this post, Debian’s RoundCube unstable package has been updated to version 0.2.1 but testing package is still version 0.1.1.

Mail Stories posts

• Installing a mail server
• Installing a webmail (this post)
• Spam reduction

Disclaimer

Configuring its own mail server can be fun, but remember that there are a lot of bad guys on the internet. I’m no expert on mail server configuration, don’t forget that you must not allow relaying mails to domains you don’t own and from people you haven’t authenticated in order to avoid spam relay. Of course, keep your softwares (operating system, mail server, etc.) up to date in order to keep your computer safe from being cracked.

Goal

I have a domain name (desgrange.net as you can see) and I would like to use it for my mails. It’s not that I don’t trust service providers, but I would like to be in charge of the mail server configuration (so I would be able to tune it as I want).

For instance, I want to be able to send mails using SMTP over SSL and retrieve emails using IMAP over SSL.

For those tasks I chose:

• Debian as operating system (tested on Debian 5.0 and Debian testing)
• Postfix (2.5.5) as SMTP server
• Courier-imap (4.4.0) as IMAP server

In this tutorial I assume that I’m installing a mail server for domain example.com on a computer named mailserver. Allowed users are the standard unix users created on the mail server.

DNS

First make sure that your DNS has your MX record pointing to your mail server. See with your domain name provider for that part. I’m using OVH so I did it on their website (which sucks a bit for that).

Postfix

Postfix is a well known SMTP server. As a lot of mail servers, it has several hundreds parameters, but here, all of them have good default values, so we just need to change some of them to get it working like we want. Let’s start with postfix installation:

Code block

$ sudo apt-get install postfix
-> General type of mail configuration: internet with smarthost
-> System mail name: example.com
-> SMTP relay host: <blank>

Now we need to install the authentication system saslauthd.

Code block

$ sudo apt-get install sasl2-bin libsasl2-modules

As told by the installation process, to enable saslauthd, edit /etc/default/saslauthd and set START=yes.

To tell postfix to use saslauthd, create a file /etc/postfix/sasl/smtpd.conf containing:

/etc/postfix/sasl/smtpd.conf

pwcheck_method: saslauthd
mech_list: PLAIN LOGIN

As postfix is running chrooted, to allow it to talk to saslauthd we need to move some files (or change the directory specified in OPTIONS in /etc/default/saslauthd):

Code block

$ sudo mkdir -p /var/spool/postfix/var/run/
$ sudo mv /var/run/saslauthd /var/spool/postfix/var/run/
$ sudo ln -s /var/spool/postfix/var/run/saslauthd/ /var/run/saslauthd
$ sudo adduser postfix sasl

Postfix is configured to run on the standard port 25 so other mail servers can transfer mails to you. This postfix process can’t use SSL only because other mail servers may not support it. So we need to start an other postfix process running over SSL, so we can use this one when we want to send an email. To allow this process to start, just uncomment the submission lines in /etc/postfix/master.cf.

Check that /etc/mailname contains your domain name. It’s used by postfix for some parameters (seems to be debian specific).

Most of postfix configuration is done in /etc/postfix/main.cf, you can edit this file directly or use the following commands:

Code block

$ sudo postconf -e 'smtpd_tls_security_level = may'
$ sudo postconf -e 'smtpd_tls_mandatory_ciphers = high'
$ sudo postconf -e 'smtpd_tls_mandatory_exclude_ciphers = aNULL'
$ sudo postconf -e 'smtpd_tls_mandatory_protocols = TLSv1'
$ sudo postconf -e 'mydestination = $myhostname localhost.$mydomain localhost $mydomain'
$ sudo postconf -e 'smtpd_recipient_restrictions = permit_sasl_authenticated,reject_unauth_destination'
$ sudo postconf -e 'relay_domains ='
$ sudo postconf -e 'home_mailbox = Maildir/'
$ sudo postconf -e 'mailbox_command ='

Line 1 tells postfix to use TLS on standard SMTP port (25) if remote server support it.
Line 2 restricts TLS ciphers to only “HIGH” ciphers.
Line 3 excludes aNULL cipher (no encryption).
Line 4 tells to use TLSv1 as protocol for the postfix over SSL process.
Line 5 list domain names for which the server will accept mails.
Line 6 tells postfix to send mails only from authenticated users or to domains postfix is in charge of.
Line 7 postfix must not relay mails to other domains.
Line 8 and 9 are for Courier which use a specific type of mailboxes.

You can have a look at the postfix configuration parameters documentation.

Courier-IMAP

Courier is a full mail server but here I’m going to use only Courier’s IMAP server component. Installation:

Code block

$ sudo apt-get install courier-imap-ssl
-> Create directories for web-based administration?: no

By default IMAP and IMAP over SSL processes are started but I don’t want the first one. To disable the standard IMAP process, set IMAPDSTART to NO in /etc/courier/imapd.

Some parameters have to be changed for the IMAP over SSL process, edit the file /etc/courier/imapd-ssl and set the following values:

/etc/courier/imapd-ssl

(…)
IMAP_TLS_REQUIRED=1
TLS_PROTOCOL=SSL23
TLS_STARTTLS_PROTOCOL=SSL23
TLS_CIPHER_LIST=HIGH
(…)

For TLS_PROTOCOL and TLS_STARTTLS_PROTOCOL I use SSL23 because I’m using Apple Mail and it does not handle TLS very well. If you use a mail client supporting TLS, replace SSL23 with TLS1.

Finishing the installation

First we need to restart everything:

Code block

$ sudo /etc/init.d/saslauthd restart
$ sudo /etc/init.d/postfix restart
$ sudo /etc/init.d/courier-authdaemon restart
$ sudo /etc/init.d/courier-imap restart
$ sudo /etc/init.d/courier-imap-ssl restart

If you type sudo netstat -lntp you should see something like that:

Code block

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      12718/master
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      12718/master
tcp6       0      0 :::993                  :::*                    LISTEN      12440/couriertcpd

There are two processes for postfix (master on ports 25 and 587 (SSL)) and one for courier on port 993.

Create your mailbox. You may want to initialize your own mailbox directory on the server:

Code block

$ maildirmake ~/Maildir

Certificates

I didn’t say anything about SSL certificates, you need certificates for all your SSL connections, here some default ones are created, but when you connect with your mail client, it may complain about the certificate not being trustworthy.

It may prevent you to read/send mails (for instance I had to tell Apple Mail to trust the certificate in order to be able to send emails).

Trouble shooting

While configuring my mail server I ran in several problems, mainly due to Apple Mail, TLS and certificates. So, if you have any trouble, don’t forget to have a look at log files in /var/log/, use netstat and ps to be sure that your servers are running and listening on the right ports.

You may even try to manually connect to the IMAP server with a command like this one:

Code block

$ openssl s_client -tls1 -connect mailserver:993 -state -debug

Have a look to the IMAP commands to check if it works.

Some of the errors I had:

• SASL authentication failure: cannot connect to saslauthd server: No such file or directory because I didn’t had moved saslauthd files to /var/spool/postfix/var/run.
• imapd-ssl: Unexpected SSL connection shutdown because Apple Mail was not trusting the certificate.
• imapd-ssl: couriertls: connect: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number because Apple Mail does not support SSL3/TLS1 on IMAP.

Update (2010-05-09):

My father’s mail client was displaying the following error message:

Code block

Filesystem notification initialization error -- contact your mail administrator (check for configuration errors with the FAM/Gamin library)

A quick solution to solve that is to replace FAM with Gamin:

Code block

$ sudo apt-get install gamin

And to restart courier.

Links

Some useful resources:

• Postfix Howtos and FAQs
• Postfix Documentation
• A tutorial on SMTP authentication over SSL
• A tutotial including a postfix/courier-imap installation

Mail Stories posts

• Installing a mail server (this post)
• Installing a webmail
• Spam reduction

One great feature introduced in Mac OS X 10.5 (Leopard) is Time Machine.

Time Machine is a backup/restore system. As a lot of backup systems it creates an incremental backup. You can create your backup to an attached hard drive, a Time Capsule or an Xserve.

If, like me, you would like to backup your Mac on a Linux server, there is a solution. It’s not perfect yet, but so far (I’m using it for a week now) it’s working quite well.

I followed the instruction written on how to create a Time Machine Server on Debian (it’s in french), which is mostly a translation of this article on how to create a Time Machine Server on Ubuntu (in english).

It’s quite easy, you just have to follow instructions. Some tips:

• On my Debian installation there’s a problem with cracklib2-dev package, I just had to install libcrack2-dev instead.
• Don’t copy/paste the XML file from the tutorial’s french version (the blog’s style sheet messed up the quotes).
• If you have to manually create a sparse bundle disk image, it seems that the size is the size of your “virtual” backup drive. For instance, I have 300 GB available on my server, I selected 200 GB as image size, so Time Machine should not use all the space available on my server but restrict itself to 200 GB (I hope so, it’s not clear as Time Machine is still telling me that my hard drive is about 1.3 TB with 300 GB available, but the sparse bundle disk image is telling me that there is 200 GB with 100 GB available (I already have 100 GB of backup)).
• On Debian, doing an update will erase your home made netatalk package. In order to prevent that you can use the following command (see Preventing Debian package upgrades):

# echo "netatalk hold" | dpkg --set-selections

I said it was not perfect because from time to time Time Machine failed while backuping, so I have to launch a new backup (ok, it’s just 2 clicks). The other problem is that my home folder is encrypted with FileVault. In that case Time Machine does not backup my home folder when I’m connected. When I logoff, FileVault do some stuff then my home folder is backuped. But now with a remote backup, the backup part is not done anymore. I have to connect with an other account and launch a backup.

iTunes Server

Like a lot of Mac users, I’m using iTunes to listen to my music. The overall software is quite good but there are several things that are just wrong. By default iTunes is only able to read mp3 and Apple‘s own audio codecs. I convert all my CDs to FLAC because it’s a lossless and open source codec.

The other problem I have is that my MacBook has a small hard drive so I can’t keep all my music on it (especially if encoded in FLAC).

The solution is to use Firefly and have it sharing my music on my local network. The music is decoded by Firefly then sent over the network, so iTunes doesn’t have to know how to decode the original file.

Installing Firefly on Debian is quite easy:

$ sudo apt-get install mt-daapd

Then edit /etc/mt-daapd.conf in order to configure it a bit. There is also a web interface to configure Firefly, connect on your server on port 3689 with your web browser (the main thing you have to set is the directory where your music is).

Now open iTunes, you should see a Firefly share in the left panel:

Some drawbacks with this method. On iTunes you only see remote playlists, so some functionalities are not working anymore (changing songs metadata, genius, sound check, creating your own local playlist from the remote one, copying a song to your iPod/iPhone (retrieving a song in local)).