Fix email date for Mail.app

After moving my emails from one server to an other with a simple cp command I realized later that lots of my emails where showing in Mail.app with the same date (which appears to be the date I did the migration).

The thing is Mail.app is not using the Date header in the email but the email's creation date on the IMAP server. This date should be the date the server actually received the email. This prevent from receiving emails in the past or the future because the sender's computer date is wrong.

So what Mail.app does is pretty clever.

But when moving the files, the creation date changed for now. I didn't noticed it at first because once Mail.app retrieved an email, it keeps the information and don't check the server value anymore. But when I reinstalled my computer, after synchronizing all my emails again, I saw that the dates were wrong.

Here is an article I found linking to a nice script:
http://www.gabrielserafini.com/blog/2006/12/18/fixing-mailapps-imap-date-problem-mostlygeek/

The script basically read the Date header from the email and set it as creation date on the file. I ran the original script in my Maildir (containing ~120,000 emails) on my server and it worked quite well except for some emails that were ignored and some others with invalid dates:

date: invalid date `Apr 03 07 12:46'
./fix_imap_time_for_apple_mail_app.sh: 61: [: Illegal number: 
1300235227.M261107P15760V0000000000000802I000A93BE_307.zed,S=159255:2,S from 201201302339 to 
touch: invalid date format `'

So I did some changes to the script to make it work on all my emails:

#!/bin/bash
#
# Date : July 4th, 2005
# Author: Benson Wong
# tummytech@gmail.com
#
# This shell script corrects email messages where the file system
# date does not match the Date: header in the email.
#
# This will fix problems with mail clients like Apple's mail.app
# which uses the file system timestamp resulting in emails with the
# wrong file system timestamp to display the wrong received date
#
# This script has to be run by a user [root] with the
# necessary privileges to read/modify files in a user's Maildir.
#

function usage() {
  if [ "$1" != "" ]; then
    echo "$1"
  fi
  echo "Usage: $0 /path/to/user/Maildir"
  exit 1
}

function email_date() {
  local DATELINE=`grep -e "^Date: " "$1" | head -1`
  local DELIVERYDATELINE=`grep -e "^Delivery-date: " "$1" | head -1`
  if [ -n "$DELIVERYDATELINE" ]; then
    local DATELINE="${DELIVERYDATELINE/elivery-d/}"
  fi

  # Fucked up date like Mon, 03 Nov 03 11:37:04 Romance Standard Time
  local regex='^Date: ([A-Za-z]{3}, [0-9]{2} [A-Za-z]{3} [0-9]{2,4} [0-9]{1,2}:[0-9]{2}:[0-9]{2}) ([A-Za-z ]*)$'
  if [[ $DATELINE =~ $regex ]]; then
    EDATE=`date -d "${BASH_REMATCH[1]}" "+%Y%m%d%H%M"`
    return 0
  fi

  # Missing "+" before timezone
  local regex='^Date: ([A-Za-z]*, [0-9]* [A-Za-z]* [0-9]{4} [0-9]{1,2}:[0-9]{2}:[0-9]{2}) ([0-9]{4})$'
  if [[ $DATELINE =~ $regex ]]; then
    EDATE=`date -d "${BASH_REMATCH[1]} +${BASH_REMATCH[2]}" "+%Y%m%d%H%M"`
    return 0
  fi

  # Remove day of the week
  local regex='^Date: ([A-Za-z]*,) (.*)$'
  if [[ $DATELINE =~ $regex ]]; then
    EDATE=`date -d "${BASH_REMATCH[2]}" "+%Y%m%d%H%M"`
    return 0
  fi

  local regex='^Date: (.*)$'
  if [[ $DATELINE =~ $regex ]]; then
    EDATE=`date -d "${BASH_REMATCH[1]}" "+%Y%m%d%H%M"`
    return 0
  fi
}

MDIR_PATH="$1"

[ $# -lt 1 ] && usage
[ ! -d "$MDIR_PATH" ] && usage "Error: $MDIR_PATH does not exist"
[ ! -r "$MDIR_PATH" ] && usage "Error: $MDIR_PATH is not readable"
[ ! -w "$MDIR_PATH" ] && usage "Error: $MDIR_PATH is not writable"

# set the internal field separator to the newline character
# instead of the default "".
# This is required for handling filenames and directories with spaces
IFS="
"
set -f
echo "start"
# Find all emails
for i in `find $MDIR_PATH -type f | egrep -v "(courierimap|maildirsize|maildirfolder)"`; do
  email_date "$i"
  if [ -z "$EDATE" ]; then
    echo ""
    echo "Unparsable date for" `basename $i`
    continue
  fi
  FDATE=`ls -l --time-style=long-iso "$i" | awk '{print $6,$7}'`
  # Reformat the date for touch.
  ODATE=`date -d "$FDATE" "+%Y%m%d%H%M"`
  if [ "$EDATE" -eq "$ODATE" ]; then
    # Skip it if the times are correct.
    echo -n "."
    continue
  fi
  echo ""
  echo `basename $i` "from $ODATE to $EDATE"
  touch -c -t "$EDATE" "$i"
done

echo ""
echo "done"

My modified version is available on GitHub, I contacted the original author to know about the license but I'm still waiting for an answer.

When everything is done, remove your mail account from Mail.app (check that the corresponding folder in ~/Library/Mail/ is deleted, deleting ~/Library/Mail/V2/MailData/Envelope Index* might be a good idea too) then add your account back again.

Next time I have to migrate emails from one server to an other I may consider using imapsync:
http://mactip.blogspot.fr/2008/04/fixing-mailapps-imap-date-problem.html

Comments Add one by emailing me.