Don’t upgrade specific packages

From time to time I don’t want to upgrade some packages on my debian server (because there is a bug with the new version or some other reasons) but I still want to do the other updates.

Apt is able to handle that but it’s not really easy to do (and to remember). There is a utility called wajig that simplify that a lot (wajig aim is to do all the package handling stuff simpler than apt and dpkg).

Install wajig as usual:

$ sudo apt-get install wajig

And here is how to use it for holding a package, unholding it or list all packages that are on hold:

$ sudo wajig hold
$ sudo wajig unhold
$ sudo wajig list-hold
Posted in Uncategorized. Tags: , . No Comments »

Multiple identities in Apple Mail 5

This is a “reminder post” to remind myself how to do that in the future, see links at the bottom for sources.

If you have email address aliases you might want to configure them in Apple Mail so you can also use the aliases to send emails. There are two ways of doing so.

Simple and easy but incomplete solution

It’s not intuitive since it’s not explained in Mail interface, but you can simply do that by writting several email addresses in the “Email Address” field of your account information. You just need to separate them with comma:

Then when you write an email, you can select your new identity:

A bit more complicated but complete solution

But as you can see in the previous method, what you entered in “Full Name” field is used for all identities and the purpose of having several identities may be to have also different full names.

In the folder ~/Library/Mail/V2/MailData, there is a file named Accounts.plist. Close Mail, open the file (with Xcode for instance) and in the element MailAccounts/Item X (replace X with the number corresponding to the email account you want to add an alias to) add “EmailAliases” array with dictionaries containing your new aliases :

Now when you write an email, the name you put is used:

Sources

  • http://simon.heimlicher.com/hints/macosx/multiple_identities_mail
  • http://blog.charismaticdog.com/2008/12/10/super-handy-solution-to-mailapp-conundrum
  • http://jonathan.tron.name/2006/11/05/multiple-identities-in-one-account-with-apple-mail-app
  • http://www.creativetechs.com/iq/use_multiple_email_aliases_in_apple_mail.html

Sony DSC-HX5V

For my next holidays I bought a new camera (since the previous one died last year) and I wanted a quite cheap camera (I suck at taking pictures) with a GPS in order to have the coordinates of the shots easily.

I decided to buy a Sony DSC-HX5V which is well rated for its pictures quality.

This post is about the GPS receiver inside the camera. If you are like me, you don’t like/want to install extra software for things like a camera. Usually camera makers provide poor softwares and they are quite always not compatible with plenty of things. So I didn’t installed the software provided with the camera.

I tried using the camera a bit and I noticed that acquiring GPS signal was painfully slow, which is quite normal for a GPS when it’s not assisted in any kind of way to know its position or the satellites positions (while a smartphone usually can know its approximate position from the cell tower it’s connected to and/or can download satellites positions from a website).

After looking a bit in the camera menu and looking on the internet, it’s possible to add a file with satellites positions for the next 30 days in the memory card and the camera use it to help locating itself.

From nearly 2 minutes to find it’s position it dropped to less than 20 seconds, which is now acceptable.

The software provided by Sony does that (writing the satellites position on the memory card) at least the Windows version. On Mac OS it doesn’t seem to work and of course, there is no version of this software running on Linux.

The manual way to do that is to:

This procedure is from Henrik Brix Andersen’s blog. Check it to find a perl script he did to automate that. One guy provided a simple application to do that automatically each time you plug your camera on Mac OS: https://code.google.com/p/gpsassist-update/.

Posted in Uncategorized. Tags: , , , . No Comments »

svn to git

Here is a quick way to migrate a subversion repository to git.

1 $ mkdir my-project
2 $ cd my-project
3 $ git svn init --stdlayout https://www.example.com/svn/my-project
4 $ git svn fetch --authors-file=authors.txt

That’s pretty much it. my-repo is now a fully functional git repository.

Notes:

  • This is for a subversion repository that contains one project and have the standard trunk/tags/branches layout.
    • If you have a different layout, on line 3 replace --stdlayout with something like -t tags -b branches -T trunk (and replacing tags/branches/trunk by their actual values).
    • If you have several projects you need to filter them out from subversion (dump/filter/load filtered dump in a new repository).
  • Line 3 also works if you have your repository locally, replace the repository URL with the actual path: git svn init --stdlayout file:///path/to/repository.
  • On line 4 --authors-file is optional. This option tells git to remap svn commit authors to something else:
    username = User Name <user.name@example.com>
    foobar = Foo Bar <foo.bar@example.com>
Posted in Uncategorized. Tags: , , . No Comments »

Restore grub

My server was installed a long time ago and it seems that the partitions were created in a way that grub does not like. And when grub does not like something you have good chances of having a computer not booting.

Since I have the problem from time to time, let’s write a reminder on how to fix this.

Sometime there are grub updates, and on the next reboot, well, nothing happens. Hopefully, my hosting provider provides an option to boot my server on a Ubuntu live CD (my server is running Debian but that’s fine).

When Ubuntu has finished booting, I can ssh on it and list my hard drive partitions:

# fdisk -l /dev/sda
(…)
 Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          12       96389+  83  Linux
/dev/sda2              13       19324   155123640   83  Linux
/dev/sda3           19325       19454     1044225   83  Linux

Here sda1 is my boot partition, sda2 is my root partition ans sda3 is swap. Let’s mount that:

# mount /dev/sda2 /mnt
# mount /dev/sda1 /mnt/boot

Now we have to change the “root” directory from the live CD to my server’s disk so we can simulate being running on my server’s disk:

# chroot /mnt

And now we can reinstall grub:

# grub-install /dev/sda

In my case I have to give the --force option to install grub. That’s all, I can now restart my Debian server.

Posted in Uncategorized. Tags: , , , . No Comments »

Do not check email adresses!

You may have noticed that on some rare websites (≈99.999 % of internet websites), you have to provide an email address when you register. And most of the time the application running the website check if the email address you entered is valid.

But you know what? Most software developers (you know, the guys staying all day long in front of computers, the guys you call when you have a problem with your computer because they have some magical power that allows them to fix your problems) don’t know what a valid email address is!

Why do I say such things? Because each time I register on a website, it complains that I should enter a valid email address… but my email address is valid actually! What I’m complaining about is not that they check email addresses of course, I’m complaining about the fact they are rejecting valid ones. Like for an email spam filter, you don’t care that some spam emails still reach your inbox, but you hate when a valid email is dropped because it was considered spam.

Unfortunately, most email address filters that have been written are rejecting a lot of valid email addresses. Why? Because checking that an email address has a valid format is very difficult. Have a look at this summary of RFC’s email address format, you will understand what I mean.

So, if you are a software developer and you have to check the validity of email addresses, don’t bother much checking the email address format (or the strict minimum like “there is a ‘@’ and a ‘.’) because even an email address that have a valid format may be invalid (in the way that the corresponding email account does not exists).

For registration process it’s easily solved by asking the user to click on a confirmation link you send them to the given email address. Only then you will know that the email address is really valid.

Logging in Weblogic console with Log4J

If you have developed a JEE web application using Log4J for logging and have it deployed on a WebLogic application server, you may wonder how to display the logs in WebLogic console:

Preparation

You simply need to create and add a Log4J appender. This appender will redirect Log4J events to WebLogic by using the NonCatalogLogger class. You can found this class in wls-api.jar or wlclient.jar (depending on your WebLogic version) from your WebLogic’s lib directory. For instance if you are using maven, you need to add one the following dependencies in your maven’s pom.xml (enter the version corresponding to your WebLogic installation):

<dependency>
    <groupId>weblogic</groupId>
    <artifactId>wlclient</artifactId>
    <version>10.3</version>
    <scope>provided</scope>
</dependency>

or

<dependency>
    <groupId>weblogic</groupId>
    <artifactId>wls-api</artifactId>
    <version>10.0</version>
    <scope>provided</scope>
</dependency>

Obviously WebLogic JARs are not in official Maven repositories (due to license/distribution restrictions, proprietary softwares always here to hassle you). So type the following command in your shell to add the API in your local maven repository:

mvn install:install-file -DgroupId=weblogic -DartifactId=wlclient -Dversion=10.3 -Dpackaging=jar -Dfile=wlclient.jar

or

mvn install:install-file -DgroupId=weblogic -DartifactId=wls-api -Dversion=10.0 -Dpackaging=jar -Dfile=wls-api.jar
Creating the appender

The appender needs to implement Log4J’s Appender interface, but it’s more convenient to extends AppenderSkeleton. WebLogic’s NonCatalogLogger class has some “debug”, “info”… methods like Log4J so the appender is just going to map one to the other.

Since you may deploy your application on something else than WebLogic (for instance I usually use Tomcat and/or Jetty for development/testing) you don’t want have it crashing your application because WebLogic classes are not here. The appender can check if the class is in the classpath (using Class.forName()) and do nothing if the NonCatalogLogger is not here.

In WebLogic’s console, there is a “Subsystem” column, we can set it in the appender to display the application name.

package sample.project;
 
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.Level;
import org.apache.log4j.spi.LoggingEvent;
 
import weblogic.logging.NonCatalogLogger;
 
public class WeblogicAppender extends AppenderSkeleton {
    private static final String SUBSYSTEM = "SampleProject";
    private NonCatalogLogger logger;
 
    public WeblogicAppender() {
        try {
            Class.forName("weblogic.logging.NonCatalogLogger");
            logger = new NonCatalogLogger(SUBSYSTEM);
        } catch (ClassNotFoundException e) {
            // Not running on WebLogic server.
        }
    }
 
    @Override
    protected void append(LoggingEvent event) {
        if (logger == null) {
            return;
        }
        if (Level.TRACE.equals(event.getLevel())) {
            logger.trace(getMessage(event), getThrowable(event));
        } else if (Level.DEBUG.equals(event.getLevel())) {
            logger.debug(getMessage(event), getThrowable(event));
        } else if (Level.INFO.equals(event.getLevel())) {
            logger.info(getMessage(event), getThrowable(event));
        } else if (Level.WARN.equals(event.getLevel())) {
            logger.warning(getMessage(event), getThrowable(event));
        } else if (Level.ERROR.equals(event.getLevel())) {
            logger.error(getMessage(event), getThrowable(event));
        } else if (Level.FATAL.equals(event.getLevel())) {
            logger.critical(getMessage(event), getThrowable(event));
        }
    }
 
    @Override
    public void close() {
        // Nothing to do here.
    }
 
    @Override
    public boolean requiresLayout() {
        return false;
    }
 
    private String getMessage(LoggingEvent event) {
        return String.valueOf(event.getMessage());
    }
 
    private Throwable getThrowable(LoggingEvent event) {
        if (event.getThrowableInformation() != null) {
            return event.getThrowableInformation().getThrowable();
        } else {
            return null;
        }
    }
}
Log4J configuration

In your log4j.xml just define a new appender using the above class and add it to the root logger. Here is an example:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
	<appender name="console" class="org.apache.log4j.ConsoleAppender">
		<param name="Target" value="System.out" />
		<layout class="org.apache.log4j.PatternLayout">
			<param name="ConversionPattern" value="%-5p %c{1} - %m%n" />
		</layout>
	</appender>
 
	<appender name="weblogic" class="sample.project.WeblogicAppender" />
 
	<root>
		<priority value="debug" />
		<appender-ref ref="console" />
		<appender-ref ref="weblogic" />
	</root>
</log4j:configuration>
Check

I have created a simple servlet logging some messages in debug, info and warn levels, let’s call it and see what happens:

It works! (OK, I was not going to write all that just to show something that does not work ;-) ). But be careful, as you can see, the debug message is not displayed even after setting Log4J’s level to debug. My WebLogic console is configured to display higher level messages. Don’t forget to check that it’s logging at the right level for you.

Notes

All source code in this post is in public domain, do what ever you want with it. I did it quickly so it may not work as you want, you are strongly advised to adapt it to your needs. I’m not mastering WebLogic at all (in fact I don’t really like that application server) so it may not be the best way to do it (and usually you should not need to do such things), and it is not at all certified in any way to be “production ready”.

Losing 2 disks on a RAID-5 array

Today I was quite disappointed when I saw that my RAID-5 array had suddenly lost 2 out of 4 drives. As you may know, losing 1 drive on RAID-5 is OK, losing 2 is not ok at all, it usually means that you have lost all your data.

In fact, my failure today was due to some electrical problems. If you are following this blog you know that my RAID drives are plugged to the server (Debian GNU/Linux) using USB, which is an extremely bad idea (don’t do that at home ;-) ). And to add more on my stupidity, in order to reduce power consumption I changed my hard drives to laptop hard drives and have them powered through the USB hub… which was not plugged to the UPS. So today there was a power failure at home and since the server’s USB was not providing enough power, two drives went off.

Since nothing was being written when it occurred, I know that the content on every drive was still good, but mdadm reported the array as degraded and reading was not really possible anymore.

So, what to do in that case? From what I have seen, the first thing is to stop the array, then to try to reassemble it with various options (but do not try to re-add the “failing” drives). Obviously I did some mistake… So, if at some point mdadm --assemble with any kind of options does not work, re-creating the array might be your last solution. At least it worked for me.

But be careful, when creating the array, you have to provide the same options (chunk size…) as it was before, and you have to keep the drives in the SAME ORDER. And when you have drives on USB, the order is a bit random (maybe I should have looked at each disk’s UUID and write the order somewhere).

So I re created the array with the following command (DON’T FORGET “—assume-clean” otherwise mdadm will start re-synchronizing your disks and it’s something you may not want):

mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 --assume-clean /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

After doing so… it was still not working. Why? Because I didn’t gave the right disk order! With 4 drives I have something like 24 different possibilities. How to find out if it’s the right one? Well, that’s quite easy, you should be able to mount the disk ;-) . Doing a fsck might also be a good idea (don’t forget the -n option as you don’t want to write on the drive until you are sure that it’s the correct order).

I was quite lucky since I found the right one on the second try.

Related posts:

Bridge between two LANs with OpenBSD

My router is sharing the internet connection for an ethernet network and a WiFi network. Both of them where on their own sub-networks.

On my iPhone (using the WiFi network), I have the Remote application, allowing to control the iTunes on my Mac (which is using the ethernet network). Since they are not in the same network, it doesn’t work (even if routes are properly specified). The solution is to create a bridge for those two networks.

I already blogged about how I configured WiFi on my OpenBSD router.

Bridge configuration

Here is the configuration of the interfaces (rl0 if the ethernet interface ans rum0 if the WiFi interface):

inet 192.168.2.254 255.255.255.0 NONE

Nothing specific as expected.

up media autoselect mode 11g mediaopt hostap nwid <SSID> wpa wpaprotos wpa2 wpaakms psk wpapsk <SHARED KEY>

Here there is a slight modification, an IP address is not needed anymore. Bridge configuration:

add rl1
add rum0
up
PF

Be sure that PF is allowing packets between the two interfaces, in /etc/pf.conf you should have something like this:

int_if="rl1"
wlan_if="rum0"
 
pass quick on $int_if no state
pass quick on $wlan_if no state

It’s a bit simplistic, you may write more sophisticated filtering rules depending on your needs.

DHCP

My router is also acting as a DHCP for ethernet and WiFi devices. To activate DHCP, add the following line in /etc/rc.conf.local:

dhcpd_flags=""

Tell dhcpd to listen on rl1 only (rum0 does not have any IP so we don’t have to bind dhcpd to it):

dhcpd configuration:

shared-network LAN {
        option domain-name "example.net";
        option domain-name-servers <primary_dns_ip>, <secondary_dns_ip>;
 
        subnet 192.168.2.0 netmask 255.255.255.0 {
                option routers 192.168.2.254;
                range 192.168.2.32 192.168.2.127;
        }
}
Links
Posted in Uncategorized. Tags: , , , , , . No Comments »

OpenBSD and PPPoE

I’m using a new ISP (since I moved in a different country) which does not provide a DSL modem as part of the subscription. So I bought the first ADSL2+ modem I found that was not also a router (since I have my own router).

I ended up with a D-Link DSL-320T. I was a bit disappointed to see that the modem DOES some routing… but quite poorly. I tried several configurations and my conclusion is that this modem is severely bugged. It’s based on an old BusyBox 0.60 (you can telnet the modem to see that and do some stuff manually (if you manage to…)). I went on D-Link website to find firmware updates… the firmware loaded in the modem is more recent that the ones I found on the website! Anyway, after some research, it looks like D-Link people have no clue about how to manage version numbers (it’s a complete mess), but it’s not a problem since the modem does not want to load any firmware (there is something in the interface to do that but it did nothing when I tried).

At some point I find out that the modem has a “bridged” mode, to it will does mostly nothing and I will have to do the authentication with the ISP on my OpenBSD 4.5 router.

PPPoE

Configuring PPPoE on OpenBSD is quite easy. The modem is connected to the rl0 interface, first we need to create a configuration file /etc/hostname.pppoe0 for the new PPPoE interface pppoe0:

inet 0.0.0.0 255.255.255.255 NONE pppoedev rl0 authproto pap authname LOGIN authkey PASSWORD up
dest 0.0.0.1
!/sbin/route add default 0.0.0.1

Replace LOGIN and PASSWORD with the credentials given by your ISP. The rl0 interface does not need any configuration except telling that the interface must be started. /etc/hostname.rl0 must contain only:

Restart network interfaces with the following command:

# sh /etc/netstart

ifconfig should now include pppoe0 configuration.

NAT and PF

I saw on some forums/mailing lists that since PF is started before the pppoe0 interface, PF might block the connection. I’m not having the problem right now, maybe for older versions of OpenBSD. Anyway, I had a different one. When PF starts, the pppoe0 interface does not have yet retrieve an IP, so PF is using “0.0.0.0”.

For instance in /etc/pf.conf, I had the following lines to create a NAT between pppoe0 and rl1 (rl1 is the interface on my local network):

ext_if="pppoe0"
int_if="rl1"
 
nat on $ext_if inet from $int_if:network to any -> $ext_if

In order to tell PF to monitor the external interface’s IP, it just needs to be put between brackets, so the NAT command becomes:

nat on $ext_if inet from $int_if:network to any -> ($ext_if)
Links