July, 2008

Miscellaneous — 4 Comments
24
Jul 08

Databases: “id” attributes are a bad idea

Never, never ever have a column named id in your relational database. They work most of the time, but it such a common used name that it is likely that it will clash with something else. Want an example? Here it is: PHP behaves oddly when fetching rows. In your database:

create table foo ( id bigint not null primary key );
insert into foo ( id ) values ( 1 );
insert into foo ( id ) values ( 2 );
insert into foo ( id ) values ( 3 );

Then the following snippet:

$result = pg_fetch_all(pg_query('select * from foo'));
echo $result[0]['id'];

will not work at all. My guess is that some PHP internal code or its database functions use id for their own purposes, so there is no way of directly retrieving values in columns with such a name. I was stuck yesterday for an hour with an absurd thing like this :-(

Systems — No Comments
15
Jul 08

Quick reminder: Resize snapshotted LVM volumes

Just in case you did not know about that, I spent some minutes today figuring out how to resize a LVM logical volume there are snapshots of it: the volume cannot be active. This is a pity if your root filesystem is inside a logical volume. Fortunately, you can stop the bootup process of recent Debian installations in the initial RAM disk by passing break=mount; unfortunately this minimal system has the vgchange tool (needed to start/stop volumes) but it does not have lvresize (guess what it is needed for?).

Finally I ended up firing up the bootup process from a Debian “netinstall” CD, entered rescue in the bootloader prompt and switched to the second console just after the installer detects hardware and before it mounts disks. Then it was only a matter of typing:

 # vgchange -an vg
 # lvresize -L +300M vg/base

Then reboot the installed system and make the XFS filesystem grow with:

 # xfs_growfs /

…and you are done :)

Systems — No Comments
4
Jul 08

Maintaining your LVM snapshots clean

As I have written before snapshots is a extremely fast and convenient way of perfoming live backups which can be mounted and handled like any other filesystem. But I also told you that LVM snaphots could render themselves as unusable when they run out of free blocks. Today I hacked up a quick script which automagically increments sizes of snapshots logical volumes. Just drop the following snippet into /etc/cron.hourly and relax:

#! /bin/bash

THRESHOLD="80"
INCREMENT="15"

set -e
IFS=':'

/sbin/lvs --noheadings --units M --separator : | \\
while read lv vg attr lsize origin snapp move log copyp
do
  # Check whether this is a snapshot or not
  [ "${snapp}" ] || continue

  snapp=${snapp%.*}
  lv=${lv// /}

  # Check whether the thing needs resizing
  [ "${snapp}" -ge "${THRESHOLD}" ] || continue

  lsize=${lsize%.*}
  isize=$(( INCREMENT * lsize / 100 ))
  echo "lvresize -L +${isize}M ${vg}/${lv}"
  /sbin/lvresize -L "+${isize}M" "${vg}/${lv}"
done

You can change the following settings:

  • THRESHOLD is the percent of usage which triggers resizing. When actual usage is greater than this value snapshots will grow.
  • INCREMENT is used to calculate how much size is added to the volume, it is a percent of the current volume size.

Of course the script could be improved (i.e. it could check whether there are space for growing in the volume group), but this naïve implementation is enough to make me happy and not to worry about checking status of my snapshots periodically :D

Systems — 1 Comment
3
Jul 08

Taking snapshots with LVM

Let us suppose we have a volume group named vg. Let us suppose we have a logical volume named base which holds our precious data in a 800 MB file system which has support for freezing (e.g. XFS). So we can take snapshots at any time with a single command:

# lvcreate --size 200M --snapshot --name snappy /dev/vg/base

Now we can mount it whenever we want to recover from the saved status:

# mount -o nouuid,ro /dev/vg/snappy /mnt

The nouuid option is needed for XFS filesystems, otherwise the driver will think it is mounting the same filesystem and will refuse to mount the snapshot. Take into account that:

  • Blocks in the source volume are stored on-demand in the snapshot volume when they are changed.
  • If mounting the snapshot in write mode, written blocks will be stored in the snapshot volume.
  • You can make the original filesystem grow, but it cannot be shrunk.
  • Snapshots can be grown and shrunk.
  • If a snapshot runs out of free blocks, it will render itself unusable: make sure you check its status periodically with lvdisplay and grow it as needed.

Currently I am using snapshots as a quick-come-on-let’s-go backup method for the root filesystem of a Debian installation I am using for a project. If somethings goes wrong one can mount the snapshot and restore files from the snapshot… or one can even boot up from a snapshot by setting root=/dev/mapper/vg-snappy in the kernel command line… :D