Lighthouse      Zap's Digital Lighthouse

Zap's Digital Lighthouse is
a Blosxom weblog for our digital outpost on the Internet

For info

Useful links:
The Reg
Wiki soekris

/FreeBSD (24)
/admin (1)
/blosxom (6)
/games (3)
/hardware (17)
/inet (4)
/misc (37)
/notwork (2)
/software (11)
/tech (1)

 2019 (1)   
 | July (1)
 2018 (6)   
 | December (1)
 | November (3)
 | January (2)
 2017 (4)   
 | December (2)
 | January (2)
 2016 (3)   
 | November (1)
 | October (1)
 | January (1)
 2015 (9)   
 | December (2)
 | November (1)
 | October (1)
 | June (1)
 | May (2)
 | February (1)
 | January (1)
 2014 (9)   
 | December (1)
 | October (1)
 | September (1)
 | August (3)
 | May (2)
 | April (1)
 2013 (20)   
 | October (3)
 | June (4)
 | May (2)
 | April (7)
 | March (1)
 | January (3)
 2012 (60)   
 | December (4)
 | October (1)
 | July (5)
 | June (7)
 | May (1)
 | April (6)
 | March (3)
 | February (14)
 | January (19)
 2011 (3)   
 | December (1)
 | November (2)
 2008 (1)   
 | October (1)



home :: FreeBSD

Mon, 31 Dec 2018

Moving a Running FreeBSD system from i386 to amd64

Recently, the little Soekris net6501 machine that I use as a home server for DNS and other similar functions decided to die after many years of useful service (it's a known problem of net6501 machines unfortunately; at some point they just self-brick and refuse to reboot :-(

So I took the Gigabyte Brix GB-BXi3-4010 that I had been using on my TV in the living room and simply removed the Windows 10 mSATA drive from it and inserted the mSATA drive from my net6501 into it and Poof! FreeBSD booted fine and It Just Worked.

Therefore in no time at all, poterne (my little home server) was back in action and no one at home even noticed, since we haven't been using the little Brix much since we bought a 4K TV that has Netflix and Youtube in firmware (maybe more on this some other day).

However, the Intel Core i3-4010U Processor at the heart of the little Brix computer is a 64-bit part, so I'd like to move FreeBSD on poterne from i386 to amd64 so that it can use all of its 8GB of RAM, since I've noticed that dmesg presently says:

real memory  = 8589934592 (8192 MB)
avail memory = 3574235136 (3408 MB)

So I searched the Internet for "freebsd upgrade i386 to amd64" and I found a few pages describing successful upgrades. Actually, most of them recommended not to do it and just reinstall, but I decided to go for it anyway.

The methods I found recommended upgrading by recompiling, so I fetched the FreeBSD 12.0-RELEASE installation media and installed the source code onto my little server. Then I started to recompile, but it took ages (maybe because I put the source and object files on the external USB disk?), so I decided to redeploy from RELEASE binaries:

cd /tmp
mdconfig -a -t vnode -f /tmp/FreeBSD-12.0-RELEASE-amd64-dvd1.iso
mount -t cd9660 -o ro /dev/md0 /mnt
cd /

The release tarballs are in /mnt/usr/freebsd-dist/ (such as src.txz, base.txz, etc.)

Of course, I then used rsync to make a backup of my server onto my local NAS machine, in case anything went wrong.

Then, I proceeded with the migration process based on recipes from various websites (notably

Turning the swap partition into a 'miniroot' seemed like a good idea, so I did that... My swap partition is is 6GB, so that worked well (6GB was enough for the 2GB of RAM I had on the Soekris net6501, but may could a bit short with the 8GB of RAM from the Brix... but then again, I probably won't need any swap space on the new machine with 8GB of RAM for a little headless server like poterne).

swapinfo -h
swapoff /dev/ada0p2
sed -i.bak -e '/swap/s;^;#;' /etc/fstab
newfs -U -n /dev/ada0p2
mount /dev/ada0p2 /media
... here, I extracted various files from the FreeBSD-12 release
... and copied various files from _/_ onto _/media_ (notably from /etc)
... sorry, I didn't write it all down :-)
file /mnt/bin/echo
cp /etc/fstab /media/etc/fstab

umount /mnt ; mdconfig -d -u0

After setting up the miniroot on /dev/ada0p2, I rebooted from there and mounted the real filesystems onto /mnt and copied over the files from the miniroot to /mnt.

I then rebooted with the new system and cleared the pkg cache and reinstalled all of my packages.

In the end, it worked fine and is now running in 64-bit mode.

/FreeBSD | Posted at 11:10 | permanent link

Mon, 12 Nov 2018

Setting up my new home NAS with FreeBSD 11.2

FreeBSD's 25th anniversary

So, I've got my new HPE Proliant Microserver Gen10 last summer, but I hadn't had time to set up it yet.

I am setting it up with FreeBSD. (It's a sobering thought that FreeBSD is now 25 years old: I've been running it since the very first version, coming from 386BSD and Microport System V/AT before that.)

I've put the 4 Seagate drives in the bays and inserted a USB key with FreeBSD 11.2 on it in the internal USB port. As reported in the previous blog entry, I had to add a "/boot/loader.conf" file with


in it.

Then, I needed to:

  • Install packages with pkg: perl5, zip/unzip, zile, mutt, rsync, sudo, samba, and smartmontools

  • Copy the user accounts from my old proliant server onto the new datasink server. I haven't decided yet if I will shut down the old server or if I will just let it run on my network.

  • Install the 4 Seagate drives under ZFS... more on this below.

  • Copy the files from my old proliant server to the new one.

  • Activate NFS on the new server. With ZFS, I don't need to start the usual NFS server processes, as ZFS supports NFS sharing directly (see below).

  • Move the large external USB drive that I use for weekly rsync backups onto the new machine.

  • Configure regular backups, monitoring, etc. on the new server.

  • Exchange the names proliant and datasink on the two machines. Verify that the Windows machines and the Macs on my home network can access the new server transparently.

  • And that should be it.

Setting up the 4 Seagate disks with ZFS

Here is how I set up the four 6TB disks on that new server:

  1. Decided to use the 4 drives in a RAIS-5 configuration, which allows me to recover from a single drive failure. If the Microserver had had 5 drives, I would have used a RAID-6 configuration, but with 4 drives, it seemed like an exageration.

  2. Decided to create a single large ZFS pool with all of the drives in it.

  3. Also decided to limit the number of ZFS filesystems I create in my ZFS pool. For the limited usage I make of my little home server, I haven't found much use over the years for splitting the disk space pool into multiple logical filesystems, apart from sharing via NFS, so this time, I'm going with just a few filesystems: a large one called /data, a smaller one for user files called '/homes' that will be exported via NFS, and the usual /var and /usr.

  4. However, I've been running a ZFS snapshot script on my machine for the past few years (as explained here), and I've found it quite useful. So I decided to continue with them. So I must account for enough freespace in the ZFS pool for all of the data from the snapshots.

  5. For other uses, I'll create symbolic links into the /data zpool for /backups and others.

  6. Note: I have used the /dev/diskid/* devices to set up my ZFS pool, rather than the /dev/ada* devices. If I get a drive failure or take a drive out, I find the renumbering of ada devices to be a scary proposition. Similarly, when I use external USB drives, I give them GPT names and use these rather than /dev/da* which can change when you unplug/replug the USB connectors.

So, here are the commands that I've run to initialize the disk space on my new server, conserved here for future reference:

  • Check out which disks we have available:

    ls /dev/diskid

  • Create the ZFS pool (the ashift option specifies the drives have 4KB blocks)

    zpool create -o ashift=12 -m /zfspool zfspool raidz diskid/DISK-ZAD4QZAT diskid/DISK-ZAD4RS61 diskid/DISK-ZAD4V4N3 diskid/DISK-ZAD4V4N9
    (question: why have a mountpoint for a ZFS pool?)

  • Create the filesystems in my ZFS pool:

    zfs create zfspool/data mountpoint=/data
    zfs create zfspool/var mountpoint=legacy
    zfs create zfspool/home mountpoint=legacy
    zfs create zfspool/usr mountpoint=legacy
    zfs create zfspool/tmp mountpoint=legacy

  • Copy the existing stuff onto the new partitions and set the real mountpoints:

    foreach i ( var home usr tmp )
    mount -t zfs zfspool/$i /mnt
    ( cd /$i ; sudo rsync -aHv . /mnt/ )
    umount /mnt
    zfs set mountpoint=/$i zfspool/$i

  • Share the /home dataset:

    zfs set sharenfs=on zfspool/home
    (check that this works... I am not sure)

/FreeBSD | Posted at 07:39 | permanent link

Sat, 15 Oct 2016

Installing FreeBSD 11

Very busy time lately, but I'm glad to see that FreeBSD 11.0-RELEASE is now out, so I started upgrading my systems to this latest release this morning. It is quite easy:

/usr/sbin/freebsd-update -r 11.0-RELEASE upgrade
/usr/sbin/freebsd-update install
then reboot
/usr/sbin/freebsd-update install
pkg upgrade

and that's it.

On one of my Soekris boxes (the net6501) that has an external USB disk for backups and general storage, the new kernel seems to try to check the filesystem faster than it used to... this means that it tries to fsck the filesystem before the USB disk has been mounted, and hence I get:

        ufs: /dev/da0s1a (/usbdisk)
Warning! Some of the devices might not be available; retrying
Waiting 30s for the root mount holders: usbus7 usbus3
uhub7: 3 ports with 3 removable, self powered

and then it retries and works fine. The filesystem is tagged as 'late' in fstab, but that doesn't appear to do the trick.

OK, back to PuTTY to complete the upgrade of my various systems.

/FreeBSD | Posted at 08:15 | permanent link

Sun, 08 Nov 2015

Adding a new USB disk to my FreeBSD 10.2 server

(I think that the stuff below is accurate, but do proceed with care if you're going to try it out... I could have made a mistake writing down the list of steps. Let me know if you spot any issue.)

I would like to add a new USB disk to my FreeBSD 10 machine. In the olden days, I used to use sysinstall, but that doesn't seem to be the modern way to do things these days.

Notably, I would like to "label" the drive, so that I can mount it with its logical name rather than its physical device name... this is particularly important for USB disks, as plugging and unplugging USB device ports may cause changes in device names, which could cause really annoying issues if fstab mounts the wrong device (as I've already written about on Feb 20th, 2012).

So, here is what one must do to set up a new disk for use on FreeBSD...

1) First of all, decide how you're going to name your device. Using volume names instead of just raw device names is highly desirable if you're in a dynamic environment (e.g. if you're going to be adding disks to the computer in the future -- including USB storage, devices may end up being renamed).

For USB disks, choosing good label names is important: you may want to replace a disk at some point if one crashes (so do not use the /dev/ufsid/... names) and/or you may have a CPU crash and you'd like to mount your disk onto some other FreeBSD system (so do not use the same names for your bootdisks on all computers, unless they are truly meant to be considered identical clones). This means that 'bootdisk' is probably not a great label, in case you ever need to mount it on another machine that also boots from a 'bootdisk'... 'myhostname-bootdisk' might be better, albeit a bit verbose.

I'm not sure if there are canonical policies for that sort of thing, but I'd be interested in hearing about recommendations in this area.

Now, getting to the actual set up of the drive.

2) connect the disk to the USB port, and check out the dmesg log to find out the correct device name, something like da0 or da1 (depending on whether it's your first USB or SCSI disk, or your second one, or ...)

# dmesg | tail
da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <WD My Book 1230 1050> Fixed Direct Access SCSI-6 device
da0: 400.000MB/s transfers
da0: 1907697MB (3906963456 512 byte sectors: 255H 63S/T 243197C)
da0: quirks=0x2<NO_6_BYTE>

In this case, it's da0 and it's connected to a USB3 port (so it can do 400 MB/s rather than 40 MB/s for USB2).

As always: make sure you select the right device name, because formatting the wrong drive is bad and will cause your data to go away. If you want to check out which disks are present, you can type:

# camcontrol devlist

You can also use the usbconfig command to see what USB hubs and devices are present on your system:

# usbconfig

You may also want to use the 'mount' command to check which disks are currently mounted, but with volume labels being used instead of phycical device names, you may not see the 'real' devices names in there (e.g. /dev/ufs/bootdisk instead of /dev/da1 or something similar).

3a) Once certain which disk to use, if a UFS filesystem is desired, proceed thusly (if a ZFS filesystem is desired, move on to step 3b):

(note: /dev/XXX represents the new disk, such as /dev/da0 or /dev/ada1)

# dd if=/dev/zero of=/dev/XXX bs=1k count=2
# gpart create -s gpt XXX
# gpart add -t freebsd-ufs XXX
# newfs -U -L myvolume /dev/XXXp1   ; create a UFS filesystem with label myvolume on partition p1
# mkdir -p /mnt/newmountpoint
# mount /dev/ufs/myvolume /mnt/newmountpoint

This creates a whole disk for FreeBSD (such as a data disk). If you want to set up a disk with multiple filesystems, you can do something like:

(This creates a 'USB boot disk')

# dd if=/dev/zero of=/dev/XXX bs=1k count=2
# gpart create -s gpt XXX
# gpart add -t freebsd-boot -s 512k -a4k -l usbboot XXX
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i1 XXX
# gpart add -t freebsd-ufs -l usbrootfs -b 1m -s 2g XXX
# gpart add -t freebsd-ufs -l usbvarfs -a 1m -s 2g XXX
# gpart add -t freebsd-ufs -l usbusrfs -a 1m XXX
# gpart show -l XXX         ; check out the partitions that we're created
# newfs -U -t /dev/gpt/usbrootfs
# newfs -U -t /dev/gpt/usbvarfs
# newfs -U -t /dev/gpt/usbusrfs

Et voilą, finished. You may want to add a line to /etc/fstab to have /dev/ufs/myvolume mounted automatically at boot time. Always use volume label names instead of physical disk drive names, especially for USB drives where the disk drive name may change across reboots depending on what USB devices are plugged into the computer's ports and USB hubs.

3b) If a ZFS filesystem is desired instead, skip step 3a and proceed thusly (with /dev/XXX representing the new disk, such as /dev/da0 or /dev/ada1 once again):

(Once again, this creates a 'USB boot disk')

# dd if=/dev/zero of=/dev/XXX bs=1k count=2
# gpart create -s gpt XXX
# gpart add -t freebsd-boot -s 512k -a4k -l usbboot XXX
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 XXX
# gpart add -t freebsd-zfs -l usb0 XXX can verify with 'gpart show XXX'
# zpool create -m /mnt zroot /dev/gpt/disk-main

Et voilą, finished.

4) At this point, your disk is up and running. Do not forget to use mount to make it available under FreeBSD, and to umount it before unplugging it.

Good luck!

Supplementary reading

You may want to check out for more info. Furthermore, may also be pertinent. And about ZFS, do check out: and of course this:

/FreeBSD | Posted at 01:02 | permanent link

Sun, 01 Feb 2015

Setting up an Unbound DNS server for a local network

Since FreeBSD 10, BIND has been replaced as the default DNS server by Unbound. This is nice for all kinds of security reasons (many of the recent FreeBSD security advisories have been because of BIND issues), but it does mean changing some 20+ years old habits.

On my local network, I like to have a small server that acts as the DNS server for all of my home machines (of which I have too many indeed), not only for speed of resolution and resilience, but also to have authoritative reverse DNS resolution for machines in the private address space (192.168.1.*)... so I do not only need a local caching server, but I need to configure it to serve my local network, and to be authoritative for a small set of addresses under This means doing a bit of configuration beyond the out of the box Unbound config in FreeBSD 10.1...

I have found a few helpful sites in setting up my Unbound DNS server:

but Google will help you find a lot more.

The FreeBSD Handbook says "By default, Unbound will provide DNS resolution to the local machine only. While the base system package can be configured to provide resolution services beyond the local machine, it is recommended that such requirements be addressed by installing Unbound from the FreeBSD Ports Collection.", however I found that for my very simple case it was easy to configure the "local_unbound" that is part of FreeBSD 10.1 do serve as my local network's DNS server...

Went to /var/unbound/, gave a cursory look to the "forward.conf", "lan-zones.conf", and "unbound.conf" that are present to see if they made sense (esp. "forward.conf" which is built from the "/etc/resolv.conf" file at installation... if your resolv.conf was wrong, you probably need to adjust it).

I then simply added a file in /var/unbound/conf.d/ to make outbound do what I need, which I called "local-setup.conf". This file simply contains:

## Unbound config for our local network
##                              Denis, 2015-01-31
        interface: ::0
        access-control: allow
        access-control: ::1 allow
        access-control: allow
        verbosity: 1
        port: 53
        do-ip4: yes
        do-udp: yes
        do-tcp: yes
        num-threads: 4

## not sure if I need this?    root-hints: "/var/unbound/etc/root.hints"

        # locally served zones can be configured for the machines on the LAN.

        local-zone: "" static

        ## don't need those, as my DNS is at
        ##      local-data: "firewall.home.lan.  IN A"
        ##      local-data: "laptop.home.lan.    IN A"
        ##      local-data: "xboxone.home.lan.   IN A"
        ##      local-data: "ps4.home.lan.       IN A"

        ## reverse DNS
        local-data-ptr: "    firewall.home.lan"
        local-data-ptr: "    denis.home.lan"

It seems to work at this point. Later on, I can explore the statistics produced by Unbound, but this is sufficient to get our little home server up and running.

/FreeBSD | Posted at 06:27 | permanent link

Mon, 29 Apr 2013

Upgrading FreeBSD

An interesting script to upgrade FreeBSD (and installed ports) from .

echo "Starting updates: `date`" | tee -a ${LOG_FILE}
echo "***"
echo "*** Checking for FreeBSD patches..."
echo "***"
/usr/sbin/freebsd-update fetch | tee -a ${LOG_FILE}
/usr/sbin/freebsd-update install | tee -a ${LOG_FILE}
echo "***"
echo "*** Updating ports tree..."
echo "***"
/usr/sbin/portsnap fetch update | tee -a ${LOG_FILE}
echo "***"
echo "*** Looking for ports to update..."
echo "***"
/usr/local/sbin/portmaster -a --no-confirm | tee -a ${LOG_FILE}
echo "***"
echo "*** Checking installed ports for known security problems..."
echo "***"
/usr/local/sbin/portaudit -Fva | tee -a ${LOG_FILE}
echo "Finished updates: `date`" | tee -a ${LOG_FILE}

This uses "freebsd-update" and "portmaster" which seem to be the "modern" way for people to manage FreeBSD installations.

P.S. Oh well, I still like to recompile world+kernel by hand...

/FreeBSD | Posted at 19:53 | permanent link

Wed, 03 Apr 2013

FreeBSD 9.1 panics resolved

A couple of weeks ago, I upgraded this small machine to FreeBSD 9.1. This is a small and very reliable Soekris net4810-50 box with 128 MB of RAM that has been running FreeBSD for years.

Since upgrading, it has been crashing daily around 03:00 with a panic message that I've finally captured today:

panic: kmem_malloc(4096): kmem_map too small: 38060032 total allocated

Some googling around let me to this thread from December 2012 on freebsd-stable.

which explains a similar situation. The recommendation in the thread is to add:

to /boot/loader.conf to avoid loading the new CAM CTL device, which seems to require lots of RAM.

So, is back on its feet, hopefully with its usual stability. I'll write again if I encounter any further issues.

I have to say though that I am somewhat surprised that the GENERIC build of FreeBSD suddenly has significantly higher RAM requirements as of FreeBSD 9.1, though of course 128MB of RAM is small in today's world.

/FreeBSD | Posted at 02:10 | permanent link

I *really* hate gettext

OK, so I know that some of this is certainly self-inflicted, but I seem to regularly run into trouble where I try to upgrade a bunch of ports on a FreeBSD installation somewhere, and bam suddenly I get build errors around locales or gettext shared libraries.


Now, I do like my applications to be able to speak French just as much as the next guy, but somehow the amount of grief that this has been causing me over the years is getting to be significant.

Latest issue is trying to rebuild ports on 3 different machines following an upgrade to FreeBSD 9.1. All went smoothly with the O/S upgrade using freebsd-update, but trying to rebuild the ports after fetching the latest versions using portsnap is proving harder than expected (or warranted).

I have downloaded the latest ports, and am trying to do the equivalent of "portmaster -a", and then I get:

./localename.c: In function '_nl_locale_name_thread_unsafe':
./localename.c:2607: error: 'locale_t' undeclared (first use in this function)
./localename.c:2607: error: (Each undeclared identifier is reported only once
./localename.c:2607: error: for each function it appears in.)
./localename.c:2607: error: expected ';' before 'thread_locale'
./localename.c:2608: error: 'thread_locale' undeclared (first use in this function)
./localename.c:2608: error: 'LC_GLOBAL_LOCALE' undeclared (first use in this function)
*** Error code 1

when compiling /usr/ports/devel/gettext/work/gettext-, which seems to indicate that "locale_t" is undefined and therefore that configure is probably not picking the requirements for locale on my system correctly, or that I have some old dependencies lying around.

So, hunting around in "/usr/ports/UPDATING" seems to indicate that I should rebuild converters/libiconv first, and then rebuild devel/gettext. So, I did that. But still not success. So it seems I would need to rebuild devel/libtool first, so I did that too. But still gettext won't compile.


This whole i18n thing seems to violate the principle of least astonishment.

Anyway, just a rant because I am frustrated. I'll go back to it and attempt to do things right this time... but I really do hate gettext.

P.S. Sigh... even sudo fails with "sudo: unable to dlopen /usr/local/libexec/ Shared object "" not found, required by """... will rebuild it without the NLS option!

P.P.S. Finally solved it by recompiling the world and kernel from source... and then "gettext" compiled correctly. Not sure what changed... this is unpleasant.

/FreeBSD | Posted at 02:08 | permanent link

Mon, 01 Apr 2013

Accessing ZFS snapshots

I make hourly snapshots of the ZFS filesystems on my home server. This is quite easy using a small script called "" (which I found on the web) and the following lines in root's crontab file:

# perform zfs snapshots
3  *  *  *  *   /root/ pool0 hourly 25
13 0  *  *  *   /root/ pool0 daily  32
23 0  *  *  0   /root/ pool0 weekly 60

7  *  *  *  *   /root/ datapool hourly 25
17 0  *  *  *   /root/ datapool daily  32
27 0  *  *  0   /root/ datapool weekly 60

This keeps a day's worth of hourly snapshots, a month's worth a daily backups, and a year's worth of weekly backups (I should perhaps also keep a decade's worth of annual backups?!).

This is useful and can be really easily accessed... For example, I restored "/usr/local/lib/" from a recent snapshot by doing:

cp /.zfs/snapshot/daily.5/usr/local/lib/libintl.* /usr/local/lib

Accessing snapshots through .../.zfs/snapshot is quite handy.

PS. For the record, the script is here.

/FreeBSD | Posted at 15:42 | permanent link

Tue, 01 Jan 2013

FreeBSD 9.1 is out

Oh, and the good news of the end of 2012 was that FreeBSD 9.1 has been released (see here for the announcement).

I'll start installing it over the weekend on some of my machines. Before then, we need to fly back home and get back to work. Happy January 1st everyone!

/FreeBSD | Posted at 03:26 | permanent link

Sun, 15 Apr 2012

ZFS and USB disks

Having played with ZFS and USB disks over the past few weeks on FreeBSD 9.0, I have come to the conclusion that multiple USB disks + ZFS raid-z is not an excellent combination... USB disks can appear or disappear dynamically, and ZFS does not deal with this beautifully.

ZFS does deal with devices appearing and disappearing, but with an understanding that devices are mostly stable. So, I will use ZFS on internal disks, and for USB disks, I will limit myself to single-disk volumes (which seem to work fine).

I would be interested in a filesystem that would deal with disks being plugged in (or plugged out) dynamically (USB disks or otherwise), but I am not sure that this exists. I will hunt around for something like this.

/FreeBSD | Posted at 11:13 | permanent link

Sat, 17 Mar 2012

Content Addressable Storage

Because I am backing up a whole bunch of machines that may have identical files on them (many copies of Windows, old backups, etc), I am interested in finding ways of having n copies of a file not occupy n times the amount of storage.

I see three ways of doing this:

  1. Using ZFS file depuplication on my backup server. This sounds good, but all of the ZFS configuration guides I've seen recommend having lots of RAM when you use dedup, and my little backup servers have only 2 to 4 GB of RAM, so this seems like a non-starter.
  2. Using a manual dedup script that uses hardlinks to free up diskspace on a Unix volume... Something like dedup from Roderick Schertler (documented here), trimstrees at CPAN, harlink from Julian Andres Klode, opendedup, or fileuniq on sourceforge (there is a list here) actually. That sounds attractive, but I am not sure how long it would take on 1 TB of data, and I am not sure hardlinks will work well on files owned by multiple users.
  3. Then, there is the possibility of Content Addressable Storage. EMC does this on its Centera product line, and it is quite useful in the Corporate space... in the Open Source world, I have been interested by Poul-Henning Kamp's stow package for a while (see here for the old version or here for the new version that phk is working on this year).

Stow looks interesting... I'll try to find some time to dig further into it.

/FreeBSD | Posted at 11:48 | permanent link

Sun, 19 Feb 2012

Going for simple FreeBSD instead of FreeNAS

So, I have been playing with my small soekris net6501-70-based NAS device for home.

Lots of things have happened since I last blogged about this:

  1. I decided not to use FreeNAS 8 on my net6501, but rather to just use FreeBSD 9 with ZFS, NFS, Samba, and rsync.
  2. I have copied my olde NSLU2 shared space onto my net6501 under ZFS... it took over 24 hours to copy over the network, the NSLU2 being fairly slow.
  3. I have set up a script to take snapshots of my ZFS storage on a regular basis using a script I found on the internet: this is almost as nice as have the Mac OS X time machine on my little NAS server (ok, not as nice, but still pretty good)
  4. I have had some difficulty with an old Maxstor 1TB USB drive, which seems to appear/disappear... it has caused corruption in my main ZFS pool, which I needed to fix manually, and then scrub the ZFS pool (which took almost 7 hours).
  5. Given all of this, I have ordered a small HP microserver. External USB drives are handled strangely by server O/Ses as they appear/disappear, change device names, etc. Let see how well the microserver with internal drives will do.
  6. By the way, when dealing with External USB disks, never use device names (such as /dev/da0 or /dev/da1s1a...); these names can change dynamically as drives are plugged in, or powered off, etc. Always label your filesystems (e.g. for UFS: tunefs -L disk1 /dev/da0s1a, which then appears as /dev/ufs/disk1; or for GPT disks: /dev/gptid/760ae9e2-5988-12f1-a136-000054cf2648) and use these names in ZFS or fstab... this will avoid a LOT of grief when reboting, or disconnecting or reconnecting a device, or indeed when a device fails.

So, lots is happening with my small NAS server.

More on this next weekend.

/FreeBSD | Posted at 19:01 | permanent link

Upgrading FreeBSD from 8.2 to 9.0

I have just finished upgraing one of my FreeBSD machines from 8.2 to 9.0, rebuilding from source as explained in the FreeBSD handbook here.

One of the painful steps in upgrading a FreeBSD machine, is the mergemaster step, as it forces you to go through a whole bunch of files, just to accept CVS tag changes... or so I thought !!! Turns out that there are optional flags to mergemaster, and running "mergemaster -FiU" does pretty much what I've been hoping mergemaster to do. Yay!

/FreeBSD | Posted at 16:12 | permanent link

Sun, 12 Feb 2012

FreeNAS, ZFS, and external USB disks

I have set up my Soekris net6501 with the latest beta of FreeNAS 8, FreeNAS-8.2.1-ALPHA-r10110M-x64. I installed 4 external 1 TB external USB disks on the machine, and set them up as a RAID-Z ZFS volume in FreeNAS and set about exporting CIFS and NFS partitions to the rest of my household. FreeNAS 8 is going to be an excellent product when it releases v.8.2.1... I look forward to seeing it in use in the coming months.

However, I have received the following FreeNAS alert twice now: The ZFS volume status is UNKNOWN: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'.

Interestingly enough, both times the error was corrected without any problem and the ZFS pool was back in HEALTHY state when I looked into it, but still that message sounds fairly omnious. Could it simply come from the fact that the drives go to sleep and are sometimes not ready when ZFS tries to access them (hence they somehow timeout)??? Or is it something more serious and one of my (new) drives is about to go south?

I wonder whether there are any recommendations against setting up a RAID-Z zpool across external USB drives that can go to sleep (and hence take 30-60 seconds to wake up). Hmmmm, I should investigate ZFS best practices and see if USB drives are usable this way. Else, I'll just stick with UFS with rsync for replication/backup.

More next weekend :-)

/FreeBSD | Posted at 17:34 | permanent link

Sun, 05 Feb 2012

Identity services

A quick follow-up to my recent blog entry about identity servers on small home networks:

  • I have been playing with FreeRadius (available on FreeBSD as part of the ported packages -- /usr/ports/net/freeradius2, I believe) to see if I could use that as an identity server for my home network... to be investigated further in the coming weeks.
  • On a loosely related topic, I find the myOpenID services interesting, especially the "myOpenID for domains" stuff. It seems like I could use this to enable OpenIDs like to work? It looks like it might be free if I have 10 IDs or less on my domain. Interesting. However, I need to see what level of independence I would need to lose on my domain (including the ability to change providers in the future) if I did that.
  • More interesting stuff to investigate on future weekends :-)

/FreeBSD | Posted at 18:13 | permanent link

Moving forward with FreeNAS

FreeNAS 8 is a nanobsd configuration, based on FreeBSD 8.2. In order to boot FreeNAS 8 on my Soekris net6501 under the amd64 image of FreeNAS, one needs to rebuild the FreeBSD kernel that is at the heart of FreeNAS with the line "device mptable" added to the kernel configuration file, as described on the unofficial Soekris wiki here.

Therefore, I used the instructions from the FreeNAS Forums here to set up a VMware virtual machine running FreeBSD to setup the FreeNAS sources, adjusted the kernel configuration file in "/usr/local/freenas/trunk/nanobsd/FREENAS.amd64" to add the device mptable line as described above, and then built a FreeNAS-8.2.1-ALPHA-r10006M-x64 image that I put onto a USB key and booted my Soekris net6501 with it.

FreeNAS 8 seems to look good on the net6501 and I am able to set it up with 2 external 2 TB USB disks partitioned with ZFS. For some strange reason, the FreeNAS 8 website seems to call for at least 6 GB of RAM to run it with ZFS, but my 2 GB net6501 seems to do just fine so far.

So, I just need to complete the set up of my FreeNAS box, export one of the disks through CIFS, NFS, and rsync, and set up a regular backup from disk1 to disk2. Once that runs fine, I can probably connect a 3rd 2 TB drive to that net6501, and set up some form of RAID-5 between the 3 drives (with snapshots, perhaps I don't need the weekly backup from disk1 to disk2, as I normally do on my NSLU2).

FreeNAS seems like a winner so far. More info later.

Miscellaneous comments & ramblings:

  • I still need Soekris to fix the USB boot code on the net6501 before I consider this completely operational: the net6501 will freeze on boot when my two 2 TB external USB drives are connected at boot time. Works fine when I connect them after the machine has booted... if at least Soekris could add an option in the BIOS to not scan certain USB ports for bootable devices, I could move forward.
  • FreeNAS 8 is based on FreeBSD 8.2, which is unfortunate as it only has ZFS v15, as opposed to FreeBSD 9.0 which comes with ZFS v28 (and perhaps the upcoming FreeBSD 8.3 also?)
  • There appears to be a certain amount of tension between the "new" FreeNAS 8 team, and the "legacy" FreeNAS 7 team. The latter has based FreeNAS 7.5 on FreeBSD 9.0 to get the newest FreeBSD stuff (including ZFS v28 as stated above) and is starting to talk about calling its branch FreeNAS 9, which is not going over too well with the FreeNAS 8 people (who actually own the FreeNAS trademark). Seems like FreeNAS 7 will either die or else it will need to fork off with another name (see here).
  • I don't have a strong opinion on this at this point, except for the fact that I am disappointed that FreeNAS 8 specifies minimum RAM requirements which seem to me to be unreasonably high (4 GB minimum, and 6 GB minimum if one wants ZFS) for some of the small machines that one could want to run FreeNAS upon. I like the FreeNAS 7 community effort, but I fear that it will end up as a dead-end effort once FreeNAS 8 is solid & feature-complete.
  • Anyway, I decided to set up my NAS server as a FreeNAS 8 server, to see if it ran OK in 2 GB of RAM (which, frankly, I don't see why it couldn't).
  • Seems like the net6501 will work as a home NAS device anyway, so finally I don't need the HP Proliant Microserver I was referring to the other day.

Anyway, enough for tonight... good night to all.

/FreeBSD | Posted at 17:39 | permanent link

Thu, 26 Jan 2012

What are people using for small identity servers at home?

So, on my home network I have:

  • a DHCP server
  • a web server
  • a CIF file server
  • an NFS file server
  • an rsync backup server
  • a network printer
  • a mail server

and various other odds and ends.

However, it strikes me that I do not have a small and simple identity server, where I could define user accounts for all of my various devices and OSes (Windows, Mac OS X, FreeBSD, and probably some Linux also).

What do people use? NIS yellow pages? that seems too unixoid and will probably not help with the Windows machines. Some sort of Microsoft Active Directory? Through Samba? that feels too microsofty perhaps. Is there something that bridges the gap? Going beyond my little home network, is there something that would let me be accepted as an OpenId authenticator? Also, something I could use to allow logins into my machines hosted outside of the home network.

Hmmm. Over the years, we have gotten pretty good at doing IP networks, file sharing, printer sharing, web serving, and even web services and the like... but we are still struggling with having some form of simple authentication mechanism standardized. Probably because it is hard to have trust in authentication mechanisms, and especially the quality of the data they contain(*).

Anyway, that's another interesting weekend project.

(*) Perhaps we are setting the bar too high? I do not need strong authentication with non-repudiation and all that jazz... I would like a simple userid/password mechanism that would return something like the old unix uid/gid, but in 64-bit range, and I would like it to be easily integrated with Windows, Unix, and Mac OS X. Hmmm... fuzzy requirements... it's always more complicated than it looks like. What are people using out there? Any solid and usable public domain reference implementations I could look at and implement? Would Radius fit the bill?

/FreeBSD | Posted at 20:36 | permanent link

Setting up a new NAS device at home

For the last few years, I have been using a couple of Linksys NSLU2 NAS devices coupled with external USB disks for shared storage across my home network. I find that this works quite well to store files from my Mac, Windows, and even FreeBSD machines. One of the NSLU2 is unslung and runs an rsync daemon that I use to save files from my various FreeBSD machines.

The NSLU2 is a nice device, but it had become fairly old (no gigabit ethernet, limited expandability of the software, etc). So I have been meaning to set up a replacement. I have decided to use a Soekris net6501 to build my own little NAS box, using a couple of external 2 TB external drives.

Now, the question is: what software will I run on the net6501? I have been quite tempted by FreeNAS, but FreeNAS 8 has a stated minimum RAM size requirement of 4 GB (with 6 GB listed as the minimum for ZFS usage), and the net6501 is limited to 2 GB RAM (actually, I think that's even a limit of the Intel Atom CPU that it is built upon). FreeNAS 0.7 has a significantly lower RAM requirement, but I am afraid that it will be orphaned, as the main development path for FreeNAS is now version 8.

Of course, I could just set up FreeBSD on the net6501 and be done with it.

Anyway, I still have a few weeks to decide, as I am waiting for a new BIOS release for the net6501, as I am having some issues with the new USB boot code of the Soekris device: if I plug in 2 large (2 TB) external USB disks, in addition to the small internal 8 GB USB key that holds the FreeBSD image, then the device hangs on boot while probing the USB devices. If I just connect the external USB drives after the machine is booted, it works perfectly. I am hoping that the USB probe code will be fixed in the next BIOS release.

Fun for the weekend :-)

/FreeBSD | Posted at 20:06 | permanent link

Sat, 21 Jan 2012

Setting up FreeBSD 9.0 with a ZFS root filesystem: Success!

So, I investigated last week's problem with ZFS and FreeBSD 9.0 and I discovered that the problem was very simple to fix. I needed to add:

    # echo 'zfs_enable="YES"'>>  /mnt/etc/rc.conf

to the procedure described last week (on January 15th) so that the zfs mountpoint options would be executed. So now, things are working fine... I am going to play with this configuration a bit to see if the fact that I haven't configured a swap space causes problem with a 2GB memory configuration when I try to compile world.

More later.

Late addition to this post: turns out that the author of the initial page at had also found the error and fixed it on the wiki, but I had not gone back to check. In the end, it's OK, this has allowed me to explore ZFS enough so that I now understand it better.

/FreeBSD | Posted at 13:49 | permanent link

Sun, 15 Jan 2012

Setting up FreeBSD 9.0 with a ZFS root filesystem

With the new version of FreeBSD being just out, I decided to try out the ZFS filesystem. Notably, I would like to find out whether it works correctly with 2GB of memory, or whether more is really needed (I read that it doesn't really work with less than 1GB of RAM, but then again, there are conflicting reports that it needs lots more, though sometimes that's qualified with 'if you want prefetch' or 'if you are trying to do dedup'). Actually, I am building a small NAS server I am building around a Soekris net6501-70 which comes with 2GB RAM, and I would like to find out if I can use ZFS with it.

So, I looked up and tried to set up a vmware virtual machine with ZFS only.

I did not bother setting up swap space, nor a mirrored disk configuration, so the commands I used were:

  • boot a vmware virtual machine from the amd64 FreeBSD9.0-RELEASE DVD
  • choose the Shell option at the partitioning dialog in bsdinstall
  • run the following commands:

    # gpart create -s gpt da0
    # gpart add -s 64K -t freebsd-boot -l boot0 da0
    # gpart add -t freebsd-zfs -l disk0 da0
    # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
    # zpool create -o altroot=/mnt zroot /dev/gpt/disk0
    # zfs create zroot/tmp
    # chmod 1777 /mnt/tmp
    # zfs create zroot/usr
    # zfs create zroot/var
    # zfs create zroot/home
    # exit
  • And then, after installation:
    (update: one must add # echo 'zfs_enable="YES"'>> /mnt/etc/rc.conf here as per the post on 21 January)

    # echo 'zfs_load="YES"'>>  /mnt/boot/loader.conf
    # echo 'vfs.root.mountfrom="zfs:zroot"'>>  /mnt/boot/loader.conf
    # zfs unmount -a
    # zpool export zroot
    # zpool import -o cachefile=/tmp/zpool.cache -o altroot=/mnt zroot
    # zfs set mountpoint=/ zroot
    # cp /tmp/zpool.cache /mnt/boot/zfs/
    # zfs unmount -a
    # zpool set bootfs=zroot zroot
    # zpool set cachefile='' zroot
    # zfs set mountpoint=legacy zroot
    # zfs set mountpoint=/tmp zroot/tmp
    # zfs set mountpoint=/usr zroot/usr
    # zfs set mountpoint=/var zroot/var
    # zfs set mountpoint=/home zroot/home

The last 4 commands (i.e. after 'zfs set mountpoint=legacy zroot') give error messages, but I think that's OK. However, once I rebooted the virtual machine, the filesystems (except for /) were not mounted and therefore FreeBSD did not come up properly.

So, this was a first try... More on this next weekend :-)

/FreeBSD | Posted at 16:57 | permanent link

Fri, 13 Jan 2012

FreeBSD 9.0 is out


Yay! FreeBSD 9.0 is out!

The announcement says:

The FreeBSD Release Engineering Team is pleased to announce the availability
of FreeBSD 9.0-RELEASE.  Some of the highlights:
    - ZFS updated to version 28
    - NFS subsystem updated, new implementation supports NFSv4 in addition to NFSv3 and NFSv2

Please see for more info.

Note also that FreeBSD 9.0-RELEASE is dedicated to the memory of Dennis M. Ritchie, one of the fathers of UNIX.

/FreeBSD | Posted at 15:19 | permanent link

Wed, 11 Jan 2012

FreeBSD 9.0-RELEASE ISO are out

Found the ISO images for FreeBSD 9.0-RELEASE on and

I've already got the amd64 version installed on a vmware player virtual image successfully from the ISO -- which works beautifully with FreeBSD under Windows 7; check it out!

I'm off to Zurich for a couple of days... more FreeBSD over the weekend.

/FreeBSD | Posted at 21:05 | permanent link

Mon, 09 Jan 2012

Waiting for the next FreeBSD release

I have been setting up a small computer -- a Soekris net6501-70 -- to use as a NAS (Network Attached Storage) device on my home network. The idea is to replace the small, reliable, Linksys NSLU2 (slugs) that I have been using for shared network usage in my home, with a new device with more disk space and Gigabit Ethernet ports since my home network (at least the wired part) is Gigabit Ethernet.

Ideally, I would have liked to run FreeNAS on the device and to run ZFS as a filesystem, but it didn't turn out that way, as the net6501 has 2 GB RAM, and FreeNAS requires a lot more than that.

Therefore I have installed FreeBSD 9.0-RC3 on my net6501-70 and I even had the pleasure to discover that I could run FreeBSD in amd64 mode (i.e. 64-bits mode).

Now, I cannot wait for FreeBSD 9.0 to be finally and officially released, and for the next iteration of the ne6501 combios to be effectively release.

Can't wait!

/FreeBSD | Posted at 18:29 | permanent link