|Zap's Digital Lighthouse|
home :: FreeBSDMon, 31 Dec 2018
Recently, the little Soekris net6501 machine that I use as a home server for DNS and other similar functions decided to die after many years of useful service (it's a known problem of net6501 machines unfortunately; at some point they just self-brick and refuse to reboot :-(
So I took the Gigabyte Brix GB-BXi3-4010 that I had been using on my TV in the living room and simply removed the Windows 10 mSATA drive from it and inserted the mSATA drive from my net6501 into it and Poof! FreeBSD booted fine and It Just Worked.
Therefore in no time at all, poterne (my little home server) was back in action and no one at home even noticed, since we haven't been using the little Brix much since we bought a 4K TV that has Netflix and Youtube in firmware (maybe more on this some other day).
However, the Intel Core i3-4010U Processor at the heart of the little Brix computer is a 64-bit part, so I'd like to move FreeBSD on poterne from i386 to amd64 so that it can use all of its 8GB of RAM, since I've noticed that dmesg presently says:
So I searched the Internet for "freebsd upgrade i386 to amd64" and I found a few pages describing successful upgrades. Actually, most of them recommended not to do it and just reinstall, but I decided to go for it anyway.
The methods I found recommended upgrading by recompiling, so I fetched the FreeBSD 12.0-RELEASE installation media and installed the source code onto my little server. Then I started to recompile, but it took ages (maybe because I put the source and object files on the external USB disk?), so I decided to redeploy from RELEASE binaries:
The release tarballs are in /mnt/usr/freebsd-dist/ (such as src.txz, base.txz, etc.)
Of course, I then used rsync to make a backup of my server onto my local NAS machine, in case anything went wrong.
Then, I proceeded with the migration process based on recipes from various websites (notably https://wiki.freebsd.org/amd64/i386Migration).
Turning the swap partition into a 'miniroot' seemed like a good idea, so I did that... My swap partition is is 6GB, so that worked well (6GB was enough for the 2GB of RAM I had on the Soekris net6501, but may could a bit short with the 8GB of RAM from the Brix... but then again, I probably won't need any swap space on the new machine with 8GB of RAM for a little headless server like poterne).
After setting up the miniroot on /dev/ada0p2, I rebooted from there and mounted the real filesystems onto /mnt and copied over the files from the miniroot to /mnt.
I then rebooted with the new system and cleared the pkg cache and reinstalled all of my packages.
In the end, it worked fine and poterne.rax.org is now running in 64-bit mode.Mon, 12 Nov 2018
So, I've got my new HPE Proliant Microserver Gen10 last summer, but I hadn't had time to set up it yet.
I am setting it up with FreeBSD. (It's a sobering thought that FreeBSD is now 25 years old: I've been running it since the very first version, coming from 386BSD and Microport System V/AT before that.)
I've put the 4 Seagate drives in the bays and inserted a USB key with
FreeBSD 11.2 on it in the internal USB port. As reported in the previous
blog entry, I had to add a "
Then, I needed to:
Setting up the 4 Seagate disks with ZFS
Here is how I set up the four 6TB disks on that new server:
So, here are the commands that I've run to initialize the disk space on my new server, conserved here for future reference:
Very busy time lately, but I'm glad to see that FreeBSD 11.0-RELEASE is now out, so I started upgrading my systems to this latest release this morning. It is quite easy:
and that's it.
On one of my Soekris boxes (the net6501) that has an external USB disk for backups and general storage, the new kernel seems to try to check the filesystem faster than it used to... this means that it tries to fsck the filesystem before the USB disk has been mounted, and hence I get:
and then it retries and works fine. The filesystem is tagged as 'late' in fstab, but that doesn't appear to do the trick.
OK, back to PuTTY to complete the upgrade of my various systems.Sun, 08 Nov 2015
(I think that the stuff below is accurate, but do proceed with care if you're going to try it out... I could have made a mistake writing down the list of steps. Let me know if you spot any issue.)
I would like to add a new USB disk to my FreeBSD 10 machine. In the olden days, I used to use sysinstall, but that doesn't seem to be the modern way to do things these days.
Notably, I would like to "label" the drive, so that I can mount it with its logical name rather than its physical device name... this is particularly important for USB disks, as plugging and unplugging USB device ports may cause changes in device names, which could cause really annoying issues if fstab mounts the wrong device (as I've already written about on Feb 20th, 2012).
So, here is what one must do to set up a new disk for use on FreeBSD...
1) First of all, decide how you're going to name your device. Using volume names instead of just raw device names is highly desirable if you're in a dynamic environment (e.g. if you're going to be adding disks to the computer in the future -- including USB storage, devices may end up being renamed).
For USB disks, choosing good label names is important: you may want to replace a disk at some point if one crashes (so do not use the /dev/ufsid/... names) and/or you may have a CPU crash and you'd like to mount your disk onto some other FreeBSD system (so do not use the same names for your bootdisks on all computers, unless they are truly meant to be considered identical clones). This means that 'bootdisk' is probably not a great label, in case you ever need to mount it on another machine that also boots from a 'bootdisk'... 'myhostname-bootdisk' might be better, albeit a bit verbose.
I'm not sure if there are canonical policies for that sort of thing, but I'd be interested in hearing about recommendations in this area.
Now, getting to the actual set up of the drive.
2) connect the disk to the USB port, and check out the dmesg log to find out the correct device name, something like da0 or da1 (depending on whether it's your first USB or SCSI disk, or your second one, or ...)
In this case, it's da0 and it's connected to a USB3 port (so it can do 400 MB/s rather than 40 MB/s for USB2).
As always: make sure you select the right device name, because formatting the wrong drive is bad and will cause your data to go away. If you want to check out which disks are present, you can type:
You can also use the usbconfig command to see what USB hubs and devices are present on your system:
You may also want to use the 'mount' command to check which disks are currently mounted, but with volume labels being used instead of phycical device names, you may not see the 'real' devices names in there (e.g. /dev/ufs/bootdisk instead of /dev/da1 or something similar).
3a) Once certain which disk to use, if a UFS filesystem is desired, proceed thusly (if a ZFS filesystem is desired, move on to step 3b):
(note: /dev/XXX represents the new disk, such as /dev/da0 or /dev/ada1)
This creates a whole disk for FreeBSD (such as a data disk). If you want to set up a disk with multiple filesystems, you can do something like:
(This creates a 'USB boot disk')
Et voilą, finished. You may want to add a line to /etc/fstab to have /dev/ufs/myvolume mounted automatically at boot time. Always use volume label names instead of physical disk drive names, especially for USB drives where the disk drive name may change across reboots depending on what USB devices are plugged into the computer's ports and USB hubs.
3b) If a ZFS filesystem is desired instead, skip step 3a and proceed thusly (with /dev/XXX representing the new disk, such as /dev/da0 or /dev/ada1 once again):
(Once again, this creates a 'USB boot disk')
Et voilą, finished.
4) At this point, your disk is up and running. Do not forget to use mount to make it available under FreeBSD, and to umount it before unplugging it.
You may want to check out http://www.macfreek.nl/memory/FreeBSD9withZFSbootdisk for more info. Furthermore, http://www.macfreek.nl/memory/FreeBSD9onSoekrisnet6501 may also be pertinent. And about ZFS, do check out: http://nex7.blogspot.ca/2013/03/readme1st.html and of course this: http://www.solarisinternals.com/wiki/index.php/ZFSEvilTuning_GuideSun, 01 Feb 2015
Since FreeBSD 10, BIND has been replaced as the default DNS server by Unbound. This is nice for all kinds of security reasons (many of the recent FreeBSD security advisories have been because of BIND issues), but it does mean changing some 20+ years old habits.
On my local network, I like to have a small server that acts as the DNS server for all of my home machines (of which I have too many indeed), not only for speed of resolution and resilience, but also to have authoritative reverse DNS resolution for machines in the private address space (192.168.1.*)... so I do not only need a local caching server, but I need to configure it to serve my local network, and to be authoritative for a small set of addresses under 1.168.192.in-addr.arpa. This means doing a bit of configuration beyond the out of the box Unbound config in FreeBSD 10.1...
I have found a few helpful sites in setting up my Unbound DNS server:
but Google will help you find a lot more.
The FreeBSD Handbook says "By default, Unbound will provide DNS resolution to the local machine only. While the base system package can be configured to provide resolution services beyond the local machine, it is recommended that such requirements be addressed by installing Unbound from the FreeBSD Ports Collection.", however I found that for my very simple case it was easy to configure the "local_unbound" that is part of FreeBSD 10.1 do serve as my local network's DNS server...
I then simply added a file in
It seems to work at this point. Later on, I can explore the statistics produced by Unbound, but this is sufficient to get our little home server up and running.Mon, 29 Apr 2013
An interesting script to upgrade FreeBSD (and installed ports) from mebsd.com: .
This uses "
P.S. Oh well, I still like to recompile world+kernel by hand...Wed, 03 Apr 2013
A couple of weeks ago, I upgraded this small machine to FreeBSD 9.1. This is a small and very reliable Soekris net4810-50 box with 128 MB of RAM that has been running FreeBSD for years.
Since upgrading, it has been crashing daily around 03:00 with a panic message that I've finally captured today:
Some googling around let me to this thread from December 2012 on
which explains a similar situation. The recommendation in the thread is to add:
to /boot/loader.conf to avoid loading the new CAM CTL device, which seems to require lots of RAM.
So, rax.org is back on its feet, hopefully with its usual stability. I'll write again if I encounter any further issues.
I have to say though that I am somewhat surprised that the GENERIC build of FreeBSD suddenly has significantly higher RAM requirements as of FreeBSD 9.1, though of course 128MB of RAM is small in today's world.
OK, so I know that some of this is certainly self-inflicted, but I seem to regularly run into trouble where I try to upgrade a bunch of ports on a FreeBSD installation somewhere, and bam suddenly I get build errors around locales or gettext shared libraries.
Now, I do like my applications to be able to speak French just as much as the next guy, but somehow the amount of grief that this has been causing me over the years is getting to be significant.
Latest issue is trying to rebuild ports on 3 different machines following an upgrade to FreeBSD 9.1. All went smoothly with the O/S upgrade using freebsd-update, but trying to rebuild the ports after fetching the latest versions using portsnap is proving harder than expected (or warranted).
I have downloaded the latest ports, and am trying to do the equivalent of "
So, hunting around in "
This whole i18n thing seems to violate the principle of least astonishment.
Anyway, just a rant because I am frustrated. I'll go back to it and attempt to do things right this time... but I really do hate gettext.
P.S. Sigh... even sudo fails with "
P.P.S. Finally solved it by recompiling the world and kernel from source... and then "
I make hourly snapshots of the ZFS filesystems on my home server. This is quite easy
using a small script called "
This keeps a day's worth of hourly snapshots, a month's worth a daily backups, and a year's worth of weekly backups (I should perhaps also keep a decade's worth of annual backups?!).
This is useful and can be really easily accessed... For example, I restored
Accessing snapshots through .../.zfs/snapshot is quite handy.
PS. For the record, the zfs-snapshot.sh script is here.Tue, 01 Jan 2013
Oh, and the good news of the end of 2012 was that FreeBSD 9.1 has been released (see here for the announcement).
I'll start installing it over the weekend on some of my machines. Before then, we need to fly back home and get back to work. Happy January 1st everyone!Sun, 15 Apr 2012
Having played with ZFS and USB disks over the past few weeks on FreeBSD 9.0, I have come to the conclusion that multiple USB disks + ZFS raid-z is not an excellent combination... USB disks can appear or disappear dynamically, and ZFS does not deal with this beautifully.
ZFS does deal with devices appearing and disappearing, but with an understanding that devices are mostly stable. So, I will use ZFS on internal disks, and for USB disks, I will limit myself to single-disk volumes (which seem to work fine).
I would be interested in a filesystem that would deal with disks being plugged in (or plugged out) dynamically (USB disks or otherwise), but I am not sure that this exists. I will hunt around for something like this.Sat, 17 Mar 2012
Because I am backing up a whole bunch of machines that may have identical files on them (many copies of Windows, old backups, etc), I am interested in finding ways of having n copies of a file not occupy n times the amount of storage.
I see three ways of doing this:
Stow looks interesting... I'll try to find some time to dig further into it.Sun, 19 Feb 2012
So, I have been playing with my small soekris net6501-70-based NAS device for home.
Lots of things have happened since I last blogged about this:
So, lots is happening with my small NAS server.
More on this next weekend.
I have just finished upgraing one of my FreeBSD machines from 8.2 to 9.0, rebuilding from source as explained in the FreeBSD handbook here.
One of the painful steps in upgrading a FreeBSD machine, is the mergemaster step, as it forces you to go through a whole bunch of files, just to accept CVS tag changes... or so I thought !!! Turns out that there are optional flags to mergemaster, and running "mergemaster -FiU" does pretty much what I've been hoping mergemaster to do. Yay!Sun, 12 Feb 2012
I have set up my Soekris net6501 with the latest beta of FreeNAS 8, FreeNAS-8.2.1-ALPHA-r10110M-x64. I installed 4 external 1 TB external USB disks on the machine, and set them up as a RAID-Z ZFS volume in FreeNAS and set about exporting CIFS and NFS partitions to the rest of my household. FreeNAS 8 is going to be an excellent product when it releases v.8.2.1... I look forward to seeing it in use in the coming months.
However, I have received the following FreeNAS alert twice now: The ZFS volume status is UNKNOWN: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'.
Interestingly enough, both times the error was corrected without any problem and the ZFS pool was back in HEALTHY state when I looked into it, but still that message sounds fairly omnious. Could it simply come from the fact that the drives go to sleep and are sometimes not ready when ZFS tries to access them (hence they somehow timeout)??? Or is it something more serious and one of my (new) drives is about to go south?
I wonder whether there are any recommendations against setting up a RAID-Z zpool across external USB drives that can go to sleep (and hence take 30-60 seconds to wake up). Hmmmm, I should investigate ZFS best practices and see if USB drives are usable this way. Else, I'll just stick with UFS with rsync for replication/backup.
More next weekend :-)Sun, 05 Feb 2012
A quick follow-up to my recent blog entry about identity servers on small home networks:
FreeNAS 8 is a nanobsd configuration, based on FreeBSD 8.2.
In order to boot FreeNAS 8 on my Soekris net6501 under the amd64 image of FreeNAS, one
needs to rebuild the FreeBSD kernel that is at the heart of FreeNAS with the line
Therefore, I used the instructions from the FreeNAS Forums here
to set up a VMware virtual machine running FreeBSD to setup the FreeNAS sources,
adjusted the kernel configuration file in "
FreeNAS 8 seems to look good on the net6501 and I am able to set it up with 2 external 2 TB USB disks partitioned with ZFS. For some strange reason, the FreeNAS 8 website seems to call for at least 6 GB of RAM to run it with ZFS, but my 2 GB net6501 seems to do just fine so far.
So, I just need to complete the set up of my FreeNAS box, export one of the disks through CIFS, NFS, and rsync, and set up a regular backup from disk1 to disk2. Once that runs fine, I can probably connect a 3rd 2 TB drive to that net6501, and set up some form of RAID-5 between the 3 drives (with snapshots, perhaps I don't need the weekly backup from disk1 to disk2, as I normally do on my NSLU2).
FreeNAS seems like a winner so far. More info later.
Miscellaneous comments & ramblings:
Anyway, enough for tonight... good night to all.Thu, 26 Jan 2012
So, on my home network I have:
and various other odds and ends.
However, it strikes me that I do not have a small and simple identity server, where I could define user accounts for all of my various devices and OSes (Windows, Mac OS X, FreeBSD, and probably some Linux also).
What do people use? NIS yellow pages? that seems too unixoid and will probably not help with the Windows machines. Some sort of Microsoft Active Directory? Through Samba? that feels too microsofty perhaps. Is there something that bridges the gap? Going beyond my little home network, is there something that would let me be accepted as an OpenId authenticator? Also, something I could use to allow logins into my machines hosted outside of the home network.
Hmmm. Over the years, we have gotten pretty good at doing IP networks, file sharing, printer sharing, web serving, and even web services and the like... but we are still struggling with having some form of simple authentication mechanism standardized. Probably because it is hard to have trust in authentication mechanisms, and especially the quality of the data they contain(*).
Anyway, that's another interesting weekend project.
(*) Perhaps we are setting the bar too high? I do not need strong authentication with non-repudiation and all that jazz... I would like a simple userid/password mechanism that would return something like the old unix uid/gid, but in 64-bit range, and I would like it to be easily integrated with Windows, Unix, and Mac OS X. Hmmm... fuzzy requirements... it's always more complicated than it looks like. What are people using out there? Any solid and usable public domain reference implementations I could look at and implement? Would Radius fit the bill?
For the last few years, I have been using a couple of Linksys NSLU2 NAS devices coupled with external USB disks for shared storage across my home network. I find that this works quite well to store files from my Mac, Windows, and even FreeBSD machines. One of the NSLU2 is unslung and runs an rsync daemon that I use to save files from my various FreeBSD machines.
The NSLU2 is a nice device, but it had become fairly old (no gigabit ethernet, limited expandability of the software, etc). So I have been meaning to set up a replacement. I have decided to use a Soekris net6501 to build my own little NAS box, using a couple of external 2 TB external drives.
Now, the question is: what software will I run on the net6501? I have been quite tempted by FreeNAS, but FreeNAS 8 has a stated minimum RAM size requirement of 4 GB (with 6 GB listed as the minimum for ZFS usage), and the net6501 is limited to 2 GB RAM (actually, I think that's even a limit of the Intel Atom CPU that it is built upon). FreeNAS 0.7 has a significantly lower RAM requirement, but I am afraid that it will be orphaned, as the main development path for FreeNAS is now version 8.
Of course, I could just set up FreeBSD on the net6501 and be done with it.
Anyway, I still have a few weeks to decide, as I am waiting for a new BIOS release for the net6501, as I am having some issues with the new USB boot code of the Soekris device: if I plug in 2 large (2 TB) external USB disks, in addition to the small internal 8 GB USB key that holds the FreeBSD image, then the device hangs on boot while probing the USB devices. If I just connect the external USB drives after the machine is booted, it works perfectly. I am hoping that the USB probe code will be fixed in the next BIOS release.
Fun for the weekend :-)Sat, 21 Jan 2012
So, I investigated last week's problem with ZFS and FreeBSD 9.0 and I discovered that the problem was very simple to fix. I needed to add:
to the procedure described last week (on January 15th) so that the zfs mountpoint options would be executed. So now, things are working fine... I am going to play with this configuration a bit to see if the fact that I haven't configured a swap space causes problem with a 2GB memory configuration when I try to compile world.
Late addition to this post: turns out that the author of the initial page at http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE had also found the error and fixed it on the wiki, but I had not gone back to check. In the end, it's OK, this has allowed me to explore ZFS enough so that I now understand it better.Sun, 15 Jan 2012
With the new version of FreeBSD being just out, I decided to try out the ZFS filesystem. Notably, I would like to find out whether it works correctly with 2GB of memory, or whether more is really needed (I read that it doesn't really work with less than 1GB of RAM, but then again, there are conflicting reports that it needs lots more, though sometimes that's qualified with 'if you want prefetch' or 'if you are trying to do dedup'). Actually, I am building a small NAS server I am building around a Soekris net6501-70 which comes with 2GB RAM, and I would like to find out if I can use ZFS with it.
So, I looked up http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE and tried to set up a vmware virtual machine with ZFS only.
I did not bother setting up swap space, nor a mirrored disk configuration, so the commands I used were:
The last 4 commands (i.e. after 'zfs set mountpoint=legacy zroot') give error messages, but I think that's OK. However, once I rebooted the virtual machine, the filesystems (except for /) were not mounted and therefore FreeBSD did not come up properly.
So, this was a first try... More on this next weekend :-)Fri, 13 Jan 2012
Yay! FreeBSD 9.0 is out!
The announcement says:
Please see http://www.FreeBSD.org for more info.
Note also that FreeBSD 9.0-RELEASE is dedicated to the memory of Dennis M. Ritchie, one of the fathers of UNIX.Wed, 11 Jan 2012
Found the ISO images for FreeBSD 9.0-RELEASE on ftp.freebsd.org and ftp4.fr.freebsd.org.
I've already got the amd64 version installed on a vmware player virtual image successfully from the ISO -- which works beautifully with FreeBSD under Windows 7; check it out!
I'm off to Zurich for a couple of days... more FreeBSD over the weekend.Mon, 09 Jan 2012
I have been setting up a small computer -- a Soekris net6501-70 -- to use as a NAS (Network Attached Storage) device on my home network. The idea is to replace the small, reliable, Linksys NSLU2 (slugs) that I have been using for shared network usage in my home, with a new device with more disk space and Gigabit Ethernet ports since my home network (at least the wired part) is Gigabit Ethernet.
Ideally, I would have liked to run FreeNAS on the device and to run ZFS as a filesystem, but it didn't turn out that way, as the net6501 has 2 GB RAM, and FreeNAS requires a lot more than that.
Therefore I have installed FreeBSD 9.0-RC3 on my net6501-70 and I even had the pleasure to discover that I could run FreeBSD in amd64 mode (i.e. 64-bits mode).
Now, I cannot wait for FreeBSD 9.0 to be finally and officially released, and for the next iteration of the ne6501 combios to be effectively release.