Blogging on a leap day

So... 2012 is an Olympic year. It is a leap year. 2012 is divisible by 4, but not by 100 -- unless it was divisible by 400 (which explains why Y2k was a leap year). So there you go, I can blog on a February 29th.

Don't forget to check your watches today... all of the ones with simple algorithms for computing the number of days in each month will probably get it wrong. Of course, you should be OK if you are wearing a German Junghans radio-controlled watch that synchronizes nightly with an atomic clock signal in Europe, Japan, or North America (see more here)... or if you are running ntp software on your machine (see more here). Funniest thing about ntp is when Tony and I installed a GPS antenna on top of the building in London to get a time signal for our global ntp servers -- we had some fun explaining why we needed a GPS receiver on a building :-)

Always synchronize your computer clocks with ntp... there's no reason not too.

Oh, and since there still time... just check out the 10000-year clock here and here.

Update: woah... check this out: a major outage of Azure apparently caused by a leap year calculation problem! This is impressive: as an industry, we've been struggling with date/time problems for 50 years or so, and we're still making leap-year or summer-time-change mistakes in 2012. Sheeeeesh!

/misc | Posted at 18:51 | permanent link

Jean Dujardin, The Artist, an Oscar...

So... it has come to this.

/misc | Posted at 15:06 | permanent link

Aren't NASes fun?

Apart from everything going on at work (and there is a lot to do there), and a fairly significant birthday in mid-week (this is where you find you've got a lot of great friends), I have found some time to play around with my NAS project.

Here's the latest:

1. The HP Proliant Microserver arrived after 2 days. This is really good service from HP France and it comes with a great rebate (33% -- check it out on the HP France website). It is quite nice hardware; well put together. If it's reliable, I'll be very pleased with my purchase.
2. I am moving 3 2TB SATA disks into it. I've installed FreeBSD 9.0 on the machine, and will put the 3 disks in a RAID-Z configuration under ZFS. This will give me a 4TB NAS box that does NFS, CIFS, and rsync (oh, and perhaps Subversion too). This is quite nice.
3. I am finding though that moving Terabytes of information around isn't easy. I am also finding that ext3 filesystem support in FreeBSD isn't great, and neither is ufs support in Linux... both are kind of OK; but in trying to copy 1.8TB of backup from my old NSLU2 onto a couple of external USB drives, I encounter limitations both on the FreeBSD and on the Linux side. Pfffff. My ultimate solution: boot liveCDs onto 2 machines, KNOPPIX on the Microserver, FreeBSD 9 on my main PC, connect the USB 3.0 external drive on the PC, fire up a rsync daemon on the PC, and rsync the data from the disk in the Microserver across the Gigabit Ethernet and onto the PC. It's overly intricate, but it does the job at a good speed and with native FS support on both sides. I will take hours to complete though.
4. I thought I could throw in my HP USB 3.0 PCIe card into the Microserver, but somehow it doesn't work... FreeBSD or Linux see the card; but the USB devices plugged into the ports don't get any power!?! I would venture that the card is fried somehow. Not sure how it happened; though.
5. While doing this, I have also changed the TimeMachine backup disk on an old iMac to a 2.5" disk instead of the 1TB disk it used to have... I'll use the later for backups on on the Microserver.

Anyway, I need some form of USB 3.0 support on the Microserver... 40 MB/s is just too slow.

More on this later.

| Posted at 15:49 | permanent link

Going for simple FreeBSD instead of FreeNAS

So, I have been playing with my small soekris net6501-70-based NAS device for home.

Lots of things have happened since I last blogged about this:

1. I decided not to use FreeNAS 8 on my net6501, but rather to just use FreeBSD 9 with ZFS, NFS, Samba, and rsync.
2. I have copied my olde NSLU2 shared space onto my net6501 under ZFS... it took over 24 hours to copy over the network, the NSLU2 being fairly slow.
3. I have set up a script to take snapshots of my ZFS storage on a regular basis using a script I found on the internet: zfs-snapshot.sh... this is almost as nice as have the Mac OS X time machine on my little NAS server (ok, not as nice, but still pretty good)
4. I have had some difficulty with an old Maxstor 1TB USB drive, which seems to appear/disappear... it has caused corruption in my main ZFS pool, which I needed to fix manually, and then scrub the ZFS pool (which took almost 7 hours).
5. Given all of this, I have ordered a small HP microserver. External USB drives are handled strangely by server O/Ses as they appear/disappear, change device names, etc. Let see how well the microserver with internal drives will do.
6. By the way, when dealing with External USB disks, never use device names (such as /dev/da0 or /dev/da1s1a...); these names can change dynamically as drives are plugged in, or powered off, etc. Always label your filesystems (e.g. for UFS: tunefs -L disk1 /dev/da0s1a, which then appears as /dev/ufs/disk1; or for GPT disks: /dev/gptid/760ae9e2-5988-12f1-a136-000054cf2648) and use these names in ZFS or fstab... this will avoid a LOT of grief when reboting, or disconnecting or reconnecting a device, or indeed when a device fails.

So, lots is happening with my small NAS server.

More on this next weekend.

/FreeBSD | Posted at 19:01 | permanent link

Upgrading FreeBSD from 8.2 to 9.0

I have just finished upgraing one of my FreeBSD machines from 8.2 to 9.0, rebuilding from source as explained in the FreeBSD handbook here.

One of the painful steps in upgrading a FreeBSD machine, is the mergemaster step, as it forces you to go through a whole bunch of files, just to accept CVS tag changes... or so I thought !!! Turns out that there are optional flags to mergemaster, and running "mergemaster -FiU" does pretty much what I've been hoping mergemaster to do. Yay!

/FreeBSD | Posted at 16:12 | permanent link

FreeNAS, ZFS, and external USB disks

I have set up my Soekris net6501 with the latest beta of FreeNAS 8, FreeNAS-8.2.1-ALPHA-r10110M-x64. I installed 4 external 1 TB external USB disks on the machine, and set them up as a RAID-Z ZFS volume in FreeNAS and set about exporting CIFS and NFS partitions to the rest of my household. FreeNAS 8 is going to be an excellent product when it releases v.8.2.1... I look forward to seeing it in use in the coming months.

However, I have received the following FreeNAS alert twice now: The ZFS volume status is UNKNOWN: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'.

Interestingly enough, both times the error was corrected without any problem and the ZFS pool was back in HEALTHY state when I looked into it, but still that message sounds fairly omnious. Could it simply come from the fact that the drives go to sleep and are sometimes not ready when ZFS tries to access them (hence they somehow timeout)??? Or is it something more serious and one of my (new) drives is about to go south?

I wonder whether there are any recommendations against setting up a RAID-Z zpool across external USB drives that can go to sleep (and hence take 30-60 seconds to wake up). Hmmmm, I should investigate ZFS best practices and see if USB drives are usable this way. Else, I'll just stick with UFS with rsync for replication/backup.

More next weekend :-)

/FreeBSD | Posted at 17:34 | permanent link

4 wins in a row for the Montreal Canadiens

I listened to the hockey game between the Montreal Canadiens and the Toronto Maple Leafs last night on ''internet radio''.

Montreal won 5-0 which gives them a 4-games winning streak (haven't seen too many of these this season!). NHL hockey is fun, but it's always particularly fun when Montreal beats Toronto or Boston :-) (Now, of course, these guys probably feel particularly smug when they beat the Canadiens, but I'm not going to worry about that :-)

/misc | Posted at 21:14 | permanent link

Using virtual machines at home

I have been architecting, recommending, and specifying the use of irtual machines at work for years, but I had never really used them seriously at home yet until recently. I had looked into using either Parallels or Fusion on my Macs to run Windows seamlessly, but in the end had always reverted to just using Apple's bootcamp, which always did the job for what I needed (except for the @!$&#^!%# graphics drivers in bootcamp which never seemed to be up to snuff).

However, given my recent purchase of a decent PC, I looked into the possibility of running FreeBSD in a virtual machine to take advantage of the CPU speed and sizable RAM to compile kernels for some of my smaller machines faster (running a whole buildkernel buildworld installkernel installworld cycle on a Soekris net4801 takes over 24 hrs which isn't fun).

I investigated various options, but it turns out that the best/most convenient is to run the vmware player under Windows 7 on my PC... using that, I can install FreeBSD 9 in either 32-bit or 64-bit mode, and just "play" whichever version of FreeBSD I need to run tests (or compilations) in a window. There is something inherently fun in having FreeBSD boot and discover the hardware of the virtual machine. Quite useful, quite powerful... just fascinating.

Of course, we are living in a world where one can run a full PC emulator written in javascript in a browser, and boot linux onto it, as I have blogged about recently here. The IT world is fascinating: every 5 years we reinvent the impossible.

/software | Posted at 19:03 | permanent link

Watching the superbowl on TV

It's nice to be able to watch Superbowl XLVI on TV, right here in Paris. The TNT channel W9 is showing the game direct from Indianapolis. It's a bit late (actually, it was already tomorrow here in Paris, as the game started :-) but it's quite nice to watch!

2nd quarter's almost over, and the NY Giants have been dominating the game from the start... but the score is only 9-3: the Patriots still have a chance if they get their act together.

...it is cold in Paris (-5 Celcius), so it's nice to watch the game under a warm blanket... I may drift to sleep during the halftime show though: my alarm clock is set for 04:40 in the morning.

Update 1: 10-9 for the Patriots at the end of the 2nd quarter. Hey, game's not over! Not the time to fall alseep yet!
Update 2: good, enjoyable, halftime show by Madonna! An impressive performer who knows how to put on a show.
Update 3: 17-15 Patriots at the end of the 3rd quarter. The Giants have been leaving too many points on the field: it's going to be tough for them. Good game so far... let's see if the 4th quarter keeps me awake :-)
Final update: Giants win 21-17. What a game. Off to sleep now.

/misc | Posted at 21:53 | permanent link

Identity services

A quick follow-up to my recent blog entry about identity servers on small home networks:

• I have been playing with FreeRadius (available on FreeBSD as part of the ported packages -- /usr/ports/net/freeradius2, I believe) to see if I could use that as an identity server for my home network... to be investigated further in the coming weeks.
• On a loosely related topic, I find the myOpenID services interesting, especially the "myOpenID for domains" stuff. It seems like I could use this to enable OpenIDs like http://openid.rax.org/my-name to work? It looks like it might be free if I have 10 IDs or less on my domain. Interesting. However, I need to see what level of independence I would need to lose on my domain (including the ability to change providers in the future) if I did that.
• More interesting stuff to investigate on future weekends :-)

/FreeBSD | Posted at 18:13 | permanent link

What this blog is not about

By the way, just in case anybody wanders here and is curious as to whether anything I write here is related to personal hobbies or work, let me state explicitly that I do not intend to blog about work-related topics or express any work-related opinions here.

Work-related social networking belongs on work-sanctioned forums or communities... so this blog is purely about personal stuff and interests, and any opinions expressed here are my own.

/notwork | Posted at 17:47 | permanent link

Moving forward with FreeNAS

FreeNAS 8 is a nanobsd configuration, based on FreeBSD 8.2. In order to boot FreeNAS 8 on my Soekris net6501 under the amd64 image of FreeNAS, one needs to rebuild the FreeBSD kernel that is at the heart of FreeNAS with the line "device mptable" added to the kernel configuration file, as described on the unofficial Soekris wiki here.

Therefore, I used the instructions from the FreeNAS Forums here to set up a VMware virtual machine running FreeBSD to setup the FreeNAS sources, adjusted the kernel configuration file in "/usr/local/freenas/trunk/nanobsd/FREENAS.amd64" to add the device mptable line as described above, and then built a FreeNAS-8.2.1-ALPHA-r10006M-x64 image that I put onto a USB key and booted my Soekris net6501 with it.

FreeNAS 8 seems to look good on the net6501 and I am able to set it up with 2 external 2 TB USB disks partitioned with ZFS. For some strange reason, the FreeNAS 8 website seems to call for at least 6 GB of RAM to run it with ZFS, but my 2 GB net6501 seems to do just fine so far.

So, I just need to complete the set up of my FreeNAS box, export one of the disks through CIFS, NFS, and rsync, and set up a regular backup from disk1 to disk2. Once that runs fine, I can probably connect a 3rd 2 TB drive to that net6501, and set up some form of RAID-5 between the 3 drives (with snapshots, perhaps I don't need the weekly backup from disk1 to disk2, as I normally do on my NSLU2).

FreeNAS seems like a winner so far. More info later.

Miscellaneous comments & ramblings:

• I still need Soekris to fix the USB boot code on the net6501 before I consider this completely operational: the net6501 will freeze on boot when my two 2 TB external USB drives are connected at boot time. Works fine when I connect them after the machine has booted... if at least Soekris could add an option in the BIOS to not scan certain USB ports for bootable devices, I could move forward.
• FreeNAS 8 is based on FreeBSD 8.2, which is unfortunate as it only has ZFS v15, as opposed to FreeBSD 9.0 which comes with ZFS v28 (and perhaps the upcoming FreeBSD 8.3 also?)
• There appears to be a certain amount of tension between the "new" FreeNAS 8 team, and the "legacy" FreeNAS 7 team. The latter has based FreeNAS 7.5 on FreeBSD 9.0 to get the newest FreeBSD stuff (including ZFS v28 as stated above) and is starting to talk about calling its branch FreeNAS 9, which is not going over too well with the FreeNAS 8 people (who actually own the FreeNAS trademark). Seems like FreeNAS 7 will either die or else it will need to fork off with another name (see here).
• I don't have a strong opinion on this at this point, except for the fact that I am disappointed that FreeNAS 8 specifies minimum RAM requirements which seem to me to be unreasonably high (4 GB minimum, and 6 GB minimum if one wants ZFS) for some of the small machines that one could want to run FreeNAS upon. I like the FreeNAS 7 community effort, but I fear that it will end up as a dead-end effort once FreeNAS 8 is solid & feature-complete.
• Anyway, I decided to set up my NAS server as a FreeNAS 8 server, to see if it ran OK in 2 GB of RAM (which, frankly, I don't see why it couldn't).
• Seems like the net6501 will work as a home NAS device anyway, so finally I don't need the HP Proliant Microserver I was referring to the other day.

Anyway, enough for tonight... good night to all.

/FreeBSD | Posted at 17:39 | permanent link

Compilers in the Open Source world

When Gnu started, way back, one of the first bricks to be laid out in the open source building, was the gcc compiler. Okay, first there was emacs, but that was always a bit of a peculiar animal (to this day, I will sometimes use alternatives such as Gnu Zile instead of plain old emacs).

So, gcc was one of the first bricks that led the open source edifice to be built: compilers were always expensive and complicated software packages before that, but then suddenly, one could compile C code freely, and the compiler wasn't too shabby either: being developed by a large group of individuals from industry and academia, it produced very decent code and its optimizer generated pretty fast code too! (I remember using gcc on a project in 1990 because the code it generated on our project under ultrix ran quite a bit faster than the code generated by the native ultrix compiler).

So it's a bit strange these days to see that there are a number of alternative compilers being used by my favorite open source projects (notably FreeBSD): the old pcc is back, and FreeBSD seems to favour the clang compiler (see also this comparison).

Of course, there's also the question of which language to code in: C, C++ (there's a new 2011 flavour of that), Objective C (hey, isn't everyone doing an iPhone app these days? :-), Java, Scheme, python, Ruby, or any number of weird and wonderful languages. Even 'lowly' Javascript is geting quite powerful... Fabrice Bellard has written a PC emulator in it that's powerful enough to run linux in your browser!

Anyway, lots of possibilities. Just have fun coding!

Update: take a look at the TIOBE Programming Community Index of language popularity.

/software | Posted at 19:06 | permanent link

Listening to music

I still purchase music CDs and DVDs -- I guess that makes me somewhat of a dinosaur. I do listen to music online (through Spotify) which allows me to listen to a whole bunch of tunes that I do not have at home but fondly remember from the '80s, the '70s, or even earlier. I also rent movies in iTunes to watch on my iMac or my PC... however, on iTunes, I only use prepaid cards: I don't like leaving my credit card number for recurring purchases... especially since I find that Visa and MasterCard will agree to a recurring credit card charge even if the credit card expiry date has passed. It used to be that you would need to submit your credit card info again, once it has expired, but no longer: they'll just charge you anyway. The first time this hit me was with Apple's MobileMe service: I had let my credit card lag beyond expiration, since I did not intend to renew for one more year, but they still charged my card. Annoying to say the least: clearly not a implementation of the Principle of least astonishment. :-(

Anyway, this being said, when watching movie content, I should ideally set up a Mac Mini or some sort of Apple TV in the living room so that I can watch rented movies (i.e. downloaded content) in the comfort of the living room. Haven't done that yet.
Something on the To Do list, I guess, but kind of low priority.

/misc | Posted at 17:33 | permanent link