Upgrading from Lucid to Maverick using a local mirror

I maintain a local Ubuntu mirror in our Lab at work. This morning the first user tried to update from Lucid to Maverick. There was a couple of teething issues but with a couple of changes it went off without a hitch.

Step 1: Correctly specify your mirror to the Software Sources dialog:

1. Open a terminal
   
   (Applications -> Accessories -> Terminal)
2. Open your list of mirrors:
   

   trastle$ sudo gedit /usr/share/python-apt/templates/Ubuntu.mirrors

3. Add a location for your local mirror. Mine looks like this:
   

   ...
   #LOC:ADL
   http://adl-mirror/ubuntu-repo/ubuntu/
   ...

4. Save and close the file.
5. Close the terminal.

Step 2: Select your mirror in the software sources dialog

1. Open the software sources dialog
   (System -> Administration -> Software Sources)
2. Open the "Ubuntu Software" tab
3. Click the "Download From:" combo box.
4. Select "Other..."
5. Select the mirror you just added from the list
6. Click "Choose Server" at the bottom right.
7. Click "Close" at the bottom right.
8. Click "Close" in the information out of date dialog that pops up.

Step 3: Disable all third party sources

1. Open the software sources dialog
   (System -> Administration -> Software Sources)
2. Open the "Other Software" tab
3. Untick every source in the list.
4. Click "Close" at the bottom right.
5. Click "Close" in the information out of date dialog that pops up.

Step 4: Do the Upgrade

1. Ensure your on a network with Internet access as well as access to your mirror.
2. Open a terminal
   (Applications -> Accessories -> Terminal)
3. Change to the root user:
   

   trastle$ sudo -i

4. Update your available software:
   

   root# apt-get update 

5. Do the upgrade
   

   root# do-release-upgrade 

6. The following message will appear shortly after starting the upgrade:
   

   Updating repository information
   WARNING: Failed to read mirror file
   
   No valid mirror found 
   
   While scanning your repository information no mirror entry for the 
   upgrade was found. This can happen if you run a internal mirror or if 
   the mirror information is out of date. 
   
   Do you want to rewrite your 'sources.list' file anyway? If you choose 
   'Yes' here it will update all 'lucid' to 'maverick' entries. 
   If you select 'No' the upgrade will cancel.

   Your using a local mirror so choose Yes
7. The release upgrade will take some time to complete.

Step 5: Re-enable third party sources

1. Open the software sources dialog
   (System -> Administration -> Software Sources)
2. Open the "Other Software" tab
3. Tick the sources you want to use.
4. Click "Close" at the bottom right.
5. Click "Reload" in the information out of date dialog that pops up.

Enjoy your upgraded system.
I hope someone finds this helpful.

Building a ARM powered Debian VM with QEMU on Ubuntu Lucid

I have recently spent some time trying to get an emulated ARM machine up and running on an x86 Ubuntu Lucid host. Initially I wanted Ubuntu for the client OS but I found that the only ARM installers available were for the Ubuntu Netbook Edition, which does not suit my needs at this point. So I went with Debian Lenny (the current Debian stable release) which has a myriad of ARM installers available.

When tracking down info about the Debian ARM port you'll quickly discover that there are two ports available ARM and ARMEL. ARM is the original port (now deprecated) and ARMEL is name of the newer code stream which supports ARM EABI.

The emulation software I am using is QEMU. An open-source project started by Fabrice Bellard (who also founded the ffmpeg project!). QEMU is fairly versatile and will emulate a myriad of CPU architectures including ARM, Sparc, PPC and even s390 (under KVM).

The Ubuntu Lucid repositories do contain packages for QEMU. However after installing via apt-get I found the binary failed out with a segmentation fault almost immediately. Not to worry the QEMU build system is easy to master.

Part 1: Building and installing QEMU:

1. Download the QEMU source (0.12.4, the current release, is available here):
   

   trastle$ wget http://download.savannah.gnu.org/releases/qemu/qemu-0.12.4.tar.gz

2. Unpack the tar ball:
   

   trastle$ tar -xvvf qemu-0.12.4.tar.gz

3. Install the required packages to build QEMU:
   

   trastle$ sudo apt-get build-dep qemu

4. Build and install QEMU:
   

   trastle$ cd qemu-0.12.4
   trastle$ ./configure
   trastle$ make
   trastle$ sudo make install

Part 2: Install Debian Lenny in QEMU:

1. Change to the directory you want to build your VM in:
   

   trastle$ mkdir ~/arm
   trastle$ cd ~/arm

2. Download the current build of the Debian Lenny ARMEL kernel (vmlinuz) and installer (initrd) images from a local Debian mirror:
   

   trastle$ wget ftp://ftp.au.debian.org/debian/dists/lenny/main/installer-armel
            /current/images/versatile/netboot/vmlinuz-2.6.26-2-versatile
   trastle$ wget ftp://ftp.au.debian.org/debian/dists/lenny/main/installer-armel
            /current/images/versatile/netboot/initrd.gz

3. Create a disk image for QEMU (more details on qemu-img).
   Importantly, the raw format will allow you to mount the image from Ubuntu once its populated:
   

   trastle$ qemu-img create -f raw armdisk.img 8G

4. Start the Debian install with QEMU:
   

   trastle$ qemu-system-arm -m 256 -M versatilepb \
            -kernel ~/arm/vmlinuz-2.6.26-2-versatile \
            -initrd ~/arm/initrd.gz \
            -hda ~/arm/armdisk.img -append "root=/dev/ram"

   QEMU will open a terminal window and within that window the Lenny installer will kick into action. Follow the installer's directions and allow the install to begin. Installation will be slower than normal. Allow 4 or 5 hours for it to complete.
   
   
5. As the install completes you'll be informed no boot loader is present, don't worry QEMU takes the place of the boot loader. Once the install completes the VM will reboot kicking off the installer again, don't proceed, just kill QEMU.

Part 3: Running your ARM Lenny install in QEMU:

1. Once the installation is complete you will need to copy the initrd from the installed system. To do this you must mount the QEMU disk image.
   

   trastle$ mkdir ~/arm/mount/
   trastle$ sudo mount -o loop,offset=32256 ~/arm/armdisk.img ~/arm/mount
   trastle$ cp ~/arm/mount/boot/initrd.img-2.6.26-2-versatile ~/arm/.
   trastle$ sudo umount ~/arm/mount
   

   32256 is not just a random number, it's the sector where the first disk partition begins.
   
   
2. Boot your Debian Lenny VM:
   

   trastle$ qemu-system-arm -m 256 -M versatilepb \
            -kernel ~/arm/vmlinuz-2.6.26-2-versatile \
            -initrd ~/arm/initrd.img-2.6.26-2-versatile \
            -hda ~/arm/armdisk.img -append "root=/dev/sda1"

Now you have a running ARM VM to do with as you please.

Boot to single user mode in Ubuntu 10.04 Lucid Lynx

This is symptomatic of how often I break X.org but I often need to boot my Ubuntu 10.04 machine into single user mode and fix some configuration file before rebooting into the GUI. I forget how to do it every time so here is how:

1. Hold the Shift key at boot to display the Grub boot menu.
2. Select the top Grub entry, it will be similar to:

   Ubuntu, with Linux 2.6.32-22-generic-pae

3. Hit e to edit the the Grub entry.
4. Find the line that looks like this:

   linux /boot/vmlinuz-2.6.32-22-generic-pae
   root=UUID=xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx
   ro vga=794 quiet splash

5. Change the line to:

   linux /boot/vmlinuz-2.6.32-22-generic-pae
   root=UUID=xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx
   init=/bin/bash rw

6. Hit crtl+x to boot

Your machine will now do a one time boot into single user mode. Once you reboot the changes you just made to Grub will be reverted.

** /bin/sh corrected to /bin/bash thanks Hola2040.

Activate the cron daemon on a DroboFS

I recently purchased a DroboFS NAS from DataRobotics. The device runs a minimal Marvell Linux OS on top of a low power ARM CPU (ARM926EJ-S).

The DroboFS supports third party development through the DroboApps platform. This allows end users to compile ARM applications using the GNU tool chain and run them on their DroboFS. On top of this these applications can be run as services using a small service API provided by DroboApps.

A number of DroboApps are available to enable dormant services on the DroboFS including the SSH, HTTP and FTP servers.

The DroboFS ships with a cron daemon tucked away at /usr/sbin/crond however there is no DroboApp available to enable the daemon. To rectify this I have written a small DroboApp script to activate the cron daemon at boot.

To get cron running as a service you'll need to do the following:

1. Enable DroboApps on your DroboFS.
   
2. Install the Dropbear SSH DroboApp.
   
3. SSH into your DroboFS
4. Create a directory for the cron DroboApp:
   

   mkdir /mnt/DroboFS/Shares/DroboApps/crond

5. Save the following service.sh into your cron DroboApp directory
   

   #!/bin/sh
   # ------------------------------------------------------------
   # service.sh for cron DroboApp
   #
   # Exposes the crond binary existing on the DroboFS as a 
   # DroboApps service.
   # ------------------------------------------------------------
   
   # Binaries used
   AWK="/usr/bin/awk"
   GREP="/bin/grep"
   CROND="/usr/sbin/crond"
   DATE="/bin/date +%Y:%m:%d-%H:%M:%S" # Nicely formatted date
   ECHO="/bin/echo"
   PS="/bin/ps"
   
   # Load the DroboApps service functions
   . /etc/service.subr
   
   # Required DroboApps variables
   prog_dir=`dirname \`realpath $0\``
   name="crond"                    # service name
   version="1.14.2"                # program version
   pidfile=${prog_dir}/crond.pid # location of pid file
   logfile=${prog_dir}/crond.log # location of log file
   
   # Start crond
   start()
   {
     # Start the service
     $CROND
   
     # Create the pidfile
     pid=`$PS | $GREP $CROND | $GREP -v grep | $AWK '{print $1}'`
     $ECHO $pid > $pidfile
   }
   
   case "$1" in
     start)
       start_service
       $ECHO "`$DATE` Started cron service" >> $logfile    
       ;;
     stop)
       stop_service
       $ECHO "`$DATE` Stopped cron service" >> $logfile    
       ;;
     restart)
       stop_service
       sleep 3
       start_service
       $ECHO "`$DATE` Restarted cron service" >> $logfile    
       ;;
     status)
       status
       ;;
     *)
       $ECHO "Usage: $0 [start|stop|restart|status]"
       exit 1
       ;;
   esac
   

6. Restart your DroboFS.

Now the cron daemon is running from boot on your DroboFS. To add tasks for the cron daemon to run you will need to SSH into your DroboFS and run the following:

mkdir -p /var/spool/cron/crontabs
crontab -e

I hope someone finds this helpful.

2010-06-24: Edit added "mkdir -p /var/spool/cron/crontabs" Thanks pimvanderzwet

Atheros AR5212 in Ubuntu 10.04 Lucid Lynx

I've just finished a fresh install of Ubuntu Lucid Lynx 10.04 on my crusty old Thinkpad T60. Everything is working great with the exception of my Atheros AR5212 wireless card.

Clicking the network manager applet in the panel I see the wireless listed as "Device not ready". This is tedious.

The solution to this issue is to remove the mainline ATK5K wireless driver from your Kernel and replace it with non-mainline MadWifi driver (source). MadWifi has superior support for the AR5212 card. This shouldn't take you more than about 20 minutes to sort out.

1. Open a terminal: Aplications -> Accessories -> Terminal
2. To check on your wireless card and make sure you have the AR5212:
   

   trastle$ sudo lshw -c network

   The result will look similar to this:
   

   *-network
       description: Wireless interface
       product: AR5212 802.11abg NIC
       vendor: Atheros Communications Inc.
       physical id: 0
       bus info: pci@0000:03:00.0
       logical name: wifi0

   Now you know your sporting an Atheros AR5212.
3. Install some software you'll need to build MadWifi:
   

   trastle$ sudo apt-get install subversion linux-kernel-headers build-essential \
   libssl-dev

4. Now go to your desktop
   

   trastle$ cd ~/Desktop 

5. Make a directory to put the MadWifi source in:
   

   trastle$ mkdir madwifi-src

6. Get the latest MadWifi source:
   

   trastle$ svn checkout http://madwifi-project.org/svn/madwifi/trunk madwifi-src

7. Change into your new source directory:
   

   trastle$ cd madwifi-src

8. Change to the root user:
   

   trastle$ sudo -i

9. Blacklist the non working ATH5K drivers:
   

   root$ echo "# Block ATH5K" >> /etc/modprobe.d/blacklist
   root$ echo "blacklist ath9k" >> /etc/modprobe.d/blacklist
   root$ echo "blacklist ath5k" >> /etc/modprobe.d/blacklist

10. Change to the MadWifi source directory
    

    root$ cd /home/[your user]/Desktop/madwifi-src

11. Build and install the MadWifi driver:
    

    root$ make && make install -d

12. Add the MadWifi driver to your Kernel
    

    root$ echo ath_pci >> /etc/modules

Now save anything else you have been working on and reboot your laptop. After the reboot your wireless will be working nicely.

2010-06-20: Edit added step 10. Thanks Xi.

Building XMLStarlet in Snow Leopard (OSX 10.6)

XMLStarlet is a command line XML toolkit. Very useful if you want to parse some data from a webpage, RSS feed of similar. XMLStarlet does not provide a binary version for Snow Leopard. They do however provide their source under the MIT licence. So your free to download, build and modify.

To get XMLStarlet up and running on Snow Leopard is nice and easy.

1. Install Xcode with the Unix the Unix Dev Support Option (see my past post about building wget for more details on this).
   
   
2. Go download yourself a copy of the XMLStarlet source (I am using v 1.0.1).
   
   
3. XMLStarlet requires libxml2 and libxslt. Snow Leopard ships with both these libraries installed, however they are not in the location expected by the XMLStarlet build script so some symlinks are required. Enter the following in a terminal:

   sudo ln -s /usr/lib/libxslt.1.dylib /usr/lib/libxslt.a
   sudo ln -s /usr/lib/libexslt.dylib /usr/lib/libexslt.a
   sudo ln -s /usr/lib/libxml2.dylib /usr/lib/libxml2.a
   
   

4. Unpack the XMLStarlet tarball by double clicking it in Finder.
   Take note of the name of the newly unpacked directory (something like xmlstarlet-1.0.1).
   
   
5. Open a terminal and cd to where you just unpacked (by default this will be the directory below).

   cd ~/Downloads/xmlstarlet-1*
   
   

6. Run the configure script:

   ./configure
   
   

7. Run the builds

   make
   
   

8. Run the installer:

   sudo make install
   
   

9. Finally, to test that the install worked correctly enter the following in a terminal:

   TA:~ troy$ xml 
   XMLStarlet Toolkit: Command line utilities for XML
   ... 
   XMLStarlet is a command line toolkit to query/edit/check/transform XML documents 
   (for more information see http://xmlstar.sourceforge.net/)
   
   

Enjoy.