Tuesday, November 29, 2011

Using --exclude-from in rsync

I've been toying with some rsync scripts tonight and after reading the rsync man page I still needed a little trial and error to determine the exact behaviour of the --exclude-from flag.

The --exclude-from flag specifies a file that contains exclude patterns (one per line). The rsync man page is VERY detailed on the subject of filters and how they can be used.

Looking at a simplified version of my scenario, consider the following directory structure:
   Source
   |---.DS_Store
   |---Alpha
   |  |---.DS_Store
   |  |---Document 1.txt
   |  |---Document 2.txt
   |  `---Temporary Items
   |     |---.DS_Store
   |     |---TI1.tmp
   |     `---TI2.tmp
   |---Beta
   |  |---.DS_Store
   |  |---Document 3.txt
   |  |---Document 4.txt
   |  `---Temporary Items
   |     |---.DS_Store
   |     |---TI2.tmp
   |     `---TI4.tmp
   `---Temporary Items
      |---.DS_Store
      `---TI5.tmp
I want backup the Source directory shown above and exclude the following from my backup:
  1. All of the ".DS_Store" files.
  2. All of the "Temporary Items" directories.
  3. All files contained in the "Temporary Items" directories.
My final backup will look like this:
   Destination
   |---Alpha
   |  |---Document 1.txt
   |  `---Document 2.txt
   `---Beta
      |---Document 3.txt
      `---Document 4.txt

Making this happen is very simple. The exclusions file (skip.txt) only needs to contain the following two lines:
Temporary Items
.DS_Store
The rsync command to perform the backup is as follows:
rsync -az  --exclude-from=./skip.txt ./Source/ ./Destination/
The output of this command is the "Destination" directory shown above.

There were two things that caused me greif getting my script to work:
  1. To exclude a file (or directory) with spaces in the name, the names DO NOT need to be quoted or escaped.
  2. To exclude all files underneath a directory you need only exclude the directory itself.
Now that this is committed to my long term memory I can move on.

Monday, November 21, 2011

Fixing unmappable character during Java compilation on Ubuntu

In the lab we have a standard header at the top of our source files, a copyright statement and licence. This header contains an ISO 8859-1 encoded copyright character (©).

Normally the header blurs into the background, the exception to this rule is the first time a do a build on a new Ubuntu system and I see the following error:

[javac] /src/com/yourcorp/HelloWorld.java:8: unmappable character for encoding ASCII
[jacac] /* ?? Copyright Yourcorp 2011

The Java compiler is expecting ASCII characters rather than ISO 8859-1. To resolve this error I do the following:
  1. Edit /var/lib/locales/supported.d/local and add:
    en_AU ISO-8859-1
    en_US ISO-8859-1
    
    
  2. Open a terminal and run:
    $ sudo dpkg-reconfigure locales
    
    
  3. Edit /etc/environment and add:
    LANG="EN_US"
    
    
  4. Reboot.
With Ubuntu being so stable these days it's a long time between re-installs so I have posted this here for next time when I forget.