Saturday, November 21, 2009

Proxy environment variables and sudo in Ubuntu

At work we use a proxy server to access the internet from our test network. We also have an internal Ubuntu mirror on the test network to install updates and packages, this server must be accessed by the test machines directly without the proxy.

Setting up the proxy and the exception for the mirror can be done in a terminal using the environment variables http_proxy, https_proxy, ftp_proxy, and no_proxy.

In Ubuntu these variables are all set in the /etc/environment file. On my test system this file contains the following:

trastle@trastle-test:~$ cat /etc/environment
...
http_proxy="http://webproxy:3128/"
ftp_proxy="ftp://webproxy:3128/"
https_proxy="https://webproxy:3128/"
no_proxy="office-mirror"

Testing that these settings have been applied to the current terminal can be done by calling printenv:

trastle@trastle-test:~$ printenv | grep proxy
http_proxy=http://webproxy:3128/
ftp_proxy=ftp://webproxy:3128/
https_proxy=https://webproxy:3128/
no_proxy=office-mirror

This output shows that all of the required environment variables are set. However when I try to perform an update from the office mirror it fails! The http_proxy is being used to access hosts clearly specified in the no_proxy variable.

trastle@trastle-test:~$ sudo apt-get update
Ign http://office-mirror jaunty Release.gpg
...
Err http://office-mirror jaunty/main Packages
503 Service Unavailable
...
W: Failed to fetch http://office-mirror/ubuntu/dists/jaunty/main/binary-i386/Packages
503 Service Unavailable
...
E: Some index files failed to download, they have been ignored, or old ones used instead.

This all makes more sense when you check the environment variables that sudo is running with:

trastle@trastle-test:~$ sudo printenv | grep proxy
http_proxy=http://webproxy:3128/

For safety sudo runs with an minimal environment. This default list is hard coded into sudo (initial_keepenv_table in env.c if you want to go read the source). Ubuntu patches this list to include http_proxy but not no_proxy. This breaks the ability to get updates from the local mirror. Sudo can be queried to determine which environment variables it preserves by default:

trastle@trastle-test:~$ sudo sudo -V
Sudo version 1.6.9p17
...
Sudoers path: /etc/sudoers
...
Environment variables to preserve:
http_proxy
XAUTHORIZATION
XAUTHORITY
TZ
PS2
PS1
PATH
MAIL
LS_COLORS
KRB5CCNAME
HOSTNAME
HOME
DISPLAY
COLORS
...

The solution to this problem is to modify the list of environment variables that sudo will preserve by editing the /etc/sudoers file and providing an explicit list of which environment variables to be preserved, overwriting the default list. The configuration that must be added to /etc/sudoers is as follows:

Defaults env_keep="no_proxy http_proxy https_proxy ftp_proxy XAUTHORIZATION \
XAUTHORITY TZ PS2 PS1 PATH MAIL LS_COLORS KRB5CCNAME HOSTNAME HOME DISPLAY COLORS"

This will preserve all of the default environment variables as well as no_proxy, https_proxy and ftp_proxy. You can test this setting to prove that sudo is now preserving these extra environment variables by running:

trastle@trastle-test:~$
sudo printenv | grep proxy
http_proxy=http://webproxy:3128/
ftp_proxy=ftp://webproxy:3128/
https_proxy=https://webproxy:3128/
no_proxy=office-mirror

With these additional environment variables preserved updates and installs from the command line using your local mirror and ignoring your proxy server will work like a charm.

Thursday, October 29, 2009

Apache Directory Studio Fails to start on Ubuntu 9.04

I needed to install Apache Directory Studio this morning and after downloading unpacking version 1.4.0 I could not get it to run. Starting ADS at the command line resulted in the following output, the splash screen and a small sad blank window.

$ ./ApacheDirectoryStudio
0    [main] INFO  org.apache.directory.studio.Application  - Entering Apache Directory Studio.
4003 [main] INFO  org.apache.directory.studio.Application  - Exiting Apache Directory Studio.



Reading the logs in ~/.ApacheDirectoryStudio/.metadata/.log revealed an exception:

!ENTRY org.eclipse.ui.workbench 4 0 2009-10-29 09:29:30.476
!MESSAGE Widget disposed too early!
!STACK 0
java.lang.RuntimeException: Widget disposed too early!
at org.eclipse.ui.internal.WorkbenchPartReference$1.widgetDisposed(WorkbenchPartReference.java:171)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:117)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1158)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1182)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1163)
...

After a bit of a hunt through the ADS mailing lists I learned that this error is due to a problem with the Mozilla XULRunner version on my system.

In order to get ADS to run correctly I had to specify the version XULRunner it should run with manually:

./ApacheDirectoryStudio -vmargs -Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner/xulrunner-1.9.1

The original Mozilla bug is detailed here, and this solution for ADS found here.

Wednesday, October 28, 2009

Building wget in Snow Leopard

I like Snow Leopard, it's a nifty OS, but occasionally something will baffle me. This evening it was the exclusion of wget from the command line tools.



Clearly this sucks and living without wget is not an option. Thankfully wget is free software so it's easy to grab the source and build it yourself.

Firstly you'll need to have Xcode installed with "Unix Dev Support". The Unix Dev Support option gives you gcc, make and other essential goodies. Xcode is on the Mac OS 10.6 DVD in the optional installs section (you can also download Xcode from Apple).

Next you'll need to grab the latest wget source from: ftp://ftp.gnu.org/pub/gnu/wget/ or click here.

To compile and install wget complete the following:

  1. Unpack the wget tarball by double clicking it in Finder.
    Take note of the name of the newly unpacked directory (something like wget-1.12).
     
  2. Open a terminal and cd to where you just unpacked (by default this will be the directory below).
    cd ~/Downloads/wget-1.*
     
  3. Run the configure script:
    ./configure
     
  4. Run the build and installer:
    sudo make install
     
  5. All done. Now you can delete the wget folder and tarball in your ~/Downloads directory. 

Once your done you'll be able to run wget in your terminal.



Much better.

Thursday, October 22, 2009

When you forget to start screen...

Screen and nohup are great tools which can be used, by someone with forethought, to keep their processes running after they disconnect from a server.

The problem with both nohup and screen is that you need to use them before starting a process. If you forgot or didn't realize the process was going to take ages then your all out of luck.

Recently on a man page hunt I discovered a workaround for this situation using disown. At the terminal with the long running process enter the following:

ctrl + z  (pause the current process and background it)
bg        (resume the last backgrounded process in the background)
disown    (disassociate the last process from the current terminal) 
exit      (to close the current terminal)

Now your disconnected, your process is still running and you can go home. If you don't want to go home and instead want to check that your process is still running, ssh back into your server and run ps aux and hunt it down in the process list.