SVN did a great job for me for at least four years now. Lately, I stumbled upon Mercurial and decided to have a closer look. It turned out that Mercurial (also called “hg”) comes with some great advantages, namely:

• Not a “Version Control System” (VCS), but a “Distributed Version Control System” (DVCS) enabling you to check in changes locally (offline, without an internet connection). The advantages of this one are also discussed in detail in PEP 374 (Python Foundation).
• Available in pure Python, therefore installable on a standard shared web server without root access (not a virtual server, just a “web host”). Thereby cutting my costs of maintaining a root server on my own or being charged by a specialized host for an SVN/VPS plan.

This post is a small guide on how to accomplish the mission of installing Mercurial (the pure Python version) on a shared web host. In this exemplary case a “WebPack Pro” that we rent from our ISP of choice, HostEurope, is going to be used. However, what is described herein should basically work for any web hosting plan that fulfills the following requirements. This article is therefore most interesting for people who already own a shared web hosting plan and wish to use that plan for hg hosting also.

Requirements

• Source version of Mercurial
• A shared web server at an arbitrary web host enabling execution of CGI scripts written in Python (HostEurope provides Python 2.4 which appears to be sufficient).
• mod_python or mod_wsgi (better).
• SSH access to that host. This is required to setup Mercurial on your shared web server. No need for root privileges.

Installation (Step by Step)

1. Unpack the Mercurial source tarball on your local system.
2. Create a temporary directory on your web server and transfer the extracted Mercurial source tree to that directory. You can do so via SSH or FTP.
3. SSH into the server (if you did not so already) and create a top-level directory named “python”. This is going to hold all the Python modules you would like to use, but which are not currently installed on the server’s root Python distribution.
4. Change to the Mercurial source directory and build the module in pure Python mode (instructions adopted from Mercurial Wiki) :

   $ python setup.py --pure build_py -c -d . build_ext -i build_mo --force
   

5. Likewise, install the module. Here, it is essential to enter the correct prefix path, as you want an alternate installation following the prefix scheme, not a standard installation. Please specify the path to your “python” directory, which you created in step 3 (in my case this is “../../python”):

   $ python setup.py --pure install --prefix ../../python --force
   

6. If everything went well setup.py has installed the Mercurial module files to python/lib/python2.4/site-packages and the hg binary to python/bin.
7. In order to get the hg binary running via SSH you have to export your custom PYTHONPATH so Python knows where to look for the hg modules. Via SSH go to your home directory on the web host and create a .bash_profile file as follows:

   $ cd ~
   $ touch .bash_profile
   

   Open the file in a text editor and add the following line to it (adjust the path so that it fits your configuration):

   export PYTHONPATH=/is/htdocs/wpXXXXX_XXX/python/lib/python2.4/site-packages
   

   In order to make hg accessible from any directory within the shell, add the following line (adjust path corresponding to your environment again):

   export PATH=/is/htdocs/wpXXXXX_XXX/python/bin:$PATH
   

8. Logout and login again via SSH. Type hg. If you get the Mercurial Distribution SCM help screen you now have finished installation and are ready to go for setting up Mercurial web access.

Setting up hgwebdir.cgi

1. First of all, create a directory on your webspace for hg. This directory must be reachable via HTTP. You may also want to set up a subdomain, e.g. hg.yourdomain.com and link that to your hg web directory.
2. Get hgwebdir.cgi from the root of the Mercurial source tree or download hgwebdir.cgi from the Mercurial website, upload it to your hg directory and rename it to index.cgi.
3. Make sure your web host is configured appropriately to handle .cgi files and index.cgi files. (You should be able to configure these settings in the scripts settings of your ISP’s administrative UI.)
4. Open index.cgi using your text editor of choice. It is a good idea to check the script environment statement that is located in the first line of the file. It should look like the following:

   #!/usr/bin/env python
   

   This should work on most servers. However, if your python binary is not linked (or located) in /usr/bin/python you may have to correct for this.

5. Find the following comment # enable importing on demand to reduce startup time. Beneath the comment line there should be two commented lines. Change these lines to something like this:

   import sys
   sys.path.insert(0, "/is/htdocs/wpXXXXX_XXX/python/lib/python2.4/site-packages")
   

   Replace the absolute path to the site-packages directory with your own one.

6. If everything works as intended, the script should now be callable via HTTP. Enter the address of your hg web directory into your web browser and check whether the script runs correctly.

Setting Up Your First Repository and Configuring hgweb.config

1. Via SSH you can setup a first test repository like this:

   $ cd /path/to/www/hg
   $ hg init test
   

   hg will create a directory named test and place a .hg subdirectory in it containing repository-specific configuration and payload data.

2. hgwebdir.cgi has to know which repositories to serve. You tell this by creating a file named hgweb.config. Simply put the config file into the directory of hgwebdir.cgi.
3. Open the config file with a text editor and add the following lines:

   [paths]
   test=test
   

   Note: more info on how to configure hgwebdir.cgi can be found on HgWebDirStepByStep (in the Mercurial Wiki).

4. Finally, the repository itself needs some configuration. In test/.hg create a new file named hgrc and add the following lines to it:

   [ui]
   user = Firstname Lastname 
   
   [web]
   push_ssl = false
   allow_push = *
   

   This will register a first user, allow for HTTP pushing, and allow pushing for everyone (we will take care of security later on).

5. That’s it. Your first repository is served via Mercurial on your shared web host.

Testing the Whole Thing

You may want to do a simple test to check whether your repository is working and you are able to pull and push changes as intended. You can do so by opening a terminal and type something like the following:

$ hg clone http://hg.yourdomain.com/index.cgi/test
$ cd test
$ touch testfile
$ echo "This is just a test" > testfile
$ hg add testfile
$ hg commit -m 'Testing...' -u username
$ hg push

If everything went right, this should add the file testfile to the repository root and push it to the web server.

For now, you hopefully have a working Mercurial repository directory service up and running on your shared web host. However, there are two major flaws. Firstly, the repository address is not as beautiful as it could be, because of the index.cgi contained in any path. Secondly, the repositories are accessible for everyone. We will fix these issues in what follows.

Beautifying hgwebdir Paths

In order to strip index.cgi from the repository URLs we will have to use mod_rewrite (and your server needs to support that apache module).

Go to your hg webdir root directory and create a file named .htaccess. Open the file in a text editor and add the following lines:

# Taken and adapted from http://www.pmwiki.org/wiki/Cookbook/CleanUrls#samedir
# Used at http://ggap.sf.net/hg/
RewriteEngine On
#write base depending on where the base url lives
RewriteBase /
RewriteRule ^$ index.cgi  [L]
# Send requests for files that exist to those files.
RewriteCond %{REQUEST_FILENAME} !-f
# Send requests to hgwebdir.cgi, appending the rest of url.
RewriteRule (.*) index.cgi/$1  [QSA,L]

Note: this apache script rewrites URLs so that any directory queried in the form hg.yourdomain.com/directory is automatically rewritten to hg.yourdomain.com/index.cgi/directory internally. Consequently, users can omit disturbing repetitive typing of index.cgi in hg repository URLs. Please note that you may have to adapt RewriteBase to your specific server environment. If you do not serve hg repositories via a subdomain you should specify the relative base directory of your hg webdir here instead (this may e.g. be /hg if you serve hg repositories via www.yourdomain.com/hg). The approach is also discussed in PublishingMultipleRepositories on the Mercurial Wiki.

Securing hgwebdir

You can secure your hg web directory in multiple ways. You may want to allow pulling by anyone and pushing only to specific core developers. Detailed information about hg authentication can be found on PublishingMultipleRepositories.

However, my intention is to have private repositories, which I share only with my friends (no public pulling). Me and my friends will use HTTP authentication and I will create a user and password for everyone who is allowed to pull and push. Fortunately, this scenario is relatively simple to configure for everyone who knows about apache authentication configuration using htaccess files.

First of all, use htpasswd to create a password file for hg users. Name the file .hgusers and place it in a location that is not accessible via HTTP. For example, you may want to do it this way:

$ cd /private-path
$ htpasswd -c hgusers username
Enter password twice, etc...

Since we already have a .htaccess file, it is even more simple. Re-open the file using a text editor and add the following lines at the end of the file:

# Authentication
AuthName "Please enter your username and password."
AuthType Basic
Require valid-user
AuthUserFile /is/htdocs/wpXXXXX_XXX/.hgusers

Please change the path to the .hgusers file conforming to your server environment.

Congratulations! You now have a running, secured, beautified, pure-python hg repository server on your cheap web hosting account.

Remarks
Building Mercurial in pure Python may have drawbacks in terms of performance. This has more thoroughly been discussed on the Mercurial mailing list, see mailing list archive on pure python Mercurial.

First, I’d like to apologize for the long time I wasn’t blogging anything. I was working on my M.Sc. Thesis which is now finished. Btw.: we used of005 for the visualization part of a fancy machine learning experiment. Perhaps you are interested in it. The doc is online here.

However, the actual reason why I type all this is that a lot of people have asked me whether I am about to provide an update of the openFrameworks 005 binding since 006 has been released. I am sorry to say that even though I plan to do so, I do not have the time to do it right now. What’s more, I don’t really know how long it will take, as there are some major changes in this of release.

WhyNotUnmount 0.3 is out now. Version 0.3 provides a number of stability improvements (multi-threading issues were fixed), improved unmounting of volumes, automatic list updating on device mounting / unmounting, and finally, several updated user interface details.

WhyNotUnmount works on Mac OS X 10.4 (Tiger) and 10.5 (Leopard) and comes as a Universal Binary.

For more information on what the application does, please see the original post on WhyNotUnmount.

Download WhyNotUnmount (DMG, 315 KB, Freeware)

A new openFrameworks-Python Binding for OS X 10.5 is finally available for download. The new binding comes with an Xcode project including the openFrameworks 005 source code and required libraries plus an example of how to use the binding. As in the previous versions, after building the binding using Xcode, you may run the test example as follows: open up a terminal, go to the build/release directory of the binding and type: ‘python animation.py’. The example script should run out of the box. If it does not please don’t hesitate to contact me.

Note: for those who want to build the binding for OS X 10.4, please see the readme file contained in the package. It contains hints on how to build the project using the 10.4 SDK.

Requirements: OS X 10.5.5, Xcode 3.1.1+, SWIG, Python 2.5(+)
Download openFrameworks-Python Binding for OS X 10.5

Recently I presented the openFrameworks-Python live coding proof of concept on this blog. Unfortunately, the first version was quite unstable, because code changes introducing syntax errors would crash the running application framework.

This has now been solved as suggested by Maddi with an exception handling block. That is, as soon as draw.py has syntax errors in it, the draw code’s exception is caught by the Python code in animation.py. Rendering stops under this condition, but does not crash the framework. What is more the type of exception is outputted on stdout. As soon as the error is corrected, rendering is automatically resumed.

Unfortunately stderr is still redirected to nirvana inside the overridden functions called by the openframeworks library. I appreciate any suggestions on this issue.

» Download openFrameworks-0.04-Python Live Coding POC

Pythonware’s PIL, the Python Imaging Library, is an extension to Python enabling you to process and arrange images quickly. It supports the popular JPEG and PNG image formats and provides font rendering using the freetype2 library. Unfortunately, PIL is not included in the standard Python environment on Mac OS X 10.4/10.5, so you have to download and build it.

First of all, you might find that there is plenty of forum/blog posts, articles etc. on the web attempting to provide useful instructions on how to build and install PIL in the right fashion. However, I found that most of them are telling different things and so I didn’t really trust them. I tried a lot and basically all of them failed. I am going to explain now, why that was.

Following the README what you need in advance is: libjpeg (6b), libpng3 and freetype2. You can get these using fink or MacPorts. Since I don’t like my machine to re-compile all the stuff that is easily available in a binary form, I used fink.

After installing the required libraries via fink I tried to build PIL using the command as described in the README file:

$ python setup.py install

… and all I got was a lousy:

--- TKINTER support ok
--- JPEG support ok
--- ZLIB (PNG/ZIP) support ok
*** FREETYPE2 supported not available

It turned out that the setup.py file did for some reason not search the correct directories for the freetype2 library. Fixing this is quite easy: just set

FREETYPE_ROOT = "/sw/lib/freetype2/lib"

Note: This does only apply as long as you use fink. MacPorts directories differ!

I again built PIL using the setup.py install command. Everything seemed OK this time. So, as suggested by the installer script, I tried to run PIL’s self test now using

python selftest.py

which would result in

Failure in example: _info(Image.open("Images/lena.jpg"))
from line #24 of selftest.testimage
Exception raised:
Traceback (most recent call last):
  File "./doctest.py", line 499, in _run_examples_inner
    exec compile(source, "", "single") in globs
  File "", line 1, in 
  File "./selftest.py", line 22, in _info
    im.load()
  File "PIL/ImageFile.py", line 180, in load
    d = Image._getdecoder(self.mode, d, a, self.decoderconfig)
  File "PIL/Image.py", line 375, in _getdecoder
    raise IOError("decoder %s not available" % decoder_name)
IOError: decoder jpeg not available
1 items had failures:
   1 of  57 in selftest.testimage
***Test Failed*** 1 failures.
*** 1 tests of 57 failed.

I googled and googled and found a lot of similar results, but no answers unfortunately. After some time I realized that PIL would run from any other directory, but only the selftest.py failed. The reason for this is that in the Imaging-1.1.6 directory there is a PIL.pth file and a PIL subdirectory. Python picks these up and tries to run PIL from there. Of course this cannot succeed, since the library is not built properly for our system there. The solution is to delete both the PIL subdirectory and the PIL.pth file and there it goes – the self test runs through without any failures.

Hope I have saved you some valuable time.

When it gets really cold in winter and you find your hands wrapped in thick gloves, it’s not that easy to pick up an iPhone call fast enough. CHCH.cc has a great solution on MacOSXHints.com: use your nose! I have googled around a bit to find some illustration of this (see YouTube video above).

Of course there are alternative solutions. For instance: iPhone-enabled Dots Gloves.

WhyNotUnmount 0.2 has been released today. A number of bugs are fixed now. WhyNotUnmount will now display all blocking processes regardless of whether they are UI or system processes.

For more information on what the program does, please see the original post on WhyNotUnmount.

Download WhyNotUnmount (DMG, 530 KB, Freeware)

Recently I promised to write another post on what cool stuff can be done with the openFrameworks-Python binding. Besides the fact that being able to quickly write openFrameworks apps with Python is cool enough actually, Python is an interpreted language and does therefore enable us to execute and change code dynamically — even at runtime. Hence, an obvious purpose for using this feature is live coding.

In order to get us there, modifications need to be made to the original source code of animation.py. More precisely, the draw function of the testApp class has to be changed to something like this:

def draw(self):
    file = open(oscwd + '/draw.py', 'r')
    source = file.read()
    exec(source)
    file.close()

This code snippet loads the file draw.py located in the current working directory, which has previously been stored to a variable called oscwd. draw.py is going to contain the actual code responsible for rendering the openFrameworks scene. As you can see on the screenshot you can now start animation.py, fire up a code editor (in this case Xcode) to edit draw.py and begin live coding. As soon as you save your changes to the draw.py file the modified code will be executed in the animation.py application.

So far so good, but there are still some issues: for some reason (which I will explain later) the code inside the testApp class has a different Python environment than the surrounding code. This causes stdout / stderr to be redirected into nirvana (what ever you try to put on stdout inside the class will not appear in the shell; errors in your code will not be prompted either). What is more, the os and sys functions do not work properly here. This is the reason why I temporarily stored os.getcwd() in the oscwd variable of the animation module. Finally, the execution of the whole application will be broken as soon as you save draw.py with syntax errors (and as mentioned previously, stderr will not be prompted on the shell in this case). Consequently, the whole approach is still quite unstable and should therefore be considered a POC.

Download the live coding POC files (ZIP file, 5.1 MB, requires Mac OS X 10.4.11+ and Python 2.5)

WhyNotUnmount is a small project which I have started to make life a little bit easier for all those people carrying external hard drives with their MacBook. Of course, it is suitable for all other kinds of Macs also. WhyNotUnmount attempts to tell you why a certain (external) hard drive, DMG or other volume cannot be unmounted. In most cases this is because a file on that disk remains opened by an application on your Mac. WhyNotUnmount tells you which Application prevents your volume from being unmounted properly.

The application is currently in alpha (prerelease) state, which means that there may still be a number of bugs and – more importantly – that it may not always detect the correct reason for why unmounting is not possible. I am working on these issues at the moment. However, WhyNotUmmount is freeware, so it may take a little while until it is completely stable. In the meantime enjoy the early version!

Requirements:
Mac OS X 10.4 (Tiger) or 10.5 (Leopard).
Details on the artefacts page / Direct download.