smc.freeimage 0.0.2

A while ago I noticed that Christoph Gohlke has unofficial Windows builds of my smc.freeimage wrapper on his hosting site. It made me aware that some people actually use my image processing software.

A few minutes ago I synced our internal SVN repository with the project site on berlios.de. The recent version uses Cython 0.14 to wrap most of FreeImage 3.15.0 and a limited subset of LCMS 2.1. It can read over 30 image formats including subsets like G3 and G4 compressed TIFFs, which aren't supported by PIL. smc.freeimage also supports limited ICC transformation with embedded or external ICC profiles (for now 24bpp RGB images only) and introspection of ICC profiles. The ICC transformation is optimized with cached transformations and no-copy processing.

smc.freeimage wraps only functionality we actually need at work. The core features are heavily tested in production. I estimate that we have processed betwann 300 TB to about half a Petabyte of data, mostly uncompressed TIFF images but also compressed TIFFs, PNGs and JPEGs.

If you are interested on adding features or building a more general solution, feel free to contact me.

How to compile Python on Ubuntu 11.04

At work we deploy and compile our own Python environment on all server in order to have full control over versions, patches and libraries. Three days ago I stumbled upon a problem in our build process on Ubuntu Natty. Several modules like zlib weren't available. It took me a while to figure out the problem. Python's setup.py simply couldn't find libz.so in it's usual search paths like /usr/lib.

Natty has introduced a new feature called multiarch. Some shared libraries are installed in architecture specific directories, e.g. /usr/lib/x86_64-linux-gnu/libz.so instead of /usr/lib/libz.so. The dynamic linker ld.so has been modified to look for libraries in the new locations. If you wonder how, /etc/ld.so.conf.d/x86_64-linux-gnu.conf does the trick. However Python's setup.py uses hard coded paths and doesn't know about the new feature. Barry's posting [1] has some insight information.

The problem has been dealt with for Python 2.7, 3.1, 3.2 and newer versions, but 2.6 and earlier won't see any fixes. Current Python releases (2.7.1, 3.2.0) suffer from the issue, too. Don't be battle-weary! The solution to the issue is rather simple.

   $ make distclean
   $ export LDFLAGS="-L/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)"
   $ ./configure
   $ make
   $ make install
   $ unset LDFLAGS

This adds an additional library search path (-L/usr/lib/x86_64-linux-gnu on my box). Now setup.py knows about the multiarch lib directory and builds zlib and all the other missing modules just fine.

Actually my build system a bit more paranoid. It also adds /lib/x86_64-linux-gnu as library search path and /usr/include/x86_64-linux-gnu as header include path for C and C++.

   $ export arch=$(dpkg-architecture -qDEB_HOST_MULTIARCH)
   $ export LDFLAGS="-L/usr/lib/$arch -L/lib/$arch"
   $ export CFLAGS="-I/usr/include/$arch"
   $ export CPPFLAGS="-I/usr/include/$arch"
   $ ./configure
   $ make
   $ make install
   $ unset arch LDFLAGS CFLAGS CPPFLAGS

[1] https://lists.ubuntu.com/archives/ubuntu-devel/2011-April/033049.html