How to add a new module search path

Once in a while Python users are asking how to add some directories to sys.path permanently. Usually a solution like the PYTHONPATH env variable are suggested to the op. Other solutions require root privileges or modify the search path for all users. PEP 370 adds another way that is more clean and easy to use. It doesn't require root privileges and it doesn't suffer from other issues. PYTHONPATH causes trouble for multiple Python versions. C extensions only work for one version of Python, most Python modules won't work on Python 2 and 3.

My preferred way adds additional search pathes just for one version of Python and just for me. It uses a .pth file as explained in the site module manual. .pth files only work in site-packages directories, either the global or the user specific directories.

The Python way

$ python2.6
>>> import os
>>> import site
>>> site.USER_SITE

'/home/heimes/.local/lib/python2.6/site-packages'

Create the directory if it doesn't exist yet

>>> if not os.path.isdir(site.USER_SITE):
... os.makedirs(site.USER_SITE)
...

mypath.pth is going to contain my list of addition search path

>>> mypth = os.path.join(site.USER_SITE, "mypath.pth")
>>> path_to_add = ["/home/heimes/modules", "/home/heimes/other_modules"]

Add a list of search paths line by line, also make sure we end with an empty line

>>> with open(mypth, "a") as f:
... f.write("\n".join(path_to_add))
... f.write("\n")
...
>>>

The bash way

$ python2.6 -m site --user-site
/home/heimes/.local/lib/python2.6/site-packages
$ mkdir -p $(python2.6 -m site --user-site)
$ echo "/home/heimes/more_modules" >> $(python2.6 -m site --user-site)/mypath.pth

Let's check if it works

check the pth file

$ cat $(python2.6 -m site --user-site)/mypath.pth
/home/heimes/modules
/home/heimes/other_modules
/home/heimes/more_modules

Let's see if the modules are in the new search path ... they aren't because the directories don't exist yet.

$ python2.6 -m site
sys.path = [
'/home/heimes',
'/usr/lib/python2.6',
'/usr/lib/python2.6/plat-linux2',
'/usr/lib/python2.6/lib-tk',
'/usr/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload',
'/home/heimes/.local/lib/python2.6/site-packages',
'/usr/lib/python2.6/dist-packages',
'/usr/lib/python2.6/dist-packages/PIL',
'/usr/lib/python2.6/dist-packages/gst-0.10',
'/var/lib/python-support/python2.6',
'/usr/lib/python2.6/dist-packages/gtk-2.0',
'/var/lib/python-support/python2.6/gtk-2.0',
'/var/lib/python-support/python2.6/pyinotify',
'/usr/lib/python2.6/dist-packages/wx-2.6-gtk2-unicode',
'/usr/local/lib/python2.6/dist-packages',
]
USER_BASE: '/home/heimes/.local' (exists)
USER_SITE: '/home/heimes/.local/lib/python2.6/site-packages' (exists)
ENABLE_USER_SITE: True

create one example directory

$ mkdir /home/heimes/modules
$ python2.6 -m site
sys.path = [
'/home/heimes',
'/usr/lib/python2.6',
'/usr/lib/python2.6/plat-linux2',
'/usr/lib/python2.6/lib-tk',
'/usr/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload',
'/home/heimes/.local/lib/python2.6/site-packages',
'/home/heimes/modules',
'/usr/lib/python2.6/dist-packages',
'/usr/lib/python2.6/dist-packages/PIL',
'/usr/lib/python2.6/dist-packages/gst-0.10',
'/var/lib/python-support/python2.6',
'/usr/lib/python2.6/dist-packages/gtk-2.0',
'/var/lib/python-support/python2.6/gtk-2.0',
'/var/lib/python-support/python2.6/pyinotify',
'/usr/lib/python2.6/dist-packages/wx-2.6-gtk2-unicode',
'/usr/local/lib/python2.6/dist-packages',
]
USER_BASE: '/home/heimes/.local' (exists)
USER_SITE: '/home/heimes/.local/lib/python2.6/site-packages' (exists)
ENABLE_USER_SITE: True

Easy, isnt' it?