To build lxml from source, you need libxml2 and libxslt properly installed, including the header files. These are likely shipped in separate -dev or -devel packages like libxml2-dev, which you need to install. The build process also requires setuptools. The lxml source distribution comes with a script called ez_setup.py that can be used to install them.
The lxml.etree and lxml.objectify modules are written in Pyrex. Since we distribute the Pyrex-generated .c files with lxml releases, however, you do not need Pyrex to build lxml from the normal release sources.
If you are interested in building lxml from a Subversion checkout or want to be an lxml developer, you do need a working Pyrex installation.
lxml 1.1 and later
Newer versions of lxml depend on features and bug fixes that are not yet available in an official Pyrex release. This includes support for the external C-API of lxml.etree, for Python 2.5 and for 64 bit architectures.
To build lxml 1.1 and later from non-release or modified sources, you must therefore use an updated Pyrex version from here:
https://github.com/lxml/lxml/tree/master/pyrex/
A subversion checkout of lxml will automatically retrieve the latest Pyrex as external project source (svn:externals). Look for the Pyrex directory in the source tree.
Since version 1.1.2, the lxml source distribution also includes this Pyrex version. It will be used if the Pyrex directory is available in the lxml root directory. If you install from SVN or delete this directory from the unpacked distribution directory, the normally installed Pyrex version will be used.
lxml 1.0 and earlier
The 1.0 series build with a standard installation of Pyrex 0.9.4.1. Note that Pyrex up to and including version 0.9.4 has known problems when compiling lxml with gcc 4.x or Python 2.4. Do not use it. If you want to build lxml from non-release sources, please install Pyrex version 0.9.4.1 or later.
Pyrex now supports EasyInstall, so you can install it by running the following command as super-user:
easy_install Pyrex
The lxml package is developed in a Subversion repository. You can retrieve the current developer version by calling:
svn co https://github.com/lxml/lxml/tree/master/trunk lxml
This will create a directory lxml and download the source into it. You can also browse the repository through the web or use your favourite SVN client to access it.
Usually, building lxml is done through setuptools. Do a Subversion checkout (or download the source tar-ball and unpack it) and then type:
python setup.py build
or:
python setup.py bdist_egg
If you want to test lxml from the source directory, it is better to build it in-place like this:
python setup.py build_ext -i
or, in Unix-like environments:
make
If you get errors about missing header files (e.g. libxml/xmlversion.h) then you need to make sure the development packages of both libxml2 and libxslt are properly installed. If this doesn't help, you may have to add the location of the header files to the include path like:
python setup.py build_ext -i -I /usr/include/libxml2
where the file is in /usr/include/libxml2/libxml/xmlversion.h
To use lxml.etree in-place, you can place lxml's src directory on your Python module search path (PYTHONPATH) and then import lxml.etree to play with it:
# cd lxml # PYTHONPATH=src python Python 2.5.1 Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import etree >>>
To recompile after changes, note that you may have to run make clean or delete the file src/lxml/etree.c. Distutils do not automatically pick up changes that affect files other than the main file src/lxml/etree.pyx.
The source distribution (tgz) and the Subversion repository contain a test suite for lxml. You can run it from the top-level directory:
python test.py
Note that the test script only tests the in-place build (see distutils building above), as it searches the src directory. You can use the following one-step command to trigger an in-place build and test it:
make test
This also runs the ElementTree and cElementTree compatibility tests. To call them separately, make sure you have lxml on your PYTHONPATH first, then run:
python selftest.py
and:
python selftest2.py
If the tests give failures, errors, or worse, segmentation faults, we'd really like to know. Please contact us on the mailing list, and please specify the version of lxml, libxml2, libxslt and Python you were using, as well as your operating system type (Linux, Windows, MacOs, ...).
This is the procedure to make an lxml egg for your platform:
The last 'upload' step only works if you have access to the lxml cheeseshop entry. If not, you can just make an egg with bdist_egg and mail it to the lxml maintainer.
The Unix environment in Mac-OS X makes it relatively easy to install Unix/Linux style package management tools and new software. However, it seems to be hard to get libraries set up for exclusive usage that Mac-OS X ships in an older version. The result can be segfaults on this platform that are hard to track down.
To make sure the newer libxml2 and libxslt versions are used (e.g. under fink), you should add the directory where you installed the libraries to the DYLD_LIBRARY_PATH environment variable. This seems to fix a lot of problems for users.
Alternatively, you can build lxml statically. A way to do this on MS Windows is described in the next section, but it should be easy to adapt it for Mac-OS. That way, you can always be sure you use the versions you compiled lxml with, regardless of the runtime environement.
Most operating systems have proper package management that makes installing current versions of libxml2 and libxslt easy. The most famous exception is Microsoft Windows, which entirely lacks these capabilities. It can therefore be interesting to statically link the external libraries into lxml.etree to avoid having to install them separately.
Download lxml and all required libraries to the same directory. The iconv, libxml2, libxslt, and zlib libraries are all available from the ftp site ftp://ftp.zlatkovic.com/pub/libxml/.
Your directory should now have the following files in it (although most likely different versions):
iconv-1.9.1.win32.zip libxml2-2.6.23.win32.zip libxslt-1.1.15.win32.zip lxml-1.0.0.tgz zlib-1.2.3.win32.zip
Now extract each of those files in the same directory. This should give you something like this:
iconv-1.9.1.win32/ iconv-1.9.1.win32.zip libxml2-2.6.23.win32/ libxml2-2.6.23.win32.zip libxslt-1.1.15.win32/ libxslt-1.1.15.win32.zip lxml-1.0.0/ lxml-1.0.0.tgz zlib-1.2.3.win32/ zlib-1.2.3.win32.zip
Go to the lxml directory and edit the file setup.py. There should be a section near the top that looks like this:
STATIC_INCLUDE_DIRS = [] STATIC_LIBRARY_DIRS = [] STATIC_CFLAGS = []
Change this section to something like this, but take care to use the correct version numbers:
STATIC_INCLUDE_DIRS = [
"..\\libxml2-2.6.23.win32\\include",
"..\\libxslt-1.1.15.win32\\include",
"..\\zlib-1.2.3.win32\\include",
"..\\iconv-1.9.1.win32\\include"
]
STATIC_LIBRARY_DIRS = [
"..\\libxml2-2.6.23.win32\\lib",
"..\\libxslt-1.1.15.win32\\lib",
"..\\zlib-1.2.3.win32\\lib",
"..\\iconv-1.9.1.win32\\lib"
]
STATIC_CFLAGS = []
Add any CFLAGS you might consider useful to the third list. As Ashish Kulkarni notes, you might have to add the standard Windows library wsock32.dll to the list of libraries to make lxml.objectify compile.
Now you should be able to pass the --static option to setup.py and everything should work well. Try calling:
python setup.py bdist_wininst --static
This will create a windows installer in the pkg directory.
Andreas Pakulat proposed the following approach.
In case dpkg-buildpackage tells you that some dependecies are missing, you can either install them manually or run apt-get build-dep lxml.
That will give you .deb packages in the parent directory which can be installed using dpkg -i.