Incompatible changes

• lxml 0.9 introduced a feature called namespace implementation. The global Namespace factory was added to register custom element classes and have lxml.etree look them up automatically. However, the later development of further class lookup mechanisms made it appear less and less adequate to register this mapping at a global level, so lxml 1.1 first removed the namespace based lookup from the default setup and lxml 2.0 finally removes the global namespace registry completely. As all other lookup mechanisms, the namespace lookup is now local to a parser, including the registry itself. Applications that use a module-level parser can easily map its get_namespace() method to a global Namespace function to mimic the old behaviour.
• Some API functions now require passing options as keyword arguments, as opposed to positional arguments. This restriction was introduced to make the API usage independent of future extensions such as the addition of new positional arguments. Users should not rely on the position of an optional argument in function signatures and instead pass it explicitly named. This also improves code readability - it is common good practice to pass options in a consistent way independent of their position, so many people may not even notice the change in their code. Another important reason is compatibility with cElementTree, which also enforces keyword-only arguments in a couple of places.
• XML tag names are validated when creating an Element. This does not apply to HTML tags, where only HTML special characters are forbidden. The distinction is made by the SubElement() factory, which tests if the tree it works on is an HTML tree, and by the .makeelement() methods of parsers, which behave differently for the XMLParser() and the HTMLParser().
• XPath now raises exceptions specific to the part of the execution that failed: XPathSyntaxError for parser errors and XPathEvalError for errors that occurred during the evaluation. Note that the distinction only works for the XPath() class. The other two evaluators only have a single evaluation call that includes the parsing step, and will therefore only raise an XPathEvalError. Applications can catch both exceptions through the common base class XPathError (which also exists in earlier lxml versions).
• Network access in parsers is now disabled by default, i.e. the no_network option defaults to True. Due to a somewhat 'interesting' implementation in libxml2, this does not affect the first document (i.e. the URL that is parsed), but only subsequent documents, such as a DTD when parsing with validation. This means that you will have to check the URL you pass, instead of relying on lxml to prevent any access to external resources. As this can be helpful in some use cases, lxml does not work around it.
• The type annotations in lxml.objectify (the pytype attribute) now use NoneType for the None value as this is the correct Python type name. Previously, lxml 1.x used a lower case none.
• Another change in objectify regards the way it deals with ambiguous types. Previously, setting a value like the string "3" through normal attribute access would let it come back as an integer when reading the object attribute. lxml 2.0 prevents this by always setting the pytype attribute to the type the user passed in, so "3" will come back as a string, while the number 3 will come back as a number. To remove the type annotation on serialisation, you can use the deannotate() function.
• The C-API function findOrBuildNodeNs() was replaced by the more generic findOrBuildNodeNsPrefix() that accepts an additional default prefix.

Enhancements

Most of the enhancements of lxml 2.0 were made under the hood. Most people won't even notice them, but they make the maintenance of lxml easier and thus facilitate further enhancements and an improved integration between lxml's features.

• lxml.objectify now has its own implementation of the E factory. It uses the built-in type lookup mechanism of lxml.objectify, thus removing the need for an additional type registry mechanism (as previously available through the typemap parameter).
• XML entities are supported through the Entity() factory, an Entity element class and a parser option resolve_entities that allows to keep entities in the element tree when set to False. Also, the parser will now report undefined entities as errors if it needs to resolve them (which is still the default, as in lxml 1.x).
• A major part of the XPath code was rewritten and can now benefit from a bigger overlap with the XSLT code. The main benefits are improved thread safety in the XPath evaluators and Python RegExp support in standard XPath.
• The string results of an XPath evaluation have become 'smart' string subclasses. Formerly, there was no easy way to find out where a string originated from. In lxml 2.0, you can call its getparent() method to find the Element that carries it. This works for attributes (//@attribute) and for text() nodes, i.e. Element text and tails. Strings that were constructed in the path expression, e.g. by the string() function or extension functions, will return None as their parent.
• Setting a QName object as value of the .text property or as an attribute value will resolve its prefix in the respective context
• Following ElementTree 1.3, the iterfind() method supports efficient iteration based on XPath-like expressions.

The parsers also received some major enhancements:

• iterparse() can parse HTML when passing the boolean html keyword.
• Parse time XML Schema validation by passing an XMLSchema object to the schema keyword argument of a parser.
• Support for a target object that implements ElementTree's TreeBuilder interface.
• The encoding keyword allows overriding the document encoding.

Deprecation

The following functions and methods are now deprecated. They are still available in lxml 2.0 and will be removed in lxml 2.1:

• The tounicode() function was replaced by the call tostring(encoding=unicode).
• CamelCaseNamed module functions and methods were renamed to their underscore equivalents to follow PEP 8 in naming.
  • etree.setDefaultParser() -> etree.set_default_parser()
  • etree.getDefaultParser() -> etree.get_default_parser()
  • etree.useGlobalPythonLog() -> etree.use_global_python_log()
  • XMLParser.setElementClassLookup() -> .set_element_class_lookup()
  • HTMLParser.setElementClassLookup() -> .set_element_class_lookup()
• The .getiterator() method on Elements and ElementTrees was renamed to .iter() to follow ElementTree 1.3.

lxml.usedoctest

A very useful module for doctests based on XML or HTML is lxml.doctestcompare. It provides a relaxed comparison mechanism for XML and HTML in doctests. Using it for XML comparisons is as simple as:

>>> import lxml.usedoctest

and for HTML comparisons:

>>> import lxml.html.usedoctest

lxml.html

The largest new package that was added to lxml 2.0 is lxml.html. It contains various tools and modules for HTML handling. The major features include support for cleaning up HTML (removing unwanted content), a readable HTML diff and various tools for working with links.

lxml.cssselect

The Cascading Stylesheet Language (CSS) has a very short and generic path language for pointing at elements in XML/HTML trees (CSS selectors). The module lxml.cssselect provides an implementation based on XPath.