Creating objectify trees

To create an objectify tree, you can either parse a document with the parser you created:

>>> from StringIO import StringIO
>>> xml = StringIO('<test/>')
>>> tree = etree.parse(xml, parser)
>>> print isinstance(tree.getroot(), objectify.ObjectifiedElement)
True

or you can call the makeelement() method of the parser to create a new root element from scratch:

>>> obj_el = parser.makeelement("test")
>>> print isinstance(obj_el, objectify.ObjectifiedElement)
True

New subelements will automatically inherit the setup. However, all independent elements that you create through the normal etree API will not be associated with the parser and therefore not support the objectify API:

>>> subel = etree.SubElement(obj_el, "sub")
>>> print isinstance(subel, objectify.ObjectifiedElement)
True

>>> independent_el = etree.Element("new")
>>> print isinstance(independent_el, objectify.ObjectifiedElement)
False

The makeelement() method of the parser has the same signature as the normal Element() factory known from lxml.etree and can therefore easily replace the respective calls.

For convenience, objectify also replicates the standard factory Element() and the fromstring() function from lxml.etree using a parser that is local to the objectify module. So, after setting up the parser based element lookup above, you can keep using the same API as in lxml.etree, except that you have to import these functions from a different module:

>>> obj_el = objectify.Element("new")
>>> print isinstance(obj_el, objectify.ObjectifiedElement)
True

>>> obj_el = objectify.fromstring("<test/>")
>>> print isinstance(obj_el, objectify.ObjectifiedElement)
True

You can change this parser with objectify.setDefaultParser(parser), which also allows to add the above support for namespace specific element classes.

Element access through object attributes

The main idea behind the objectify API is to hide XML element access behind the usual object attribute access pattern. Asking an element for an attribute will return the sequence of children with corresponding tag names:

>>> root = objectify.Element("root")
>>> b = etree.SubElement(root, "b")
>>> print root.b[0].tag
b
>>> root.index(root.b[0])
0
>>> b = etree.SubElement(root, "b")
>>> print root.b[0].tag
b
>>> print root.b[1].tag
b
>>> root.index(root.b[1])
1

For convenience, you can omit the index '0' to access the first child:

>>> print root.b.tag
b
>>> root.index(root.b)
0
>>> del root.b

Iteration and slicing also obey the requested tag:

>>> x1 = etree.SubElement(root, "x")
>>> x2 = etree.SubElement(root, "x")
>>> x3 = etree.SubElement(root, "x")

>>> [ el.tag for el in root.x ]
['x', 'x', 'x']

>>> [ el.tag for el in root.x[1:3] ]
['x', 'x']

>>> [ el.tag for el in root.x[-1:] ]
['x']

>>> del root.x[1:2]
>>> [ el.tag for el in root.x ]
['x', 'x']

If you want to iterate over all children or need to provide a specific namespace for the tag, use the iterchildren() method. Like the other methods for iteration, it supports an optional tag keyword argument:

>>> [ el.tag for el in root.iterchildren() ]
['b', 'x', 'x']

>>> [ el.tag for el in root.iterchildren(tag='b') ]
['b']

>>> [ el.tag for el in root.b ]
['b']

XML attributes are accessed as in the normal ElementTree API:

>>> c = etree.SubElement(root, "c", myattr="someval")
>>> print root.c.get("myattr")
someval

>>> root.c.set("c", "oh-oh")
>>> print root.c.get("c")
oh-oh

In addition to the normal ElementTree API for appending elements to trees, subtrees can also be added by assigning them to object attributes. In this case, the subtree is automatically deep copied and the tag name of its root is updated to match the attribute name:

>>> el = objectify.Element("yet_another_child")
>>> root.new_child = el
>>> print root.new_child.tag
new_child
>>> print el.tag
yet_another_child

>>> root.y = [ objectify.Element("y"), objectify.Element("y") ]
>>> [ el.tag for el in root.y ]
['y', 'y']

The latter is a short form for operations on the full slice:

>>> root.y[:] = [ objectify.Element("y") ]
>>> [ el.tag for el in root.y ]
['y']

You can also replace children that way:

>>> child1 = etree.SubElement(root, "child")
>>> child2 = etree.SubElement(root, "child")
>>> child3 = etree.SubElement(root, "child")

>>> el = objectify.Element("new_child")
>>> subel = etree.SubElement(el, "sub")

>>> root.child = el
>>> print root.child.sub.tag
sub

>>> root.child[2] = el
>>> print root.child[2].sub.tag
sub

Note that special care must be taken when changing the tag name of an element:

>>> print root.b.tag
b
>>> root.b.tag = "notB"
>>> root.b
Traceback (most recent call last):
  ...
AttributeError: no such child: b
>>> print root.notB.tag
notB

Tree generation with the E-factory

To simplify the generation of trees even further, you can use the E-factory:

>>> E = objectify.E
>>> root = E.root(
...   E.a(5),
...   E.b(6.1),
...   E.c(True),
...   E.d("how")
... )

>>> print etree.tostring(root, pretty_print=True)
<root>
  <a>5</a>
  <b>6.1</b>
  <c>true</c>
  <d>how</d>
</root>

This allows you to write up a specific language in tags:

>>> ROOT = objectify.E.root
>>> TITLE = objectify.E.title
>>> TYPE = objectify.E.type

>>> root = ROOT(
...   TITLE("The title"),
...   TYPE(5)
... )

>>> print etree.tostring(root, pretty_print=True)
<root>
  <title>The title</title>
  <type>5</type>
</root>

Namespace handling

Namespaces are handled mostly behind the scenes. If you access a child of an Element without specifying a namespace, the lookup will use the namespace of the parent:

>>> root = objectify.Element("{ns}root")
>>> b = etree.SubElement(root, "{ns}b")
>>> c = etree.SubElement(root, "{other}c")

>>> print root.b.tag
{ns}b
>>> print root.c
Traceback (most recent call last):
    ...
AttributeError: no such child: {ns}c

You can access elements with different namespaces via getattr():

>>> print getattr(root, "{other}c").tag
{other}c

For convenience, there is also a quick way through item access:

>>> print root["{other}c"].tag
{other}c

The same approach must be used to access children with tag names that are not valid Python identifiers:

>>> el = etree.SubElement(root, "{ns}tag-name")
>>> print root["tag-name"].tag
{ns}tag-name

>>> new_el = objectify.Element("{ns}new-element")
>>> el = etree.SubElement(new_el, "{ns}child")
>>> el = etree.SubElement(new_el, "{ns}child")
>>> el = etree.SubElement(new_el, "{ns}child")

>>> root["tag-name"] = [ new_el, new_el ]
>>> print len(root["tag-name"])
2
>>> print root["tag-name"].tag
{ns}tag-name

>>> print len(root["tag-name"].child)
3
>>> print root["tag-name"].child.tag
{ns}child
>>> print root["tag-name"][1].child.tag
{ns}child

or for names that have a special meaning in lxml.objectify:

>>> root = objectify.XML("<root><text>TEXT</text></root>")

>>> print root.text.text
Traceback (most recent call last):
  ...
AttributeError: 'NoneType' object has no attribute 'text'

>>> print root["text"].text
TEXT

Recursive tree dump

To see the data types that are currently used, you can call the module level dump() function that returns a recursive string representation for elements:

>>> root = objectify.fromstring("""
... <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...   <a attr1="foo" attr2="bar">1</a>
...   <a>1.2</a>
...   <b>1</b>
...   <b>true</b>
...   <c>what?</c>
...   <d xsi:nil="true"/>
... </root>
... """)

>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 1 [IntElement]
      * attr1 = 'foo'
      * attr2 = 'bar'
    a = 1.2 [FloatElement]
    b = 1 [IntElement]
    b = True [BoolElement]
    c = 'what?' [StringElement]
    d = None [NoneElement]
      * xsi:nil = 'true'

You can freely switch between different types for the same child:

>>> root = objectify.fromstring("<root><a>5</a></root>")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 5 [IntElement]

>>> root.a = 'nice string!'
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 'nice string!' [StringElement]

>>> root.a = True
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = True [BoolElement]

>>> root.a = [1, 2, 3]
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 1 [IntElement]
    a = 2 [IntElement]
    a = 3 [IntElement]

>>> root.a = (1, 2, 3)
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 1 [IntElement]
    a = 2 [IntElement]
    a = 3 [IntElement]

Recursive string representation of elements

Normally, elements use the standard string representation for str() that is provided by lxml.etree. You can enable a pretty-print representation for objectify elements like this:

>>> objectify.enableRecursiveStr()

>>> root = objectify.fromstring("""
... <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...   <a attr1="foo" attr2="bar">1</a>
...   <a>1.2</a>
...   <b>1</b>
...   <b>true</b>
...   <c>what?</c>
...   <d xsi:nil="true"/>
... </root>
... """)

>>> print str(root)
root = None [ObjectifiedElement]
    a = 1 [IntElement]
      * attr1 = 'foo'
      * attr2 = 'bar'
    a = 1.2 [FloatElement]
    b = 1 [IntElement]
    b = True [BoolElement]
    c = 'what?' [StringElement]
    d = None [NoneElement]
      * xsi:nil = 'true'

This behaviour can be switched off in the same way:

>>> objectify.enableRecursiveStr(False)

Type annotations

The "type hint" mechanism deploys an XML attribute defined as lxml.objectify.PYTYPE_ATTRIBUTE. It may contain any of the following string values: int, long, float, str, unicode, none:

>>> print objectify.PYTYPE_ATTRIBUTE
{http://codespeak.net/lxml/objectify/pytype}pytype
>>> ns, name = objectify.PYTYPE_ATTRIBUTE[1:].split('}')

>>> root = objectify.fromstring("""\
... <root xmlns:py='%s'>
...   <a py:pytype='str'>5</a>
...   <b py:pytype='int'>5</b>
...   <c py:pytype='none' />
... </root>
... """ % ns)

>>> print root.a + 10
510
>>> print root.b + 10
15
>>> print root.c
None

Note that you can change the name and namespace used for this attribute through the setPytypeAttributeTag(tag) module function, in case your application ever needs to. There is also a utility function annotate() that recursively generates this attribute for the elements of a tree:

>>> root = objectify.fromstring("<root><a>test</a><b>5</b></root>")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 'test' [StringElement]
    b = 5 [IntElement]

>>> objectify.annotate(root)

>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    a = 'test' [StringElement]
      * py:pytype = 'str'
    b = 5 [IntElement]
      * py:pytype = 'int'

A second way of specifying data type information uses XML Schema types as element annotations. Objectify knows those that can be mapped to normal Python types:

>>> root = objectify.fromstring('''\
...    <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...      <d xsi:type="double">5</d>
...      <l xsi:type="long"  >5</l>
...      <s xsi:type="string">5</s>
...    </root>
...    ''')
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    d = 5.0 [FloatElement]
      * xsi:type = 'double'
    l = 5L [LongElement]
      * xsi:type = 'long'
    s = '5' [StringElement]
      * xsi:type = 'string'

The utility function deannotate() can be used to get rid of 'py:pytype' and/or 'xsi:type' information:

>>> root = objectify.fromstring('''\
... <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...   <d xsi:type="double">5</d>
...   <l xsi:type="long"  >5</l>
...   <s xsi:type="string">5</s>
... </root>''')
>>> objectify.annotate(root)
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    d = 5.0 [FloatElement]
      * xsi:type = 'double'
      * py:pytype = 'float'
    l = 5L [LongElement]
      * xsi:type = 'long'
      * py:pytype = 'long'
    s = '5' [StringElement]
      * xsi:type = 'string'
      * py:pytype = 'str'
>>> objectify.deannotate(root)
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    d = 5 [IntElement]
    l = 5 [IntElement]
    s = 5 [IntElement]

The DataElement factory

For convenience, the DataElement() factory creates an Element with a Python value in one step. You can pass the required Python type name or the XSI type name:

>>> root = objectify.Element("root")
>>> root.x = objectify.DataElement(5, _pytype="long")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    x = 5L [LongElement]
      * py:pytype = 'long'

>>> root.x = objectify.DataElement(5, _pytype="str", myattr="someval")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    x = '5' [StringElement]
      * py:pytype = 'str'
      * myattr = 'someval'

>>> root.x = objectify.DataElement(5, _xsi="integer")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    x = 5L [LongElement]
      * py:pytype = 'long'
      * xsi:type = 'integer'

There is a side effect of the type lookup. If you assign a string value using attribute assignment and that string value turns out to be valid for any of the type checks, you will end up with the resolved type instead of a StringElement:

>>> root = objectify.Element("root")
>>> root.s = "5"
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    s = 5 [IntElement]

You can use the DataElement() factory to avoid this behaviour and thus provide the type of a data element by hand:

>>> root = objectify.Element("root")
>>> root.s = objectify.DataElement(5,  _pytype="str")
>>> print objectify.dump(root)
root = None [ObjectifiedElement]
    s = '5' [StringElement]
      * py:pytype = 'str'

Defining additional data classes

You can plug additional data classes into objectify that will be used in exactly the same way as the predefined types. Data classes can either inherit from ObjectifiedDataElement directly or from one of the specialised classes like NumberElement or BoolElement. The numeric types require an initial call to the NumberElement method self._setValueParser(function) to set their type conversion function (string -> numeric Python type). This call should be placed into the element _init() method.

The registration of data classes uses the PyType class:

>>> class ChristmasDate(objectify.ObjectifiedDataElement):
...     def callSanta(self):
...         print "Ho ho ho!"

>>> def checkChristmasDate(date_string):
...     if not date_string.startswith('24.12.'):
...         raise ValueError # or TypeError

>>> xmas_type = objectify.PyType('date', checkChristmasDate, ChristmasDate)

The PyType constructor takes a string type name, an (optional) callable type check and the custom data class. If a type check is provided it must accept a string as argument and raise ValueError or TypeError if it cannot handle the string value.

PyTypes are used if an element carries a py:pytype attribute denoting its data type or, in absence of such an attribute, if the given type check callable does not raise a ValueError/TypeError exception when applied to the element text.

If you want, you can also register this type under an XML Schema type name:

>>> xmas_type.xmlSchemaTypes = ("date",)

XML Schema types will be considered if the element has an xsi:type attribute that specifies its data type. The line above binds the XSD type date to the newly defined Python type. Note that this must be done before the next step, which is to register the type. Then you can use it:

>>> xmas_type.register()

>>> root = objectify.fromstring(
...             "<root><a>24.12.2000</a><b>12.24.2000</b></root>")
>>> root.a.callSanta()
Ho ho ho!
>>> root.b.callSanta()
Traceback (most recent call last):
  ...
AttributeError: no such child: callSanta

If you need to specify dependencies between the type check functions, you can pass a sequence of type names through the before and after keyword arguments of the register() method. The PyType will then try to register itself before or after the respective types, as long as they are currently registered. Note that this only impacts the currently registered types at the time of registration. Types that are registered later on will not care about the dependencies of already registered types.

If you provide XML Schema type information, this will override the type check function defined above:

>>> root = objectify.fromstring('''\
...    <root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...      <a xsi:type="date">12.24.2000</a>
...    </root>
...    ''')
>>> print root.a
12.24.2000
>>> root.a.callSanta()
Ho ho ho!

To unregister a type, call its unregister() method:

>>> root.a.callSanta()
Ho ho ho!
>>> xmas_type.unregister()
>>> root.a.callSanta()
Traceback (most recent call last):
  ...
AttributeError: no such child: callSanta

Be aware, though, that this does not immediately apply to elements to which there already is a Python reference. Their Python class will only be changed after all references are gone and the Python object is garbage collected.