Skip to content
Tutorials Center
My learning

Topics

Lesson 57 of 149 ยท Modules and the Standard Library

XML with ElementTree

Read and write XML in Python with ElementTree. Parse documents, find elements with XPath, handle namespaces, build and save XML, and use defusedxml.

XML (Extensible Markup Language) is an older cousin of JSON for structured data. It looks like HTML: tags in angle brackets, nested inside each other, with attributes. JSON has taken over most web APIs, but XML is still everywhere: RSS and Atom news feeds, sitemaps, SVG images, Office documents (a .docx file is a zip of XML files), Android layouts, bank and government data exchanges, and many configuration files.

Python's standard library reads and writes XML with the xml.etree.ElementTree module, usually imported as ET. In this lesson you'll parse XML text and files, navigate and search the tree, use ElementTree's XPath subset, deal with namespaces, change and build documents, and learn the one security rule that matters for XML from strangers.

The shape of an XML document

An XML document is a tree of elements. Each element has a tag (its name), optional attributes (name/value pairs), optional text, and child elements. There's exactly one root element that contains everything else:

XML
<?xml version="1.0" encoding="UTF-8"?>
<library city="Lisbon">
  <book id="b1" lang="en">
    <title>The Salt Road</title>
    <year>2019</year>
  </book>
  <book id="b2" lang="pt">
    <title>Mar Aberto</title>
    <year>2023</year>
  </book>
</library>

Unlike HTML, XML is strict: every tag must be closed, tags are case-sensitive, attribute values need quotes, and elements must nest properly. A document that breaks these rules isn't "almost XML"; parsers reject it.

Parsing XML

ET.fromstring() parses a string and returns the root element. For files, ET.parse() returns an ElementTree object, and .getroot() gives you the root:

Python Parsing and exploring elements

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The key parts of an Element:

Attribute or method What it gives you
el.tag the tag name, such as "book"
el.attrib a dictionary of attributes
el.get(name, default) one attribute, or the default
el.text the text right after the opening tag (or None)
el.tail the text after the closing tag, before the next sibling
list(el), el[0], len(el) the child elements

Everything XML gives you is text. Convert numbers and dates yourself, as with int(book.find("year").text) above.

To read a file instead, use ET.parse("library.xml").getroot(). The parser reads the encoding from the <?xml ...?> declaration, so you can pass a file name or a file opened in binary mode.

Finding elements

Looping over direct children only gets you one level. ElementTree has four search methods:

  • find(path) returns the first matching element, or None.
  • findall(path) returns a list of all matches.
  • findtext(path, default) returns the text of the first match, or the default.
  • iter(tag) walks the whole subtree, at any depth.
Python find, findall, findtext and iter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The most common beginner mistake is the second-to-last line: find() and findall() look at direct children unless the path says otherwise. Use a path like "department/product", the .// prefix described next, or iter().

The XPath subset

The path you pass to find() and findall() is a small subset of XPath, the standard query language for XML:

Path Matches
tag child elements named tag
* all child elements
. the current element
.//tag tag elements at any depth below
.. the parent (only within a path)
tag[@attr] tag elements that have the attribute
tag[@attr='value'] ...whose attribute equals the value
tag[child] ...that have a child element
tag[child='text'] ...whose child has exactly that text
tag[2], tag[last()] by position (counting from 1)
Python XPath queries

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The subset has limits: there are no comparisons like @stock > 5, no functions such as contains(), and .. can't climb above the element you started from. Filter those cases in Python, which is usually clearer anyway:

Python
low_stock = [p for p in root.iter("product") if int(p.get("stock")) < 3]

If you need full XPath 1.0, the third-party lxml library offers it along with a faster parser, using almost the same API.

Namespaces

Many real-world formats put their tags in a namespace, declared with xmlns, to avoid name clashes between vocabularies. ElementTree stores namespaced tags in "Clark notation": the namespace URI in braces, then the name. Searching with plain tag names then finds nothing, until you pass a prefix map:

Python Searching a namespaced feed

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The prefix in your dictionary (atom) is your own choice; only the URI has to match the document. Python 3.8 and later also accept {*}entry to match entry in any namespace.

Changing a document

Elements are mutable. You can change text and attributes, add children with ET.SubElement() or append(), and delete them with remove():

Python Editing XML

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

findall() returns a list, so removing elements while looping over it is safe. Don't remove children while iterating directly over root, for the same reason you shouldn't change a list while looping over it.

Building and writing XML

To create a document from your own data, start with ET.Element() for the root and add children with ET.SubElement(). Set text through .text, attributes as keyword arguments or with .set(). Then wrap the root in an ElementTree and write() it:

Python Building a sitemap and saving it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the third URL: the & became &amp; automatically. ElementTree escapes special characters (<, >, & and quotes in attributes) for you. Never build XML by gluing strings together with f-strings: one stray & or < in your data produces a broken document, and data from users could inject tags of its own.

A few output details:

  • ET.tostring(el, encoding="unicode") returns a str; without encoding you get UTF-8 bytes.
  • write(..., xml_declaration=True) adds the <?xml version=...?> line; always pass encoding="utf-8" when writing files.
  • ET.indent() adds whitespace for humans. Machines don't need it, so skip it for large files.

Large files with iterparse

ET.parse() loads the whole document into memory. For huge files (hundreds of megabytes of log or export data), ET.iterparse() hands you elements as they finish parsing, and you can clear each one after use so memory stays flat:

Python Streaming with iterparse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Security: XML from untrusted sources

XML has features that can be abused: a tiny document can define entities that expand into gigabytes of text (the "billion laughs" attack), or reference outside files. Python's ElementTree doesn't fetch external files, and modern versions of its underlying parser (expat) limit runaway entity expansion, but the Python documentation still recommends extra caution with data from strangers.

For XML from users, uploads or the internet, use the third-party defusedxml package. It's a drop-in replacement (import defusedxml.ElementTree as ET) that refuses the dangerous features. For XML your own program created, the standard library is fine.

XML or JSON?

For new data you design yourself, JSON is simpler: it maps directly onto dictionaries and lists, and there's no text-vs-attribute question. Use XML when a format, partner or tool requires it, or for documents that mix text and markup. Either way, the pattern is the same: parse at the edge of your program, convert to plain Python values, and work with those.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. root is <shop><aisle><item/></aisle></shop>. What does root.find("item") return?

    Plain paths match direct children. Use root.find(".//item") or root.iter("item") to search at any depth.
  2. 2. How do you read the id attribute of an element el, getting None if it's missing?

    el.get() returns the attribute value or a default; el.attrib is a dictionary of all attributes.
  3. 3. Which path finds product elements anywhere below the current element that have a sale child?

    [sale] tests for a child element named sale, while [@sale] would test for an attribute.
  4. 4. Why shouldn't you build XML by joining strings with f-strings?

    ElementTree escapes special characters automatically when it writes. Hand-built strings make that your job, and mistakes create broken or injectable XML.

Summary

  • Import the module as import xml.etree.ElementTree as ET; ET.fromstring() parses text and ET.parse(file).getroot() parses files.
  • An element has .tag, .attrib, .get(), .text, .tail and child elements you can loop over and index; all values are strings.
  • find(), findall() and findtext() search direct children unless the path says otherwise; .//tag and iter() search at any depth.
  • The XPath subset supports *, .//, .., [@attr], [@attr='v'], [child], [child='text'] and positions.
  • Namespaced tags need a prefix map such as {"atom": "http://..."} (or {*}tag).
  • Build with Element/SubElement, tidy with ET.indent(), and save with write(file, encoding="utf-8", xml_declaration=True); ElementTree escapes special characters for you.
  • iterparse() streams large files; use defusedxml for untrusted XML.

Next: Regular Expressions, where you'll search and transform text with patterns.