Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 38 ยท Projects and Practice

Project: Analyse a Burp Proxy History Export with Python

Build a Python tool that reads a Burp-style XML export of proxy history, decodes Base64 requests and responses, summarises endpoints and flags missing security headers and scope leaks.

Burp can save selected requests and responses to a file (right-click items, Save items), and many tools and extensions export the same format. The file is XML, and the raw HTTP messages inside it are Base64-encoded. Once you can read that format with a script, you can do things the GUI makes tedious: count endpoints across thousands of requests, search for headers, produce a list of parameters for later testing, or check that nothing out of scope slipped into your history.

In this project you build that analyser. To keep it runnable on this page, the sample export is generated inside the script from a few fake HTTP messages for a lab host (lab.example.test), so no real traffic is involved. On your own machine you would read the same structure from a file you exported from your lab project.

The export format

A simplified Burp "Save items" XML file looks like this (labelled example):

XML
<?xml version="1.0"?>
<items burpVersion="2025.x" exportTime="...">
  <item>
    <time>Mon Nov 10 10:41:07 UTC 2026</time>
    <url><![CDATA[http://lab.example.test/shop?category=books]]></url>
    <host ip="127.0.0.1">lab.example.test</host>
    <port>80</port>
    <protocol>http</protocol>
    <method><![CDATA[GET]]></method>
    <path><![CDATA[/shop?category=books]]></path>
    <status>200</status>
    <responselength>5120</responselength>
    <mimetype>HTML</mimetype>
    <request base64="true"><![CDATA[R0VUIC9zaG9w...]]></request>
    <response base64="true"><![CDATA[SFRUUC8xLjEg...]]></response>
  </item>
</items>

Each <item> has metadata fields and the full request and response as Base64 text, flagged with base64="true". Extra fields exist, but these are enough for the project.

Step 1: build a sample export and read it

First write the code that creates sample items and parses them. Parsing does three things: read the XML, decode Base64, then split each HTTP message into a header block and a body.

Python Step 1: parse items and decode messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that xml.etree hands back the Base64 text, and base64.b64decode restores the exact HTTP message. Everything else is string work.

Step 2: summarise endpoints and parameters

From the records you can list unique endpoints (path without query), count methods and status codes, and collect query parameter names. A parameter inventory is a handy starting list for insertion points.

Python Step 2: endpoint and parameter inventory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 3: flag missing headers and scope leaks

Two simple audits make your script worth its time:

  1. Security headers: which responses lack X-Content-Type-Options, Content-Security-Policy, and for HTTPS Strict-Transport-Security?
  2. Scope leaks: does the history contain any request to a host that is not in your scope list? (Refer back to the scope checker.)
Python Step 3: audit headers and scope

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The scope audit finds ads.tracker.test: a background request from a page, not a test target. In a real project you would exclude it from history and make sure you never sent anything to it beyond the browser's own traffic.

Step 4: take it further

Extend the tool on your own machine with the real export from your lab:

  • Read the file with ET.parse("history.xml") instead of the built-in string.
  • Write findings to a CSV for the report (report assembly).
  • Add a check for cookies set without Secure on HTTPS.
  • List responses whose Server or X-Powered-By headers expose versions.
  • Detect JSON responses containing keys like password or token (the start of a data-exposure check).
  • Compare two exports (before and after a fix) to produce a re-test diff.

Exports hold real traffic. Keep them in the engagement folder, redact before sharing, and delete them on schedule. Never upload a client export to an online tool you do not control.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How are HTTP requests and responses stored in a Burp XML export?

    Items carry base64=true elements that you decode to get the raw HTTP message.
  2. 2. Which Python modules did the project use to read the export?

    ElementTree parses the XML and base64 decodes the message text.
  3. 3. Why scan history for hosts outside the scope list?

    A scope leak check helps ensure nothing out of scope was tested.
  4. 4. What is a good reason to keep exports encrypted and delete them later?

    Exports are sensitive evidence and must be protected.

Summary

  • Burp's XML export holds metadata plus Base64-encoded requests and responses.
  • Parse with ElementTree, decode with base64, then split headers from bodies.
  • Build inventories of endpoints and parameters, and audit security headers, cookie flags and scope leaks.
  • Extend the tool to CSV output, diffs for re-tests and data-exposure checks.
  • The sample export is generated in code with fake data; handle real exports as sensitive material.

Next: Practice questions and cheat sheet