|
1 | | -# xml_viewer_editor |
2 | | -xml parser with editor capability |
| 1 | +# XML file processing with Python lxml Module |
| 2 | + |
| 3 | +This article were to explain about lxml general functionality and demonstrate how lxml can provide XML content parsing and reading efficiently with the aim to make programmer life easier. lxml consider as one of the most feature-rich and easy-to-use library for processing XML and HTML in the Python language. This article we are going to walk through come of the core feature lxml can provides. lxml package has a quite different way of representing documents as trees. |
| 4 | +In the DOM, trees are built out of nodes represented as Node instances. |
| 5 | +Some nodes are Element instances, representing whole elements as lists. |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | +Example XML sample file, [sample.xml](sample.xml). |
| 10 | + |
| 11 | + |
| 12 | + |
| 13 | +At its most fundamental, XML schema file needs to be [parse][1] and process. We may utilize parse function to quickly convert an XML file into an ElementTree. |
| 14 | + |
| 15 | +**General way to import lxml as etree, and assign xml file name/path as source** |
| 16 | + |
| 17 | +```python |
| 18 | +from lxml import etree |
| 19 | +tree = etree.parse('sample.xml', parser=etree.XMLParser()) |
| 20 | +``` |
| 21 | + |
| 22 | + |
| 23 | +**Top Level Element** |
| 24 | + |
| 25 | +```python |
| 26 | +print(tree.getroot()) |
| 27 | +``` |
| 28 | +Output: |
| 29 | +> <Element PurchaseOrders at 0x1fb8c81ed40> |
| 30 | +
|
| 31 | +**Element nodes for its element children.** |
| 32 | + |
| 33 | +```python |
| 34 | +print(tree.getroot().getchildren()) |
| 35 | +``` |
| 36 | +Output: |
| 37 | +> [<Element PurchaseOrder at 0x1dae421eec0>, <Element PurchaseOrder at 0x1dae421f200>, <Element PurchaseOrder at 0x1dae421f2c0>] |
| 38 | +
|
| 39 | +**Attribute nodes for its attributes.** |
| 40 | + |
| 41 | +```python |
| 42 | +for e in tree.getroot().getchildren(): |
| 43 | + print(e.attrib) |
| 44 | +``` |
| 45 | +Output: |
| 46 | +> {'PurchaseOrderNumber': '99504', 'OrderDate': '2001-10-20'} |
| 47 | +> {'PurchaseOrderNumber': '99505', 'OrderDate': '2001-10-22'} |
| 48 | +> {'PurchaseOrderNumber': '99503', 'OrderDate': '2001-10-22'} |
| 49 | +
|
| 50 | +**Text nodes for textual content.** |
| 51 | + |
| 52 | +```python |
| 53 | +for e in tree.getroot().getchildren()[0]: |
| 54 | + print(e.text) |
| 55 | +``` |
| 56 | +Output: |
| 57 | +> Please leave packages in shed by driveway. |
| 58 | +
|
| 59 | +## XML schema structure |
| 60 | + |
| 61 | +Each Element has an assortment of child nodes of various types: |
| 62 | + |
| 63 | + |
| 64 | + |
| 65 | +Supported XML [schema][2] format can refer to below link: |
| 66 | + |
| 67 | +## Serialise XML element objects as string type |
| 68 | + |
| 69 | +Serialize an element to an encoded string representation of its XML tree element. |
| 70 | + |
| 71 | +```python |
| 72 | +print(etree.tostring(tree.getroot().getchildren()[0]).decode("utf-8")) |
| 73 | +``` |
| 74 | +Output: |
| 75 | +``` |
| 76 | +<PurchaseOrder PurchaseOrderNumber="99504" OrderDate="2001-10-20"> |
| 77 | + <Address Type="Shipping"> |
| 78 | + <Name>Amy Adams</Name> |
| 79 | + <Street>123 Maple Street</Street> |
| 80 | + <City>Mill Valley</City> |
| 81 | + <State>CA</State> |
| 82 | + <Zip>10999</Zip> |
| 83 | + <Country>USA</Country> |
| 84 | + </Address> |
| 85 | + <Address Type="Billing"> |
| 86 | + <Name>Chong Wei</Name> |
| 87 | + <Street>8 Oak Avenue</Street> |
| 88 | + <City>Old Town</City> |
| 89 | + <State>PA</State> |
| 90 | + <Zip>95819</Zip> |
| 91 | + <Country>USA</Country> |
| 92 | + </Address> |
| 93 | + <DeliveryNotes>Please leave packages in shed by driveway.</DeliveryNotes> |
| 94 | + <Items> |
| 95 | + <Item PartNumber="872-AC"> |
| 96 | + <ProductName>Lawnmower</ProductName> |
| 97 | + <Quantity>1</Quantity> |
| 98 | + <USPrice>148.95</USPrice> |
| 99 | + <Comment>Confirm this is electric</Comment> |
| 100 | + </Item> |
| 101 | + <Item PartNumber="926-AD"> |
| 102 | + <ProductName>Dell Monitor</ProductName> |
| 103 | + <Quantity>2</Quantity> |
| 104 | + <USPrice>39.98</USPrice> |
| 105 | + <ShipDate>1999-05-21</ShipDate> |
| 106 | + </Item> |
| 107 | + </Items> |
| 108 | + </PurchaseOrder> |
| 109 | +``` |
| 110 | + |
| 111 | +## XML Content search |
| 112 | + |
| 113 | +lxml provides multiple function to locate ElemenTree (ET) [element path][3]. For this particular demonstration findall seem to be a good fit to locate matching keyword within which child element, and return its index number. |
| 114 | + |
| 115 | +Set search element path |
| 116 | + |
| 117 | +```python |
| 118 | +roottree = tree.getroot() |
| 119 | +subelement = roottree[0].tag # PurchaseOrder |
| 120 | +findalltree = tree.findall(subelement) |
| 121 | + |
| 122 | +print(findalltree) |
| 123 | +``` |
| 124 | + |
| 125 | +> [<Element PurchaseOrder at 0x16675a1f040>, <Element PurchaseOrder at 0x16675a1f380>, <Element PurchaseOrder at 0x16675a1f440>] |
| 126 | +
|
| 127 | +Setup search argument and enumeration expression and condition. |
| 128 | +For this particular use case, the objective were to identify interested information reside within which sub-element object. For demonstration purposes, "PartNumber" used as unique keyword to identify sub-element object index id, and sub-element objects. |
| 129 | + |
| 130 | +```python |
| 131 | +keyword = 'PartNumber="456-NF"' |
| 132 | +for h, i in enumerate(findalltree): |
| 133 | + |
| 134 | + if keyword in etree.tostring(i).decode("utf-8"): |
| 135 | + |
| 136 | + print(f'index: {h} \n{etree.tostring(i, pretty_print=True).decode("utf-8")}') |
| 137 | +``` |
| 138 | +``` |
| 139 | +index: 1 |
| 140 | +<PurchaseOrder PurchaseOrderNumber="99505" OrderDate="2001-10-22"> |
| 141 | + <Address Type="Shipping"> |
| 142 | + <Name>anna kendrick</Name> |
| 143 | + <Street>456 Main Street</Street> |
| 144 | + <City>Buffalo</City> |
| 145 | + <State>NY</State> |
| 146 | + <Zip>98112</Zip> |
| 147 | + <Country>USA</Country> |
| 148 | + </Address> |
| 149 | + <Address Type="Billing"> |
| 150 | + <Name>anna kendrick</Name> |
| 151 | + <Street>456 Main Street</Street> |
| 152 | + <City>Buffalo</City> |
| 153 | + <State>NY</State> |
| 154 | + <Zip>98112</Zip> |
| 155 | + <Country>USA</Country> |
| 156 | + </Address> |
| 157 | + <DeliveryNotes>Please notify me before shipping.</DeliveryNotes> |
| 158 | + <Items> |
| 159 | + <Item PartNumber="456-NF"> |
| 160 | + <ProductName>Power Supply</ProductName> |
| 161 | + <Quantity>1</Quantity> |
| 162 | + <USPrice>45.99</USPrice> |
| 163 | + </Item> |
| 164 | + </Items> |
| 165 | + </PurchaseOrder> |
| 166 | +``` |
| 167 | + |
| 168 | +## Element removal action: |
| 169 | + |
| 170 | +The final product obtained allow us to work on the interested sub-element as we wish. For example, we may use the index number to remove unwanted element. |
| 171 | + |
| 172 | +```python |
| 173 | +print(roottree.getchildren()) |
| 174 | + |
| 175 | +roottree.remove(findalltree[1]) |
| 176 | + |
| 177 | +print(roottree.getchildren()) |
| 178 | +``` |
| 179 | + |
| 180 | +## The Result: |
| 181 | + |
| 182 | +Available sub-elements: |
| 183 | +> [<Element PurchaseOrder at 0x23fee01f140>, <Element PurchaseOrder at 0x23fee01f000>, <Element PurchaseOrder at 0x23fee01f3c0> |
| 184 | +
|
| 185 | +Reduced sub-elements after remove sub-element index 1: |
| 186 | +> [<Element PurchaseOrder at 0x23fee01f140>, <Element PurchaseOrder at 0x23fee01f3c0>] |
| 187 | +
|
| 188 | + |
| 189 | + |
| 190 | +[1]: https://lxml.de/apidoc/lxml.etree.html#lxml.etree.parse |
| 191 | +[2]: https://www.w3schools.com/XML/schema_schema.asp |
| 192 | +[3]: https://lxml.de/tutorial.html#elementpath |
| 193 | + |
| 194 | +--- |
| 195 | + |
| 196 | +## Activate virtual environment |
| 197 | + |
| 198 | +``` |
| 199 | +python -m venv .venv |
| 200 | +
|
| 201 | +Windows: |
| 202 | +.\.venv\Scripts\activate |
| 203 | +
|
| 204 | +Linux & Unix: |
| 205 | +source .venv/bin/activate |
| 206 | +
|
| 207 | +pip install -r requirements.txt |
| 208 | +``` |
0 commit comments