I found this project : https://github.com/blackrock/xml_to_parquet Convert XML files into Apache Parquet format using just an XSD schema and XML file. This process uses the XSD to transform all content from the XML into a corresponding Parquet file, maintaining nested data structures that replicate the XML paths. Convert a small XML file to a Parquet file python xml_to_parquet.py -x PurchaseOrder.xsd PurchaseOrder.xml INFO - 2021-01-21 12:32:38 - Parsing XML Files.. INFO - 2021-01-21 12:32:38 - Processing 1 files DEBUG - 2021-01-21 12:32:38 - Generating schema from PurchaseOrder.xsd DEBUG - 2021-01-21 12:32:38 - Parsing PurchaseOrder.xml DEBUG - 2021-01-21 12:32:38 - Saving to file PurchaseOrder.xml.parquet DEBUG - 2021-01-21 12:32:38 - Completed PurchaseOrder.xml More links : https://stackoverflow.com/questions/36289548/is-there-a-way-to-create-parquet-file-from-xml-json-input-file-without-avsc-fil https://medium.com/@sonradata/convert-xml-with-spark-to-parquet-1c5ba561b193

How to convert XML files to Parquet without losing any data. In my scenario the xml file is in complex nested format and I am unable to capture all fields?

Moka Rupesh 0

User's image

phemanth 8,645 Reputation points Microsoft Vendor

2024-04-23T14:13:01.8733333+00:00

@Moka Rupesh Following up to see if the above below was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Amira Bedhiafi 20,176 Reputation points

2024-04-22T14:36:43.83+00:00
I found this project : https://github.com/blackrock/xml_to_parquet

Convert XML files into Apache Parquet format using just an XSD schema and XML file. This process uses the XSD to transform all content from the XML into a corresponding Parquet file, maintaining nested data structures that replicate the XML paths.

Convert a small XML file to a Parquet file python xml_to_parquet.py -x PurchaseOrder.xsd PurchaseOrder.xml INFO - 2021-01-21 12:32:38 - Parsing XML Files.. INFO - 2021-01-21 12:32:38 - Processing 1 files DEBUG - 2021-01-21 12:32:38 - Generating schema from PurchaseOrder.xsd DEBUG - 2021-01-21 12:32:38 - Parsing PurchaseOrder.xml DEBUG - 2021-01-21 12:32:38 - Saving to file PurchaseOrder.xml.parquet DEBUG - 2021-01-21 12:32:38 - Completed PurchaseOrder.xml

More links :

https://stackoverflow.com/questions/36289548/is-there-a-way-to-create-parquet-file-from-xml-json-input-file-without-avsc-fil

https://medium.com/@sonradata/convert-xml-with-spark-to-parquet-1c5ba561b193
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Share via

How to convert XML files to Parquet without losing any data. In my scenario the xml file is in complex nested format and I am unable to capture all fields?

1 answer