开发者

lxml pretty_print python memory overload

开发者 https://www.devze.com 2023-03-25 12:11 出处:网络
I have a poorly formatted xml file with over 350 MB of data. Basically, all the data was consolidated into one line. I am trying to pretty_print this into a new file to ma开发者_JAVA百科ke life easier

I have a poorly formatted xml file with over 350 MB of data. Basically, all the data was consolidated into one line. I am trying to pretty_print this into a new file to ma开发者_JAVA百科ke life easier, but am running into memory issues. Am I doing anything wrong here and is there a way around this? My computer has 4GB of RAM and is a Quad-Core i5-2410M (2.30Ghz)

import os
from lxml import etree

parser = etree.XMLParser(remove_blank_text=True)
tree = etree.parse('filename',parser)
f = open('filename',"w")
f.write(etree.tostring(tree,pretty_print=True))
f.close()


You might want to try using the write method directly with the file handle rather than calling tostring. Change this line:

f.write(etree.tostring(tree,pretty_print=True))

to this:

tree.write(f, pretty_print=True)

This should hopefully reduce the memory usage by half.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号