开发者

Does Plone have a limit on the size of data being uploaded

开发者 https://www.devze.com 2023-04-11 17:42 出处:网络
I developed a Plone site and I am using csvreplicata add-on to upload data f开发者_如何学Pythonrom csv files. I managed to upload a file of about 6000 rows, however when I try to upload a csv with abo

I developed a Plone site and I am using csvreplicata add-on to upload data f开发者_如何学Pythonrom csv files. I managed to upload a file of about 6000 rows, however when I try to upload a csv with about 120,000 rows the plone site hangs. Does anyone know if there is a limit on the size of data uploaded please?

Thanks in advance


Looking at the product code i can't see any transaction savepoint, so I think that the real issue here is that your server goes out of memory because of the huge size of the single transaction and it hangs because it starts swapping. Try to monitor your server's memory usage and if necessary add this code every N elements processed during your import:

import transaction

...
transaction.savepoint(optimistic=True)

This will make a "sub-transaction" of the transaction, moving data from memory to the hard disk.

Be sure to choose an appropriate N dimension: for very huge transactions you risk to save your memory and going out of disk space because every savepoint multiplies the data dimension on the hdd.

More info: http://docs.zope.org/zope3/Book/zodb/savepoints/show.html

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号