My code can download any file from a given url. The problem is that i have to download search results that are in .gz file format which is being continuously updated.
my code is downloading incomplete file as the开发者_如何学Python server's search is going on. I have already tried polling, ie i downloaded the file twice and matched the size etc etc.
Is there any other method that can be tried?
It seems you are not properly process response headers - if server returns 'chunked' response, you should load it by parts. To ensure it, user firefox/firebug and check response headers for test request.
See also http://en.wikipedia.org/wiki/Chunked_transfer_encoding
Also check response for other headers like Content-Length/Keep-Alive.
精彩评论