开发者

curl can not get complete response from a url

开发者 https://www.devze.com 2023-04-07 12:30 出处:网络
i use curl in centos 5.5 and curl like below to get the response from the url: curl 7.21.2 (x86_64-unknown-linux-gnu) libcurl/7.21.2 OpenSSL/0.9.8b zlib/1.2.3libidn/0.6.5

i use curl in centos 5.5 and curl like below to get the response from the url:

curl 7.21.2 (x86_64-unknown-linux-gnu) libcurl/7.21.2 OpenSSL/0.9.8b zlib/1.2.3     libidn/0.6.5
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smtp smtps  telnet tftp 
Features: IDN IPv6 Largefile NTLM SSL libz

http://www.51buy.com/portal.html, and i can get the response but it's not complete, for example you can not find '934', but '934' is really on that html source code.

on the other hand on my windows, i used the curl version like below

curl 7.21.2 (i386-pc-win32) libcurl/7.21.2 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.18 libssh2/1.2.7 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp  scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN Largefile NTLM SSL SSPI libz

and to get the same开发者_JAVA技巧 url, it can get the complete response.

can anyone give me some clue to help me to resolve the issue? Thank you in advance.


Most often when you get different responses, it is because the site tries to do different things depending on your user-agent or similar.

To counterfeit that, figuring out the exact request your browser sends and then trying to mimic that as closely as possible is often a winning approach.

In this case, since the windows version of curl seems to work, you could try to set the user-agent in the linux case to look like the windows curl version...

As you didn't include any specific protocol details, we can't guess any further about the actual contents.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号