开发者

Can I use Watir to scrape data from a website on a linux server without monitor?

开发者 https://www.devze.com 2023-01-18 22:46 出处:网络
Can I use Watir to sc开发者_StackOverflowrape data from a website (AJAX used) but on a linux server without monitor? (linode.com) ?There are several ways to do this:

Can I use Watir to sc开发者_StackOverflowrape data from a website (AJAX used) but on a linux server without monitor? (linode.com) ?


There are several ways to do this:

  1. Use HtmlUnit, either Celerity or watir-webdriver (through the remote Selenium2/WebDriver server).

  2. Use a real browser + a virtual X server (Xvfb). I'd recommend using watir-webdriver's Firefox driver and the Headless gem for a simple way to control this from Ruby.

This is basically a tradeoff between speed and realism. Personally I'd go with #2 if the site has any complex JavaScript or invalid HTML, but both approaches could be worth investigation.

For the future, I'm keeping an eye on this project, which looks like a terrific idea.

0

精彩评论

暂无评论...
验证码 换一张
取 消