开发者

How to insert a million records into a mysql database?

开发者 https://www.devze.com 2023-01-22 13:23 出处:网络
I have a two tables. TABLE 1 has id(PRIMARY)|RANK|WEBSITE |ADDR开发者_运维问答ESS This is filled with a million websites.

I have a two tables.

TABLE 1 has id(PRIMARY)|RANK|WEBSITE |ADDR开发者_运维问答ESS
This is filled with a million websites.

TABLE 2 has id(PRIMARY)|tag1|tag2|tag3 ......|tag30

I am generating the tags by scraping for the meta tags and other attributes.

I have no problems populating the database for the first few thousand websites. After that, I am not able to populate it. I guess it timed out.

How should I proceed to populate the entire database?

EDIT:

I am getting the data by scraping the urls of the websites dynamically


It would be good if you specify error message in your question, output of mysql_error etc (if any).

If your script execution is interrupted because of PHP timeout, you may try to set corresponding configuration parameters (see here for php.ini configuration for details, and here for configuration in runtime).

If reconfiguration is not possible, you may store last record ID you updated in file / database table, and continue execution starting from next ID next time you run your PHP script.

0

精彩评论

暂无评论...
验证码 换一张
取 消