Results 1 to 2 of 2

Thread: Stopping PHP from timing out

  1. #1

    Stopping PHP from timing out

    If you are writing scripts that need to run for several minutes or even hours in PHP such as scrapers, I found an easy way to stop the server timing out.

    You probably don't want to change the default timeout of say 30 seconds because you want your other buggy scripts to time out in a reasonable time frame.

    But in your PHP code, you simply add a line like this to reset the time-out clock:

    PHP Code:
    set_time_limit(20); // reset the clock for another 20 seconds before timing out 
    And if your script is going to run for a long time, you want to have a way to check it is still running OK such as writing files to your server, or at least save a time stamp to a text file e.g.

    PHP Code:
    And in the loop of your script, to randomize it's web access timing, a simple trick is to add a random delay such as:

    PHP Code:
    usleep(rand(50000,300000)); // halt for n microseconds 
    And if you are concerned about throwing up a red flag due to your IP address accessing the same site too many times, then A. get drowned in the noise of the other traffic to that site (by not visiting too often) and B. go through a selection of proxy servers (but maybe this would be detected?).

    In case you are interested: in my case, I was compiling the destination URL's for the Click Bank redirects since these are not published in their XML database of products. So I think this is a legitimate need to probe the same site many times via a script.

    p.s. if anyone has tips on using CURL with a list of proxies with PHP scripting, it would be good to learn about it. Or maybe post links to good tutorials on the topic?

  2. #2
    It worked very well, several hours later the script ended OK.

Similar Threads

  1. [WTS] 4 articles on Stopping Smoking
    By Greylin in forum Content
    Replies: 0
    Last Post: 16 October, 2009, 11:09 AM

Tags for this Thread


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts