/    Sign up×
Community /Pin to ProfileBookmark

Long running scripts

Copy linkTweet thisAlerts:
Apr 05.2009

I have an application that will run for a very long time (hours) without any I/O between the server and the browser. It continually fails after about 7-8 minutes of wall-clock time with apache errors of

[code]
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.
[/code]

The error_log shows:

[code]
[Sun Apr 05 13:45:06 2009] [error] [client 206.72.99.10] Premature end of script headers: tool-XXCatalogConvert.php, referer: http://test.ez-om.com/om/Tests/tool-XXCatalogConvert.php

[Sun Apr 05 13:45:06 2009] [warn] (104)Connection reset by peer: mod_fcgid: read data from fastcgi server error.
[/code]

This is NOT a php set_time_limit() issue (I reset it at the top of each loop).
I have set the following Apache directives in hopes of allowing this to run but can’t seem to find the magic one that will make it work.

[code]
Timeout 7200
KeepAlive Off
MaxKeepAliveRequests 0
KeepAliveTimeout 3600
[/code]

And within the VirtualServer block I do:

[code]
IPCCommTimeout 7200
IPCConnectTimeout 10
ProcessLifeTime 7200
IdleTimeout 7200
BusyTimeout 3600
[/code]

But still, after about 7 minutes it dies with the above errors….
This script does a lot of DB access and connects to 2 remote sites via ftp (persistent connection outside of the loop).

Anyone have the magic answer?

to post a comment
PHP

6 Comments(s) ↴

Copy linkTweet thisAlerts:
@NightShift58Apr 05.2009 — Use [b]set_time_limit(int $seconds)[/b].

Here an excerpt from the PHP manual:

Seconds: The maximum execution time, in seconds. If set to zero, no time limit is imposed.
Copy linkTweet thisAlerts:
@tbirnsethauthorApr 05.2009 — As I said, I am doing a set_time_limit(10) at the top of each loop. It is not terminating with the PHP Maximum Execution Time exceeded message. I didn't put the value for set_time_limit() in the original post because it was just mean to convey that I was in fact doing it.
Copy linkTweet thisAlerts:
@SyCoApr 07.2009 — I had a similar issue and solved it by breaking up the main scripts task into smaller ones and writing the tasks to a database. Then a script is called to check to see what tasks are not yet complete. The server can be rebooted mid job and it's doesn't cause a problem. The script just picks up where it left off.
Copy linkTweet thisAlerts:
@tbirnsethauthorApr 07.2009 — That's basically what I ended up doing (thought I just stuffed it into the SESSION) and recall the script with the counter to resume at.

But I'd sure like to know the root cause.
Copy linkTweet thisAlerts:
@NightShift58Apr 07.2009 — I am quite confident that, in the end, it may well be a time-out issue. This may be on your server or on the remote servers that you are calling or the methods that your are using to call the remote pages.

What I don't understand is why you would set a time limit of 10 seconds when your know that the procedure will run for hours nor do I understand why you keep resetting this value at the "top of each loop".

Based on your general description of the script, I would set it once and set it to 0 seconds (unlimited time).
Copy linkTweet thisAlerts:
@tbirnsethauthorApr 07.2009 — 
What I don't understand is why you would set a time limit of 10 seconds when your know that the procedure will run for hours nor do I understand why you keep resetting this value at the "top of each loop".

Based on your general description of the script, I would set it once and set it to 0 seconds (unlimited time).[/QUOTE]

Setting it to 10 seconds at the top of each loop ensures that if something does go wrong within the loop that it will exit. If I set it to zero, then no other blocking error condition would cause it to exit. 10 seconds is plenty of time (cpu time) for the loop to complete but not enough to just hang on a problem.

I'm investigating the php setting 'max_input_time' which I believe is a wall-clock timer for no i/o from the script (versus max_execution_time which is CPU time).
×

Success!

Help @tbirnseth spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 — BETA 5.17,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @AriseFacilitySolutions09,
tipped: article
amount: 1000 SATS,

tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,
)...