/    Sign up×
Community /Pin to ProfileBookmark

problem with sphider spider.php

I know i should direct this at the creator of Sphider but things are slow there, its quite simple what i need to do in theory, currently the spider can have pages excluded from the index, which it noindex, no follows, i want to to noindex, follow.

Here’s the piece of code where (i think) it all happens:

[code=php] if (isset($_SERVER[‘argv’]) && $_SERVER[‘argc’] >= 2) {
$command_line = 1;
$ac = 1; //argument counter
while ($ac < (count($_SERVER[‘argv’]))) {
$arg = $_SERVER[‘argv’][$ac];

if ($arg == ‘-all’) {
$all = 1;
break;
} else if ($arg == ‘-u’) {
$url = $_SERVER[‘argv’][$ac+1];
$ac= $ac+2;
} else if ($arg == ‘-f’) {
$soption = ‘full’;
$ac++;
} else if ($arg == ‘-d’) {
$soption = ‘level’;
$maxlevel = $_SERVER[‘argv’][$ac+1];;
$ac= $ac+2;
} else if ($arg == ‘-l’) {
$domaincb = 1;
$ac++;
} else if ($arg == ‘-r’) {
$reindex = 1;
$ac++;
} else if ($arg == ‘-m’) {
$in = str_replace(“\n”, chr(10), $_SERVER[‘argv’][$ac+1]);
$ac= $ac+2;
} else if ($arg == ‘-n’) {
$out = str_replace(“\n”, chr(10), $_SERVER[‘argv’][$ac+1]);
$ac= $ac+2;
} else {
commandline_help();
die();
}

}
}

[/code]

to post a comment
PHP

0Be the first to comment 😎

×

Success!

Help @danpoulton spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 BETA 6.16,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @nearjob,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,

tipper: @meenaratha,
tipped: article
amount: 1000 SATS,
)...