Menu
Community /Pin to ProfileBookmark
@NogDogApr 10.2016 — #How big a file are we talking about? Is it possible you're running out of memory due to the PHP array getting too big? Perhaps it's time to think about using a database to store this data, since what you are trying to do is something a database can do easily and [i]much[/i] more quickly?@NogDogApr 10.2016 — #40,000,000 lines? :eek: If I were doing it, I'd definitely find a way to leverage a database.
Do you know where it breaks? Do you have all error reporting turned on?
@NogDogApr 11.2016 — #This works with a small data set, so it might be interesting to see how it scales with a larger file:
@NogDogApr 12.2016 — #Well, yeah, you'll need to set up a database, db user, etc., and make sure your connection string is correct for it. I just used something I had sitting around on my PC. ?@NogDogApr 12.2016 — #Afraid it's an issue with Windows using back-slashes as path separators. From http://dev.mysql.com/doc/refman/5.1/en/load-data.html
So you'll either need to change those "" to "/" or to "
Caculate the number that repeated more than others,,,
hi
i have a problem,,look at below example
[CODE]
1 390
1 382
1 383
1 357
1 1
1 1
1 376
3 24
we need code that says the number that is repeated more than others in left side is 1 that repeated 7 time and minimum number is 3 that repeated 1 time..and in right side 1 is repeated 2 time,then its most repeated and one of other that is repeated one time (doesn’t matter which one) like 24 that repeated one time is 1 time, then its in minimum place of right numbers…
so i have code for this that exist in below
[code=php]
<?php
$time_start = microtime(true);
ini_set(‘max_execution_time’, 700);
echo “<br>” ;
ini_set(‘memory_limit’, ‘2048M’);
$sumpage1=array();
$sumpage2=array();
$avgin;
$avgout;
$t1=0;
$t2=0;
$file = fopen( “yes.txt”, “r” ) or exit ( “Unable to open file!” ) ;
$str = fgets($file);
while ( !feof ( $file ) )
{
$str = fgets($file);
$r= explode(” “, $str);
/* $r[1]=trim($r[1]);
*/
if(empty($sumpage1[$r[0]])){
$sumpage1[$r[0]]=1;
}
else {
$sumpage1[$r[0]]++;
}
if(empty($sumpage1[$r[1]])){
$sumpage2[$r[1]]=1;
echo $r[1];
echo “<br>”;
}
else {
$sumpage2[$r[1]]++;
echo “dd”.$r[1];
echo “<br>”;
}
}
echo $sumpage2[4];
echo “<center>”;
echo “<br>”;
$x1=max($sumpage1);
$x2=array_search($x1,$sumpage1);
echo “mux number in right: “.$x2.” and this number repeated “.$x1.”times”;
echo “<br>”;
$x11=min($sumpage1);
$x22=array_search($x11,$sumpage1);
echo “min number in right: : “.$x22.” and this number repeated “.$x11.”times”;
echo “<br><br><br>”;
$y1=max($sumpage2);
$y2=array_search($y1,$sumpage2);
echo “mux number in left: “.$y2.” and this number repeated”.$y1.”times”;
echo “<br>”;
$y11=min($sumpage2);
$y22=array_search($y11,$sumpage2);
echo “min number in left: “.$y22.”and this number repeated “.$y11.”times”;
echo “<br>”;
$time_end = microtime(true);
$time = $time_end – $time_start;
echo “proccess time: {$time}”;
echo “</center>”;
?>
whats your point?
this code in big files,cant calculate the minimum right,,,some thing is wrong but…
Sign in
to post a comment9 Comments(s) ↴
0
@NogDogApr 10.2016 — #How big a file are we talking about? Is it possible you're running out of memory due to the PHP array getting too big? Perhaps it's time to think about using a database to store this data, since what you are trying to do is something a database can do easily and [i]much[/i] more quickly?0
@NogDogApr 10.2016 — #40,000,000 lines? :eek: If I were doing it, I'd definitely find a way to leverage a database.Do you know where it breaks? Do you have all error reporting turned on?
[code=php]
<?php
ini_set('display_errors', true);
error_reporting(E_ALL);
[/code]0
@NogDogApr 11.2016 — #This works with a small data set, so it might be interesting to see how it scales with a larger file:[code=php]
<?php
ini_set('display_errors', 1);
error_reporting(E_ALL);
// some test data:
$data = <<<EOD
1 390
1 382
1 383
1 357
1 1
1 1
1 376
3 24
EOD;
$fileName = 'yes.txt';
file_put_contents($fileName, $data);
// just want to see the test data:
echo "<pre>".file_get_contents('yes.txt')."</pre>n";
try {
$pdo = new PDO('mysql:dbname=test;host=127.0.0.1', 'root');
$sql = "CREATE TEMPORARY TABLE temp_tbl (col_1 INT, col_2 INT)";
if($pdo->query($sql) == false) {
death('Create failed: ', $pdo->errorInfo());
}
$sql = "LOAD DATA INFILE '".dirname(__FILE__)."/$fileName' INTO TABLE temp_tbl FIELDS TERMINATED BY ' '";
if($pdo->query($sql) == false) {
death('LOAD failed:', $pdo->errorInfo());
}
$sqls = array(
"SELECT 'left_max' AS what, col_1, COUNT(*) AS ct FROM temp_tbl GROUP BY col_1 ORDER BY ct DESC LIMIT 1",
"SELECT 'left_min' AS what, col_1, COUNT(*) AS ct FROM temp_tbl GROUP BY col_1 ORDER BY ct ASC LIMIT 1",
"SELECT 'right_max' AS what, col_2, COUNT(*) AS ct FROM temp_tbl GROUP BY col_2 ORDER BY ct DESC LIMIT 1",
"SELECT 'right_min' AS what, col_2, COUNT(*) AS ct FROM temp_tbl GROUP BY col_2 ORDER BY ct ASC LIMIT 1"
);
echo "<h2>Results</h2>n<pre>";
foreach($sqls as $sql) {
if (($stmt = $pdo->query($sql)) == false) {
death('SELECT failed', $pdo->errorInfo);
}
$row = $stmt->fetch(PDO::FETCH_BOTH);
echo "{$row['what']}: {$row[1]} ({$row['ct']})n";
}
echo "</pre>n";
unlink($fileName);
} catch (PDOException $e) {
die(print_r($e,1));
}
function death($title, $info)
{
die("<pre>$title:n".print_r($info,1)."</pre>");
}
[/code]0
@NogDogApr 12.2016 — #Well, yeah, you'll need to set up a database, db user, etc., and make sure your connection string is correct for it. I just used something I had sitting around on my PC. ?0
@NogDogApr 12.2016 — #Afraid it's an issue with Windows using back-slashes as path separators. From
The file name must be given as a literal string. On Windows, specify backslashes in path names as forward slashes or doubled backslashes. As of MySQL 5.1.6, the character_set_filesystem system variable controls the interpretation of the file name.
[/quote]
So you'll either need to change those "" to "/" or to "