Array Memory limitation

@tmr0012Mar 01.2012

Hi there,

I tried to load xml file to my php web page, and there is a issue that when characters inside my xml tag is greater than 999994 the ~~[B]~~array [/B]will fail to load it, and if the tag has contain less or equal to 999994 and vice versa.

I use array to store data from xml.

So, does anyone know the limitation of array? o how can I extend it?

to post a comment

PHP

9 Comments(s) _↴

@NogDogMar 02.2012 — #I don't think the limitation is specifically the array, but the amount of total memory allocated to your script under your server configuration (see the PHP [url=http://www.php.net/manual/en/ini.core.php#ini.memory-limit]memory_limit[/url] setting). That being said, PHP arrays are somewhat notorious for being a bit memory-hungry. Depending on what you are trying to do, maybe dumping everything into an array is not the optimal solution?

@tmr0012authorMar 02.2012 — #thx for reply NogDog,

I try to change the php.ini but it doesnt work, I also try upload it on live server, I found the boundary number is narrow down to 99994 on live server.

Could that be my computer is 64 bit, and live server is 32 bit. So, memory contain different.

@CharlesMar 02.2012 — #I'm with NogDog. Try to find a way to avoid dumping all the data into an array. Are you sure that you can't step through the XML with XMLParser plucking out just what you really need?

@tmr0012authorMar 02.2012 — #I agree with you guys, but I have no idea how to separate data into different array, or limit it when it load a certain amount of data.

I was using "preg_match_all" to load xml files, do you guys know how to load just a certain amount?

@CharlesMar 02.2012 — #Good Lord, that's not the way to do things. Give us an example of the XML and describe what it is that you are looking for.

@tmr0012authorMar 03.2012 — #the xml file is generate from some other system which include

<article><header>ABC </header><body>abcdef...</body></article>

<article><header>XMY </header><body>abcdf.....</body></article>

<article><header>123 </header><body>abcdf....</body></article>

since the xml style is fixed, what I can do is read article's header and body and store it to array, and post it to web. However, I have no control of how big the article will be, which the article content will be place in body section and when the body is over than certain amount is will not store data in array.

but I do find the way to reset the data limit, but my question is can you count the body length before store it to array?

@NogDogMar 03.2012 — #Outside of finding a more efficient way to get the desired data than XML, I might be inclined to load it into a database, such as via MySQL's LOAD XML INFILE. Simply create a table that includes a header column and a body column (using a TEXT type large enough to hold the largest expected article).

Then you could extract the desired article from the DB as needed, rather than loading everything into one run-time array or other data structure.

@CharlesMar 03.2012 — #This could be tightened up quite a bit. Matching character data is a bit more tricky than matching attribute values. Give this a try:

[code=php]<?php
 
 date_default_timezone_set ('America/New_York');
 
 $file = 'test.xml';
 $article_name = 'foe';
 $current_element = '';
 
 $parser = xml_parser_create();
 xml_parser_set_option ($parser, XML_OPTION_CASE_FOLDING, false);
 xml_set_element_handler($parser, 'element_names', false);
 xml_set_character_data_handler ($parser, 'find_article');
 if (!($fp = fopen($file, "r"))) $parser('could not open XML input');
 while ($data = fread($fp, 4096)) xml_parse($parser, $data, feof($fp));
 xml_parser_free($parser);
 
 function element_names ($parser, $name, $attrs) {
 global $current_element;
 $current_element = $name;
 }
 
 function find_article ($parser, $data) {
 global $current_element, $article_name;
 if ($current_element == 'header' && $data == $article_name) {
 xml_set_element_handler($parser, 'start_output', 'end_output');
 xml_set_character_data_handler ($parser, 'data_output'); 
 }
 }
 
 function data_output ($parser, $data) {
 echo $data;
 }
 
 function end_output ($parser, $name) {
 if ($name != 'header') echo "</$name>";
 if ($name == 'body') {
 xml_set_element_handler($parser, false, false);
 xml_set_character_data_handler ($parser, false); 
 }
 }
 
 function start_output ($parser, $name, $attrs) {
 echo "<";
 echo $name;
 foreach ($attrs as $key => $value) echo " $key="$value"";
 echo ">";
 }
 
 ?>[/code]

@CharlesMar 03.2012 — #Here's a tightened up version:

[code=php]<?php
 
 date_default_timezone_set ('America/New_York');
 
 $file = 'test.xml';
 $article_name = 'foe';
 $current_element = '';
 $name_cache = '';
 $done_parsing = false;
 
 $parser = xml_parser_create();
 xml_parser_set_option ($parser, XML_OPTION_CASE_FOLDING, false);
 xml_set_element_handler($parser, 'element_names', 'end_element');
 xml_set_character_data_handler ($parser, 'find_article');
 if (!($fp = fopen($file, "r"))) $parser('could not open XML input');
 while (!$done_parsing && ($data = fread($fp, 4096))) xml_parse($parser, $data, feof($fp));
 xml_parser_free($parser);
 
 function element_names ($parser, $name, $attrs) {
 global $current_element;
 $current_element = $name;
 }
 
 function find_article ($parser, $data) {
 global $current_element, $name_cache;
 if ($current_element == 'header') $name_cache .= $data;
 }
 
 function end_element ($parser, $name) {
 global $current_element, $article_name, $name_cache;
 if ($current_element == 'header') {
 if ($article_name == trim ($name_cache)) {
 xml_set_element_handler($parser, 'start_output', 'end_output');
 xml_set_character_data_handler ($parser, 'data_output'); 
 }
 $name_cache = '';
 }
 
 }
 
 function data_output ($parser, $data) {
 echo $data;
 }
 
 function end_output ($parser, $name) {
 if ($name != 'header') echo "</$name>";
 if ($name == 'body') {
 xml_set_element_handler($parser, false, false);
 xml_set_character_data_handler ($parser, false); 
 $done_parsing = true;
 }
 }
 
 function start_output ($parser, $name, $attrs) {
 echo "<";
 echo $name;
 foreach ($attrs as $key => $value) echo " $key="$value"";
 echo ">";
 }
 
 ?>[/code]

Also in #PHP _↴

Have one website index redirect to another No output after submit detect an odd or even number function

Success!

Help @tmr0012 spread the word by sharing this article on Twitter...

Tweet This

Array Memory limitation

9 Comments(s) _↴

Also in #PHP _↴

Success!

Social

Version

Array Memory limitation

9 Comments(s) ↴

Also in #PHP ↴

Success!

The web is an endless sea of information. Don't miss the boat... Subscribe!

Social

Version

9 Comments(s) _↴

Also in #PHP _↴