Quantcast
Channel: PHPExcel Forum Rss Feed
Viewing all articles
Browse latest Browse all 2707

New Post: Need help tweaking phpexcel to deal with limited memory and big xlsx file

$
0
0
I've tested iterating one row at a time. It still runs out of memory after the first chunk iteration on the larger (200k rows) xlsx file and does finish executing OK the smaller (65k rows) xls file.

What I don't get is why it process the first chunk and only then it runs out of memory. Does the memory usage increase after each chunk iteration? Shouldn't it be replacing the same amount of data in memory for each chunk?

Maybe I should be killing the last chunk data in memory somehow?

EDIT/
It certainly is that. The xls test output proves it's not filtering from where the last chunk stopped, it's loading again every row from the first to the new limit instead. For instance: the first iteration prints 1 to 2001 and the second prints 1 to 4002. I'll try moving the filter object creation into the loop and see what it does.

Code (notice I've removed the database writing part for testing):
// detect filetype
$filetype = PHPExcel_IOFactory::identify('./uploads/'.$filename);

// target sheet
$sheet_name = 'D2D SP';

// start and end rows by user input
$start_row = ($this->input->post('start_row')||$this->input->post('start_row')<2)?$this->input->post('start_row'):2;
$end_row = ($this->input->post('end_row')||$this->input->post('end_row')<=$this->input->post('start_row'))?$this->input->post('end_row'):$start_row+101;

// split sheet into chunks to save memory
$chunk_size = ($end_row - $start_row)+1 < 2000? $end_row - $start_row + 1 : 2000; 
$chunkFilter = new chunkReadFilter();

// create reader based on detected filetype
$phpExcelReader = PHPExcel_IOFactory::createReader($filetype);
$phpExcelReader->setLoadSheetsOnly($sheet_name);
$phpExcelReader->setReadDataOnly(TRUE);
$phpExcelReader->setReadFilter($chunkFilter);

for ($i=$start_row; $i <= $end_row; $i += $chunk_size+1) {

    $chunkFilter->setRows($i, $chunk_size);

    // create THE object
    $phpExcel = $phpExcelReader->load('./uploads/'.$filename);

    // get worksheet dimensions
    $sheet = $phpExcel->getActiveSheet();

    foreach ($sheet->getRowIterator() as $row) {
        echo $row->getRowIndex().'</br>';
    }

    echo '======= yet another chunk =======';

}

Viewing all articles
Browse latest Browse all 2707

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>