Programming Forums > Web Development and Design > PHP

grimpirate · Jul 22nd, 2008, 1:56 AM

I currently use this function to retrieve data from a file

 PHP Syntax (Toggle Plain Text) 
function file_retrieve_contents($filename, $offset = 0, $bytes = null){
	$handle = false;
	if(false === $handle = @fopen($filename, "rb")) return false;
 
	$filesize = '0';
	if(false === $filesize = filesize($filename)) return false;
	$filesize = sprintf('%u', $filesize);
 
	$contents = '';
 
	for($i = 0; $i < intval(bcdiv($filesize, '8192', 0)); $i++){
		$data = false;
		if(false === $data = @fread($handle, 8192)) return false;
		$contents .= $data;
	}
 
	if(false === $data = @fread($handle, intval(bcmod($filesize, '8192')))) return false;
	$contents .= $data;
 
	if(!fclose($handle)) return false;
 
	if(isset($bytes))
		return substr($contents, $offset, $bytes);
	else
		return substr($contents, $offset);
}
function file_retrieve_contents($filename, $offset = 0, $bytes = null){
	$handle = false;
	if(false === $handle = @fopen($filename, "rb")) return false;

	$filesize = '0';
	if(false === $filesize = filesize($filename)) return false;
	$filesize = sprintf('%u', $filesize);
	
	$contents = '';
	
	for($i = 0; $i < intval(bcdiv($filesize, '8192', 0)); $i++){
		$data = false;
		if(false === $data = @fread($handle, 8192)) return false;
		$contents .= $data;
	}
	
	if(false === $data = @fread($handle, intval(bcmod($filesize, '8192')))) return false;
	$contents .= $data;
	
	if(!fclose($handle)) return false;

	if(isset($bytes))
		return substr($contents, $offset, $bytes);
	else
		return substr($contents, $offset);
}

I realize the use of bcmath slows me down, but if a file happens to be greater than the largest integer size that PHP can handle then bcmath would be able to handle it (or at least that's what I hoped, which is why I coded it this way). However, if anyone can suggest a way to improve the performance of this function I'd be thankful for the criticism.

The Dark · Jul 22nd, 2008, 8:42 AM

I wouldn't be too concerned with the calculation speed, as most of the time will be spent doing the read. You could move the bcdiv calculation out of the loop, so it is only calculated once.
Having said that, if you have a size bigger than the largest integer php can hold, you probably don't want to be loading it into memory.

Another optimisation you might want to look at is to only read the part of the file that you a want to return. If you only want to return $bytes bytes, then there is no point reading the whole file into $content and then throwing the rest away. Similarly, you can use $offset to specify where you start reading the file (using fseek).

Sane · Jul 22nd, 2008, 10:02 AM

Indeed, you don't want to be reading in a file that's 2 billion bytes.

A) It will probably take PHP at least a minute to read a file that size.
B) If you're storing it in memory (even temporarily), I don't know of any common webserver which will give you an additional 2 GB of RAM.

If your files are unfortunately 2 GB, and you only need to return portions of it, then use fseek like The Dark said.

grimpirate · Jul 22nd, 2008, 9:53 PM

Yes originally the function used fseek, but I gathered that would fail if the integer portion was too high to be held. Also, regarding the loop, someone on an IRC php channel suggested that. However, I was under the impression that initial value and test condition of a for loop are only computed once, not at each cycle. For instance, if you were to traverse an array and unset values in it the array size is not recomputed as you unset values, it remains as what the original size of the array was. I actually opted to use file_get_contents instead. Originally I had moved away from this function because I seemed to recall it creating some sort of read error (but that may have had something to do with how I was interpreting what was ACTUALLY occurring). The PHP manual actually states that the function is binary-safe and for some reason I thought it performed the read as text.

Jul 22nd, 2008, 1:56 AM	#1
grimpirate King of Portal Join Date: Sep 2005 Posts: 431 Rep Power: 4	Could use some suggestions for improvement I currently use this function to retrieve data from a file PHP Syntax (Toggle Plain Text) function file_retrieve_contents($filename, $offset = 0, $bytes = null){ $handle = false; if(false === $handle = @fopen($filename, "rb")) return false; $filesize = '0'; if(false === $filesize = filesize($filename)) return false; $filesize = sprintf('%u', $filesize); $contents = ''; for($i = 0; $i < intval(bcdiv($filesize, '8192', 0)); $i++){ $data = false; if(false === $data = @fread($handle, 8192)) return false; $contents .= $data; } if(false === $data = @fread($handle, intval(bcmod($filesize, '8192')))) return false; $contents .= $data; if(!fclose($handle)) return false; if(isset($bytes)) return substr($contents, $offset, $bytes); else return substr($contents, $offset); } function file_retrieve_contents($filename, $offset = 0, $bytes = null){ $handle = false; if(false === $handle = @fopen($filename, "rb")) return false; $filesize = '0'; if(false === $filesize = filesize($filename)) return false; $filesize = sprintf('%u', $filesize); $contents = ''; for($i = 0; $i < intval(bcdiv($filesize, '8192', 0)); $i++){ $data = false; if(false === $data = @fread($handle, 8192)) return false; $contents .= $data; } if(false === $data = @fread($handle, intval(bcmod($filesize, '8192')))) return false; $contents .= $data; if(!fclose($handle)) return false; if(isset($bytes)) return substr($contents, $offset, $bytes); else return substr($contents, $offset); } I realize the use of bcmath slows me down, but if a file happens to be greater than the largest integer size that PHP can handle then bcmath would be able to handle it (or at least that's what I hoped, which is why I coded it this way). However, if anyone can suggest a way to improve the performance of this function I'd be thankful for the criticism. __________________ Lo, there do I see my father. 'Lo, there do I see My mother, and my sisters, and my brothers. 'Lo, there do I see The line of my people... Back to the beginning. 'Lo, they do call to me. They bid me take my place among them. In the halls of Valhalla... Where the brave... May live... ...forever.. GrimBB \| Mimesis

Jul 22nd, 2008, 8:42 AM	#2
The Dark Expert Programmer Join Date: Jun 2005 Posts: 852 Rep Power: 4	Re: Could use some suggestions for improvement I wouldn't be too concerned with the calculation speed, as most of the time will be spent doing the read. You could move the bcdiv calculation out of the loop, so it is only calculated once. Having said that, if you have a size bigger than the largest integer php can hold, you probably don't want to be loading it into memory. Another optimisation you might want to look at is to only read the part of the file that you a want to return. If you only want to return $bytes bytes, then there is no point reading the whole file into $content and then throwing the rest away. Similarly, you can use $offset to specify where you start reading the file (using fseek).

Jul 22nd, 2008, 10:02 AM	#3
Sane Programming Guru Join Date: Apr 2005 Location: Waterloo, Ontario Posts: 1,885 Rep Power: 5	Re: Could use some suggestions for improvement Indeed, you don't want to be reading in a file that's 2 billion bytes. A) It will probably take PHP at least a minute to read a file that size. B) If you're storing it in memory (even temporarily), I don't know of any common webserver which will give you an additional 2 GB of RAM. If your files are unfortunately 2 GB, and you only need to return portions of it, then use fseek like The Dark said. __________________ Waterloo's Canadian Computing Competition (CCC) - Stage 2 Problems, Solutions, and Test Data

Jul 22nd, 2008, 9:53 PM	#4
grimpirate King of Portal Join Date: Sep 2005 Posts: 431 Rep Power: 4	Re: Could use some suggestions for improvement Yes originally the function used fseek, but I gathered that would fail if the integer portion was too high to be held. Also, regarding the loop, someone on an IRC php channel suggested that. However, I was under the impression that initial value and test condition of a for loop are only computed once, not at each cycle. For instance, if you were to traverse an array and unset values in it the array size is not recomputed as you unset values, it remains as what the original size of the array was. I actually opted to use file_get_contents instead. Originally I had moved away from this function because I seemed to recall it creating some sort of read error (but that may have had something to do with how I was interpreting what was ACTUALLY occurring). The PHP manual actually states that the function is binary-safe and for some reason I thought it performed the read as text. __________________ Lo, there do I see my father. 'Lo, there do I see My mother, and my sisters, and my brothers. 'Lo, there do I see The line of my people... Back to the beginning. 'Lo, they do call to me. They bid me take my place among them. In the halls of Valhalla... Where the brave... May live... ...forever.. GrimBB \| Mimesis

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Building up a good portfolio - Suggestions?	kruptof	Coder's Corner Lounge	3	Jun 9th, 2007 8:39 AM
Not encoding nor zip - suggestions	markbadger	Other Programming Languages	2	Dec 29th, 2005 8:41 AM
Suggestions for language to use?	JMunaretto	Other Programming Languages	14	Oct 28th, 2005 8:21 PM
New Project- need suggestions or ideas!	zeiofen	C++	27	May 10th, 2005 10:13 AM
Code suggestions	Mad_guy	Perl	0	Feb 18th, 2005 10:31 PM

Programming Forums > Web Development and Design > PHP

Could use some suggestions for improvement