Programming Forums > Script Programming > Perl

mbd · Dec 21st, 2007, 7:03 PM

 perl Syntax (Toggle Plain Text) 
#!/usr/bin/perl
 
@log = <STDIN>;
$log = join '', @log;
while ($log =~ m/(\d+\.\d+\.\d+\.\d+\s)/g)
{
	$ip = $1;
	chomp $ip;
	$count = ($log =~ s/$1/$1/g);
	print "$ip $count\n";
}
#!/usr/bin/perl

@log = <STDIN>;
$log = join '', @log;
while ($log =~ m/(\d+\.\d+\.\d+\.\d+\s)/g)
{
	$ip = $1;
	chomp $ip;
	$count = ($log =~ s/$1/$1/g);
	print "$ip $count\n";
}

(Toggle Plain Text)

$ time cat ipaddr.txt | perl count.pl > report.txt

real	4m9.138s
user	3m7.281s
sys	0m58.301s

if i were going to run this more than once, i would probably change it to your method like this

 perl Syntax (Toggle Plain Text) 
#!/usr/bin/perl
 
@log = <STDIN>;
$log = join '', @log;
while ($log =~ m/(\d+\.\d+\.\d+\.\d+\s)/g)
{
	$ip = $1;
	chomp $ip;
	++ $count{$ip};
}
foreach $ip (keys %count)
{
	print "$ip $count{$ip}\n";	
}
#!/usr/bin/perl

@log = <STDIN>;
$log = join '', @log;
while ($log =~ m/(\d+\.\d+\.\d+\.\d+\s)/g)
{
	$ip = $1;
	chomp $ip;
	++ $count{$ip};
}
foreach $ip (keys %count)
{
	print "$ip $count{$ip}\n";	
}

(Toggle Plain Text)

$ time cat ipaddr.txt | perl hashcount.pl > report.txt

real	0m0.501s
user	0m0.433s
sys	0m0.052s

happy holidays all

Sane · Dec 22nd, 2007, 9:48 AM

Hah, I can't believe Perl can actually do that in a matter of minutes. But it's still pretty clear that the associative array takes the cake with half a second.

Thanks for that demo. Cheers.

lynxus · Dec 22nd, 2007, 10:42 AM

Hey guy's,

Glad your all debating how to do this..

I have NO idea what that code does so could anyone clarify what i need to do to achieve my goal hehe.

Thanks again.
Graham

Sane · Dec 22nd, 2007, 11:26 AM

mbd's posted code snippet outputs how many instances, of each IP address, are in the log file that Jessehk posted. I'd recommend using his second snippet (which is the psuedocode that I suggested in my first post), since it takes less than half a second to execute. If you know Perl you should be able to modify it to fit your needs.

vikas1234 · Sep 24th, 2008, 8:19 AM

Nice code in C++ .... But perl regex is best to handle this type of problem

Fall Back Son · Sep 24th, 2008, 11:36 AM

^ Why? If he just supplied c++ code that finds a solution in half a second, I would say that the language is perfectly good to handle the problem.

KevinADC · Sep 24th, 2008, 2:56 PM

Quote:

Originally Posted by Sane

An O(N^2) algorithm? What if the file is 100kb?

Even if it compared 10,240 bytes per second. It would still take 12 days to run it only once.

(Toggle Plain Text)

print (100*1024)**2/10240.0/3600.0/24.0

That's one of the worst implementations for such a simple problem I've ever come across.

An O(N) algorithm, such as using an associative array, is just as simple to implement, and will take 10 seconds under the exact same circumstances...

(Toggle Plain Text)

print (100*1024)/10240.0

Did your entire foot fit into your mouth?

Freaky Chris · Sep 24th, 2008, 3:04 PM

ZIP, im gonna think infuture

KevinADC · Sep 24th, 2008, 6:57 PM

if all that is in the file is an ip addresses per line there is no need to use a regexp:

(Toggle Plain Text)

#!/usr/bin/perl

@log = <>;
 ++$count{$_} for @log;
foreach $ip (keys %count) {
   print "$ip $count{$ip}\n" if ;	
}

that should run very fast.

Dec 22nd, 2007, 9:48 AM	#12
Sane Programming Guru Join Date: Apr 2005 Location: Waterloo, Ontario Posts: 1,886 Rep Power: 5	Re: Count number of occourences of a IP address in a log file. Hah, I can't believe Perl can actually do that in a matter of minutes. But it's still pretty clear that the associative array takes the cake with half a second. Thanks for that demo. Cheers.

Dec 22nd, 2007, 10:42 AM	#13
lynxus Newbie Join Date: Dec 2007 Posts: 4 Rep Power: 0	Re: Count number of occourences of a IP address in a log file. Hey guy's, Glad your all debating how to do this.. I have NO idea what that code does so could anyone clarify what i need to do to achieve my goal hehe. Thanks again. Graham

Dec 22nd, 2007, 11:26 AM	#14
Sane Programming Guru Join Date: Apr 2005 Location: Waterloo, Ontario Posts: 1,886 Rep Power: 5	Re: Count number of occourences of a IP address in a log file. mbd's posted code snippet outputs how many instances, of each IP address, are in the log file that Jessehk posted. I'd recommend using his second snippet (which is the psuedocode that I suggested in my first post), since it takes less than half a second to execute. If you know Perl you should be able to modify it to fit your needs.

Sep 24th, 2008, 8:19 AM	#15
vikas1234 Newbie Join Date: Aug 2008 Posts: 5 Rep Power: 0	Re: Count number of occourences of a IP address in a log file. Nice code in C++ .... But perl regex is best to handle this type of problem __________________ FREE COMPUTER SCIENCE AND PROGRAMMING BOOKS MORE BOOKS

Sep 24th, 2008, 11:36 AM	#16
Fall Back Son Professional Programmer Join Date: Oct 2006 Posts: 257 Rep Power: 2	Re: Count number of occourences of a IP address in a log file. ^ Why? If he just supplied c++ code that finds a solution in half a second, I would say that the language is perfectly good to handle the problem.

Sep 24th, 2008, 3:04 PM	#18
Freaky Chris Professional Programmer Join Date: Dec 2007 Location: England Posts: 270 Rep Power: 1	Re: Count number of occourences of a IP address in a log file. ZIP, im gonna think infuture __________________ Steven Skiena - Algorithms Last edited by Freaky Chris; Sep 24th, 2008 at 3:15 PM.

Sep 24th, 2008, 6:57 PM	#19
KevinADC Newbie Join Date: Jul 2008 Posts: 5 Rep Power: 0	Re: Count number of occourences of a IP address in a log file. if all that is in the file is an ip addresses per line there is no need to use a regexp: (Toggle Plain Text) #!/usr/bin/perl @log = <>; ++$count{$_} for @log; foreach $ip (keys %count) { print "$ip $count{$ip}\n" if ; } #!/usr/bin/perl @log = <>; ++$count{$_} for @log; foreach $ip (keys %count) { print "$ip $count{$ip}\n" if ; } that should run very fast.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Jumping to a specific line number in a text file	Adeil	C++	4	Nov 5th, 2007 9:54 AM
PigLattin Converter, count number of words used in dictionary.	MrSmiley	Python	2	Oct 17th, 2005 4:47 PM
After execution - Error cannot locate /Skin File?	wchar	Visual Basic	1	Mar 5th, 2005 9:04 PM
airport Log program using 3D linked List : problem reading from file	gemini_shooter	C++	0	Mar 2nd, 2005 4:12 PM
Specific file address...	Monkey_features	Delphi	0	Feb 8th, 2005 5:05 PM

Programming Forums > Script Programming > Perl

Count number of occourences of a IP address in a log file.