Programming Forums
User Name Password Register
 

RSS Feed
FORUM INDEX | TODAY'S POSTS | UNANSWERED THREADS | ADVANCED SEARCH

Reply
 
Thread Tools Display Modes
Old Nov 2nd, 2005, 4:25 PM   #1
Pizentios
Programming Guru
 
Pizentios's Avatar
 
Join Date: May 2004
Location: Brandon, Manitoba, Canada
Posts: 2,023
Rep Power: 7 Pizentios is on a distinguished road
Send a message via ICQ to Pizentios Send a message via MSN to Pizentios
Backup Script :-)

Needed a backup script at work, just finished it, probably needs some testing/debuging, but i thought that i would post it anyways :-)

here's the perl script:
#!/usr/bin/env perl

####################################################################################
# This perl script while not only being the first perl that i have ever written, it#
# will backup a list of directories by taring, bziping them, then creating md5	 #
# check sums in order to check them later if i have to resote any of the files.	#
####################################################################################


use 5.8.6;
use strict;
use Getopt::Long;
use IO::File;
use Archive::Tar;
use Digest::MD5;
use Cwd;


sub gen_arch_name {
	my $filename = $_[0]; #should only be passing just the locations of the file/dir to back up.
	my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime(time); #get info about the date, to gen the backup file names.
	$mon = $mon + 1; #human readable month.
	$year = $year + 1900; # human readable year.
	$filename=~ s/\//_/g; #replace all / with _
	$filename = reverse $filename; #reverse the file name so we can get rid of the leading _
	chop $filename; #get rid of the last _, which is really the first _
	$filename = reverse $filename; #get it back to the original filename, instead of backwards.
	my $arch_name = "$filename$year-$mon-$mday.tar.bz2";
	
	return $arch_name;

}

sub Check {
	my $list = $_[0]; #only ever one parm.
	my $md5 = Digest::MD5->new;
	my $check = 1;
	open(LIST, $list) or die "Error, can't open $list for reading. Please make sure that $list is there.";
	
	while (<LIST>) {
		my ($file, $sum, $location) = split(/\|/, $_, 3); #split the line into stuff we can use.
		chomp $location;
		my $full_file = "$location/$file";
		open(FILE, $full_file) or die "Couldn't open $full_file for md5 checksum verification.";
		binmode(FILE);
		my $c_sum = $md5->addfile(*FILE)->hexdigest;
		close FILE;
		if ($c_sum ne $sum) {
			$check = 1;
			last;
		} else {
			$check = 0;
		}
	}
	return $check;
}

sub Backup {
	my ($list, $g, $d)  = @_;
	if ($g) {
		#if the gen option is set, then we need to generate the restore file as well.
		# $g hold the location of were the file is to be outputed to.
		open(LIST, $list) or die "Error, can't open $list for reading. Please make sure that $list is there.";
		open(RES, ">>$g") or die "Error, can't open $g for writing. Please make sure that the path $g is there.";
		my $arch_name;
		my $md5 = Digest::MD5->new; #create the md5 object.
		while (<LIST>) {
			#loop and do stuff with the data.
			chomp $_;
			#make sure that it's not a symlink.
			if ((-e $_) && (-s $_) && (! -l $_)) {
				#we can continue.
				#now tar and bzip the directory.
		 	$arch_name = gen_arch_name($_); #correctly formatted backup name. 
		 	system("tar -cjf $arch_name $_"); #tar and bz2 the directory.
		 	#now to make the md5 check sum, and input the archive and the key into the output file that the user wanted.
		 	open(FILE, $arch_name) or die "Couldn't open $arch_name for md5 checksum generation."; #open the archive for md5 checksum generation.
				binmode(FILE);
		 	my $sum = $md5->addfile(*FILE)->hexdigest; #create and store the md5 checksum in the var $sum.
				close FILE;
		 	#k, now we have the checksum, dump the archive file name and the checksum into the output file, if it's enabled.
				if ($d) {
		 		print RES "$arch_name|$sum|$d\n"; #print the archive name, the checksum and the backup location to the restore file.
		 		#now to move the tar.bz2's to the backup directory.
					system("mv $arch_name $d");
				} else {
					my $c_dir = getcwd;
		 		print RES "$arch_name|$sum|$c_dir\n"; #print the archive name, the checksum and the current working directory since that's were the backs are going to get stored.
				}
			} #else do nothing.
		}
		close LIST;
		close RES;
		#now check all files in the restore file, against a new md5 checksum, to make sure that it's all good.
		my $check = Check($g);
		if ($check == 0)
		{
			#check works, let the user know.
			print "All file checksums check out.\n";
		} else {
			print "One or more checksums do not check out.\n";
		}
	} else {
		#do not generate teh restore file.
		open(LIST, $list) or die "Error can't open $list for reading. Please make sure that $list is there.\n";

		while(<LIST>) {
			chomp $_;
			if ((-e $_) && (-s $_) && (! -l $_)) {
		 	my $arch_name = gen_arch_name($_); #correctly formated archive name.
		 	system("tar -cjf $arch_name $_"); #tar and bz2 the dir/file.
				if ($d) {
					system("mv $arch_name $d");
				}
			}		
		}
		close LIST;
	}
}

sub Restore {
	my $list = $_[0]; #should only ever be one parm.
	my $check = Check($list);
	if ($check == 0) {
		open(LIST, $list) or die "Error, can't open $list for reading. Please make sure that $list is there.\n";
		
		while (<LIST>) {
			my ($file, $sum, $location) = split(/\|/, $_, 3);
			chomp $location;
			my $file_l = "$location/$file"; #the current location of the file we are going to restore.
			my $restore_p = $file;
			$restore_p =~ s/_/\//g; #replace the _ with /
			$restore_p =~ s/(\d\d\d\d)-(\d+)-(\d+).tar.bz2//g;
			#now we have the correct restore point.
			#however we still need to handle single files correctly.
			my $len = length($restore_p);
			my $offset = $len - 1;
			my $check = substr($restore_p, $offset, 1);
			if ($check ne "/") {
		 	#we are working with a single file. Strip the last part of the location off, since we only need the directories.
				$restore_p = reverse $restore_p;
				my @path = split(//, $restore_p);
				my ($i, $word);
				for ($i=0; $i<$len; $i++) {
					if (@path[$i] ne "/") {
		 			$word = "$word@path[$i]";
					} else {
						last;
					}
				}
				$word = reverse $word;
				$restore_p = reverse $restore_p;
				$restore_p =~ s/$word//g;
				$restore_p= "/$restore_p";
			} #else we are working with a directory and need to change nothing.
			system ("tar -xvjf $file_l -C /");
		}
		close LIST;
	} else {
		print "Error, one or more of your backups might be bad or have been tampered with. Checksums do not match!!!";
	}
}

sub arg_check {
	#Check the arguments.
	my ($b, $c, $r) = @_; #grab the sub args, then assign them.
	my $answer = 0; #set answer to 0
	if ($b) {
		$answer = $answer + 1;
	}
	
	if ($c) {
		$answer = $answer + 1;
	}

	if ($r) {
		$answer = $answer + 1;
	}

	if ($answer > 1) {
		#if answer is larger than 1, then the user has set to options that don't belong.
		#return failure. else the user has the right options.
		return 1;
	} else {
		return 0;
	}

}
#Use getops to parse the stuff the user passes to us.
my ($l_location, $check, $backup, $restore, $gen, $dir);
GetOptions("list=s" => \$l_location, #the location of the list we want to backup.
	 "check" => \$check, #if the user wants to run a check on a set of backups, defined in $l_location.
	   "backup" => \$backup, #if the user wants to backup the files listed in the file in $l_location. 
	   "restore"=> \$restore, #if the user want to restore the files listed in the file in $l_location.
	 "generate=s"=> \$gen, #if enabled, generate a resotre file, instead of just outputing to the screen.
	 "dir=s"	=> \$dir, #the location of the directory where the backups will be placed. If there is no dir option set, the current working directory will be used.
	   );

die "Error, you must supply a file/directory listing with the list option!" unless defined $l_location;
if ($backup || $check || $restore) {
	my $arg_c = arg_check($check, $backup, $restore);
	if ($arg_c == 1) {
		#error, more switches have been set than needed.
		die "Error, you have supplied more command options that needed!";
	} else {
		#there is only one command option set.
		if ($backup) {
			#backup proceedure.
			Backup($l_location, $gen, $dir);
		} elsif ($check) {
			#check proceedure.
			my $check = Check($l_location);
			if ($check == 0)
			{
				#checked out.
				print "All checksums check out.\n";
			} else {
		 	print "One or more of the checksums don't check out.\n";
			}
		} elsif ($restore) {
			#restore proceedure.
			Restore($l_location);
		}
	}
} else {
	die "Error, you must set atleast one command option!";
}

to use it you'd do something like this:
./backup.pl --backup --generate /home/pizentios/restore_file --dir /home/pizentios/backup/ --list /home/pizentios/backup_list
that would be to backup all files/directories listed in the file backup_list in my home directory. the --generate option tells the script that i want to make a restore file, so that it's easy for me to restore at the event of my files being deleted/smashed/broken what ever. you don't need the --generate option for it to work, it will create the backups for you, but you won't have the luxary of using the restore feature later on.

This script uses md5 check sums to check the backups before moving them to the backup dir, which could be a normal directory or a removable drive.

So to restore the backup that i did above i would do this:
./backup.pl --restore --list /home/pizentios/restore_file

or if i just wanted to run a check on the backups (maybe because i though we had a break in, or maybe i think one of the backups could be corupt), i would do something like this:
./backup.pl --check --list /home/pizentios/restore_file


anyways, this was my first perl program, so i know that it's sloppy, and could most likely be optimized....just thought that i would share.
__________________
Profanity is the one language that all programmers understand.

Check out my Blog <---updated Nov 30 2007!
Pizentios is offline   Reply With Quote
Old Nov 2nd, 2005, 4:27 PM   #2
Senger
Programmer
 
Join Date: Oct 2005
Location: Vancouver, Canada
Posts: 47
Rep Power: 0 Senger is on a distinguished road
In Win2000, I just do:
xcopy /e project project1
(Sorry, I couldn't resist....)
__________________
It can be proved that you can watch streaming video on a Turing machine. The question is, why would you want to?
Senger is offline   Reply With Quote
Old Nov 2nd, 2005, 4:28 PM   #3
Pizentios
Programming Guru
 
Pizentios's Avatar
 
Join Date: May 2004
Location: Brandon, Manitoba, Canada
Posts: 2,023
Rep Power: 7 Pizentios is on a distinguished road
Send a message via ICQ to Pizentios Send a message via MSN to Pizentios
i shoudl note that the list for the backup operation shoudl be a list of full paths to the file/directories that you want to backup, for all other operations the list should be your restore file.
__________________
Profanity is the one language that all programmers understand.

Check out my Blog <---updated Nov 30 2007!
Pizentios is offline   Reply With Quote
Old Nov 2nd, 2005, 4:28 PM   #4
zorin
Hobbyist Programmer
 
Join Date: Apr 2005
Posts: 218
Rep Power: 4 zorin is on a distinguished road
Nice, I wish my programs were like that the first time I tried.
zorin is offline   Reply With Quote
Old Nov 2nd, 2005, 6:06 PM   #5
sykkn
Hobbyist Programmer
 
Join Date: Apr 2004
Location: Texas
Posts: 106
Rep Power: 5 sykkn is on a distinguished road
        $filename=~ s/\//_/g; #replace all / with _
        $filename = reverse $filename; #reverse the file name so we can get rid of the leading _
        chop $filename; #get rid of the last _, which is really the first _
        $filename = reverse $filename; #get it back to the original filename, instead of backwards.

why?

        $filename=~ s/\//_/g; #replace all / with _
        $filename=~ s/^_//;   #replace the leading _

I don't like using chop because it just drops the last character ... it doesn't care if that character is actually what you wanted to drop ...

perldoc says: "It is much more efficient than 's/.$//s' because it neither scans nor copies the string."

but you reverse the string twice and I think that probably removes any advantage there may have been ...


Just a comment ... if you disagree, I'd liked to hear your reasoning for my own benefit ... I learn something new every day
__________________
[ [ SykkN alloc ] initWithThePowerTo: destroyYouAll ];
/* Don't make me use it! */
sykkn is offline   Reply With Quote
Old Nov 3rd, 2005, 8:27 AM   #6
Infinite Recursion
Programming Guru
 
Infinite Recursion's Avatar
 
Join Date: Jul 2004
Location: United States
Posts: 3,467
Rep Power: 8 Infinite Recursion is on a distinguished road
Send a message via MSN to Infinite Recursion Send a message via Yahoo to Infinite Recursion
Looks good Pizentios... will test it out this weekend on my network at home.

Sykkn, it more than likely boils down to personal preference... because he could have done:

$filename = substr($filename,1,length($filename)-1);
__________________
http://jasonpowers.net

"There are a thousand hacking at the branches of evil to one who is striking at the root."
Infinite Recursion is offline   Reply With Quote
Old Nov 3rd, 2005, 9:27 AM   #7
Pizentios
Programming Guru
 
Pizentios's Avatar
 
Join Date: May 2004
Location: Brandon, Manitoba, Canada
Posts: 2,023
Rep Power: 7 Pizentios is on a distinguished road
Send a message via ICQ to Pizentios Send a message via MSN to Pizentios
yeah i wrote the part with chop, before i learned more about regular expression etc... i will most likely be changing it to something a little nicer :-)
__________________
Profanity is the one language that all programmers understand.

Check out my Blog <---updated Nov 30 2007!
Pizentios is offline   Reply With Quote
Old Nov 3rd, 2005, 10:27 AM   #8
Infinite Recursion
Programming Guru
 
Infinite Recursion's Avatar
 
Join Date: Jul 2004
Location: United States
Posts: 3,467
Rep Power: 8 Infinite Recursion is on a distinguished road
Send a message via MSN to Infinite Recursion Send a message via Yahoo to Infinite Recursion
At least you got the end result that was needed
__________________
http://jasonpowers.net

"There are a thousand hacking at the branches of evil to one who is striking at the root."
Infinite Recursion is offline   Reply With Quote
Old Nov 3rd, 2005, 11:26 AM   #9
Pizentios
Programming Guru
 
Pizentios's Avatar
 
Join Date: May 2004
Location: Brandon, Manitoba, Canada
Posts: 2,023
Rep Power: 7 Pizentios is on a distinguished road
Send a message via ICQ to Pizentios Send a message via MSN to Pizentios
Heh, yeah, but i like pretty code, rather than ugly code :-)
__________________
Profanity is the one language that all programmers understand.

Check out my Blog <---updated Nov 30 2007!
Pizentios is offline   Reply With Quote
Old Nov 9th, 2005, 3:13 PM   #10
Pizentios
Programming Guru
 
Pizentios's Avatar
 
Join Date: May 2004
Location: Brandon, Manitoba, Canada
Posts: 2,023
Rep Power: 7 Pizentios is on a distinguished road
Send a message via ICQ to Pizentios Send a message via MSN to Pizentios
UPDATE: i am working on a feature to restore only one file out of a directory backup, i'll post code when it's done.
__________________
Profanity is the one language that all programmers understand.

Check out my Blog <---updated Nov 30 2007!
Pizentios is offline   Reply With Quote
Reply

Bookmarks

« Previous Thread in Forum | Next Thread in Forum »

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump




DaniWeb IT Discussion Community
All times are GMT -5. The time now is 6:55 PM.

Powered by vBulletin® Version 3.7.0, Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Copyright ©2007 DaniWeb® LLC