View Single Post
Old Dec 1st, 2006, 5:01 PM   #3
mackenga
Professional Programmer
 
Join Date: Mar 2005
Location: Glasgow, Scotland
Posts: 317
Rep Power: 4 mackenga is on a distinguished road
I wouldn't actually write software to do this; I'd just use wget to pull in the files then grep my local copies. Two shell commands isn't enough hassle to convince me to eliminate the manual element.

If you do want to write a program to do this, you could still use wget to do most of the work if you don't mind calling an external program; if all the links to the (say) 1000 files are at http://host/some/path/index.html then you can do:

wget -r -l 1 http://host/some/path/index.html

If the situation is a little more complex you might need a fruitier collection of arguments to wget, but this is definitely the way I'd go.

I'd tend to go for Perl for this problem if you really want to write the software to retrieve the files and skim through them for a substring yourself; Perl's regular expressions make it ideal for the searching part of the job. JavaScript - by which I assume you mean JScript running inside WSH rather than JavaScript embedded in a web page (which can't do much) - could manage the skimming part but I'm not sure off the top of my head how I'd go about retrieving the files over HTTP. Heh, I'd probably call wget from the JScript then using the FileSystemObject to open the files and read through them actually.

Anyway, hope this helps.
__________________
"I'm not a genius. Why do I have to suffer?"
mackenga is offline   Reply With Quote