Programming Forums
User Name Password Register
 

RSS Feed
FORUM INDEX | TODAY'S POSTS | UNANSWERED THREADS | ADVANCED SEARCH

Reply
 
Thread Tools Display Modes
Old Oct 10th, 2005, 1:00 PM   #1
waffch
Newbie
 
Join Date: Oct 2005
Posts: 3
Rep Power: 0 waffch is on a distinguished road
command to download a webpage

i am very new with bash scripting..shell scripting in general!! does anyone know how i can download a webpage and then display all the links in it??? something to do with the html code..a=href...

light peace love
chim wafflekisser
waffch is offline   Reply With Quote
Old Oct 10th, 2005, 1:13 PM   #2
Dizzutch
Professional Programmer
 
Dizzutch's Avatar
 
Join Date: Dec 2004
Location: Worcester, MA
Posts: 441
Rep Power: 4 Dizzutch is on a distinguished road
Send a message via ICQ to Dizzutch Send a message via AIM to Dizzutch Send a message via MSN to Dizzutch Send a message via Yahoo to Dizzutch
you'll need to download the page with 'wget' then parse it using 'egrep' and regular expressions or so, check the man pages of these two programs to learn about them.

I think the regex will look something like this, but don't quote me because i could be wrong

s/<a href="(.*)">(.*)</a>/

now $1 will be the linked page, and $2 the title of the link.

-Dizz
__________________
naked pictures of you | PFO F@H stats
Dizzutch is offline   Reply With Quote
Old Nov 1st, 2005, 7:22 AM   #3
waffch
Newbie
 
Join Date: Oct 2005
Posts: 3
Rep Power: 0 waffch is on a distinguished road
how about if i need to recursively search thru a directory for all the TEXT files contained in it? i need to supply one or two keywords for this, which i must use as arguments..any idea?? or is my question a bit unclear?? hee..thanks!!
waffch is offline   Reply With Quote
Old Nov 1st, 2005, 7:42 AM   #4
tempest
Programming Guru
 
tempest's Avatar
 
Join Date: Oct 2004
Posts: 1,041
Rep Power: 6 tempest is on a distinguished road
Send a message via ICQ to tempest Send a message via AIM to tempest Send a message via Yahoo to tempest
find `pwd` -name "*.txt"
__________________

tempest is offline   Reply With Quote
Old Nov 1st, 2005, 11:03 AM   #5
waffch
Newbie
 
Join Date: Oct 2005
Posts: 3
Rep Power: 0 waffch is on a distinguished road
i will try this, thanks. is that how you do it with shell scripting? i need to write it in .sh format..i need to write my code in vi and then run it then supply my arguments.. first the directory and then the filename..

.:."I'll tell you what's the matter!! This parrot is dead!!".:.
waffch is offline   Reply With Quote
Old Nov 1st, 2005, 11:45 AM   #6
Dizzutch
Professional Programmer
 
Dizzutch's Avatar
 
Join Date: Dec 2004
Location: Worcester, MA
Posts: 441
Rep Power: 4 Dizzutch is on a distinguished road
Send a message via ICQ to Dizzutch Send a message via AIM to Dizzutch Send a message via MSN to Dizzutch Send a message via Yahoo to Dizzutch
in the future, start a new thread with a new question, that way the forum becomes more searchable for other users.
__________________
naked pictures of you | PFO F@H stats
Dizzutch is offline   Reply With Quote
Old Feb 21st, 2006, 4:25 PM   #7
mackenga
Professional Programmer
 
Join Date: Mar 2005
Location: Glasgow, Scotland
Posts: 321
Rep Power: 4 mackenga is on a distinguished road
In answer to the "download web page from the shell prompt/in a shell script, display links" question at the top; I think wget does this conveniently. If you're on a linux box you probably already have it; if not, it's free and small. Google for it.
mackenga is offline   Reply With Quote
Old May 18th, 2006, 4:35 AM   #8
DaveQB
Newbie
 
DaveQB's Avatar
 
Join Date: Apr 2006
Location: Sydney
Posts: 14
Rep Power: 0 DaveQB is on a distinguished road
Send a message via ICQ to DaveQB Send a message via Skype™ to DaveQB
Quote:
Originally Posted by waffch
i will try this, thanks. is that how you do it with shell scripting? i need to write it in .sh format..i need to write my code in vi and then run it then supply my arguments.. first the directory and then the filename..

.:."I'll tell you what's the matter!! This parrot is dead!!".:.

Not sure of the question, but I think your asking how to use arguments with shell scripts ??

Each argument is given a number. So the first argument to the file is $1 the second $2 and so forth.

The script itself is $0. Which is handy to use in case the file name changes down the track.

Functions in shell scripts also use this same format, so its wise to assign the arguments to better variables at the start of the script. EG

var_one="$1"
var_two="$2"
DaveQB is offline   Reply With Quote
Reply

Bookmarks

« Previous Thread in Forum | Next Thread in Forum »

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump




DaniWeb IT Discussion Community
All times are GMT -5. The time now is 7:50 PM.

Powered by vBulletin® Version 3.7.0, Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Copyright ©2007 DaniWeb® LLC