Programming Forums
User Name Password Register
 

RSS Feed
FORUM INDEX | TODAY'S POSTS | UNANSWERED THREADS | ADVANCED SEARCH

Reply
 
Thread Tools Display Modes
Old Aug 22nd, 2005, 12:08 PM   #1
aled29
Newbie
 
Join Date: Aug 2005
Posts: 2
Rep Power: 0 aled29 is on a distinguished road
Extracting and submitting web data

Hi,

I'm looking for a way to extract data from a website that uses Javascript. I'd like to be able to write a program that monitors the site and build a data file of the information displayed on it. I've no idea how to go about doing this. Any ideas?

Also, similarly, I'd like to be able to automatically input data into the javascript forms on the website, i.e. to submit information.

Thanks

Aled
aled29 is offline   Reply With Quote
Old Aug 22nd, 2005, 12:35 PM   #2
Polyphemus_
Expert Programmer
 
Polyphemus_'s Avatar
 
Join Date: Aug 2005
Location: Rotterdam, the Netherlands
Posts: 942
Rep Power: 4 Polyphemus_ is on a distinguished road
I'm not really sure what you want, but can't you let users input data input in the form, post the data to another webpage, and pass the values as parameters for your applet?
Polyphemus_ is offline   Reply With Quote
Old Aug 22nd, 2005, 1:02 PM   #3
Dizzutch
Professional Programmer
 
Dizzutch's Avatar
 
Join Date: Dec 2004
Location: Worcester, MA
Posts: 441
Rep Power: 4 Dizzutch is on a distinguished road
Send a message via ICQ to Dizzutch Send a message via AIM to Dizzutch Send a message via MSN to Dizzutch Send a message via Yahoo to Dizzutch
you can use perl, php, python, ruby or whatever to open a socket to the website, download the page, and grab whatever data you need. The problem might be the javascript, but you should ignore that, cuz it blows anyway.
-Dizz
__________________
naked pictures of you | PFO F@H stats
Dizzutch is offline   Reply With Quote
Old Aug 22nd, 2005, 2:58 PM   #4
Infinite Recursion
Programming Guru
 
Infinite Recursion's Avatar
 
Join Date: Jul 2004
Location: United States
Posts: 3,473
Rep Power: 8 Infinite Recursion is on a distinguished road
Send a message via MSN to Infinite Recursion Send a message via Yahoo to Infinite Recursion
yeah, i was actually goint to suggest Perl or PHP but I didn't know if this individual was die-hard javascript or not... lol
__________________
http://jasonpowers.net

"There are a thousand hacking at the branches of evil to one who is striking at the root."
Infinite Recursion is offline   Reply With Quote
Old Aug 22nd, 2005, 4:04 PM   #5
aled29
Newbie
 
Join Date: Aug 2005
Posts: 2
Rep Power: 0 aled29 is on a distinguished road
Sorry. Its the website that uses javascript. The data is in a javascript website, www.betfair.com, an I'd like to be able to read data from it / input data into it somehow.
aled29 is offline   Reply With Quote
Old Aug 22nd, 2005, 4:48 PM   #6
Dizzutch
Professional Programmer
 
Dizzutch's Avatar
 
Join Date: Dec 2004
Location: Worcester, MA
Posts: 441
Rep Power: 4 Dizzutch is on a distinguished road
Send a message via ICQ to Dizzutch Send a message via AIM to Dizzutch Send a message via MSN to Dizzutch Send a message via Yahoo to Dizzutch
maybe you could use links2 or lynx to get the data, save it to an html file, and parse it from there, not sure quite sure. That's assuming you're on linux. If you're on windows, you're probably screwed.
__________________
naked pictures of you | PFO F@H stats
Dizzutch is offline   Reply With Quote
Old Aug 22nd, 2005, 5:52 PM   #7
Arevos
Programming Guru
 
Arevos's Avatar
 
Join Date: Aug 2005
Location: England
Posts: 1,499
Rep Power: 5 Arevos is on a distinguished road
Take a look at HTTPUnit; it's a unit testing framework for web applications that supports most javascript functions. Obviously you don't want a unit testing framework, but the libraries that HTTPUnit uses could point you in the right direction.
Arevos is offline   Reply With Quote
Reply

Bookmarks

« Previous Thread in Forum | Next Thread in Forum »

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump




DaniWeb IT Discussion Community
All times are GMT -5. The time now is 6:28 PM.

Powered by vBulletin® Version 3.7.0, Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Copyright ©2007 DaniWeb® LLC