Programming Forums > Application Development > C#

tayspen · Jan 31st, 2006, 4:30 PM

Hi,

I have a question about screen scraping and Regular Expressions. There is a website, and of the source code, I want to get this line

(Toggle Plain Text)

PLAYERS:<b><br><font color="#ea0437"><b>Be Text Here</b></font><br><font color="#ea0437"><b>Be Text Here</b></font><br><font color="#ea0437"><b>Be Text Here</b></font><br><br><font color="#1796cb"><b>Be Text Here</b></font><br><font color="#1796cb"><b>Be Text Here</b></font><br>

Then after i get that line i want to go further and get all the text where the

Quote:

Originally Posted by it says Be Text Here but it will be differ

Be Text Here

is, then i want to list those in a listbox. Can this be done? Also that line might have more instances of Be Text Here it will vary....

Dameon · Jan 31st, 2006, 7:29 PM

If the source code is XHTML compliant, use the nifty features of the XML namespace. Load it as an XML document and use XPath to select the nodes you want.

tayspen · Jan 31st, 2006, 8:18 PM

English please? Lol, serioulsy

Quote:

Originally Posted by Dameon said this....

If the source code is XHTML compliant

how do i tell?

Quote:

Originally Posted by again...

use the nifty features of the XML namespace. Load it as an XML document and use XPath to select the nodes you want.

What are the features? And what is XPath?

*Goes to look on MSDN*

-T

hoffmandirt · Feb 8th, 2006, 6:09 PM

The source doesn't look XHTML compliant so it looks like you are just going to have to hope the html code never changes. After you get the first line that you want, you will probably have to parse that line based on the second bold tag you come across. This is why screen scraping is not recommended, but sometimes there is no other choice. If someone takes the bold tags out then your code will not work correctly anymore. Just a chance you gotta take.

kurifu · Feb 12th, 2006, 12:03 AM

That is one of the great things about writing screen scrapers, putting terms in the contract which make you more money when the web source does change

Of course, this is when you are writing the code for someone else, who is much less programming savy

hoffmandirt · Feb 13th, 2006, 9:49 AM

Hmmm I never thought about it like that

Sridhar · Feb 15th, 2006, 1:09 PM

hello everyone, Can anyone help me... I am right now working on a project to create a Microsoft Outlook 2003 Addin... Tell me how to integrate a stand alone application in to Microsoft Outlook 2003

Dameon · Feb 15th, 2006, 7:43 PM

1. Read the posting guidelines
2. Make your own thread. Hijacking is bad.
3. ???
4. Success!

J_Kay · Feb 15th, 2006, 10:36 PM

Quote:

Originally Posted by Dameon

1. Read the posting guidelines
2. Make your own thread. Hijacking is bad.
3. ???
4. Success!

I thought step 3 was always Profit. Just a thought......

Jan 31st, 2006, 7:29 PM	#2
Dameon Troll Join Date: Apr 2005 Location: Texas Posts: 732 Rep Power: 4	If the source code is XHTML compliant, use the nifty features of the XML namespace. Load it as an XML document and use XPath to select the nodes you want. __________________ MD5(sig) = bcef75433db02e9ad9bf81d6f7c5c270

Feb 12th, 2006, 12:03 AM	#5
kurifu Expert Programmer Join Date: Jul 2004 Location: Halifax, Nova Scotia (Canada) Posts: 784 Rep Power: 5	That is one of the great things about writing screen scrapers, putting terms in the contract which make you more money when the web source does change Of course, this is when you are writing the code for someone else, who is much less programming savy __________________ Clifford Matthew Roche <geek@cliffordroche.com> Web Hosting: http://www.crd-hosting.com Consulting: http://www.crdev-consulting.com

Feb 15th, 2006, 7:43 PM	#8
Dameon Troll Join Date: Apr 2005 Location: Texas Posts: 732 Rep Power: 4	1. Read the posting guidelines 2. Make your own thread. Hijacking is bad. 3. ??? 4. Success! __________________ MD5(sig) = bcef75433db02e9ad9bf81d6f7c5c270

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Feb 8th, 2006, 6:09 PM	#4
hoffmandirt Hobbyist Programmer Join Date: Jul 2005 Location: PA Posts: 125 Rep Power: 4	The source doesn't look XHTML compliant so it looks like you are just going to have to hope the html code never changes. After you get the first line that you want, you will probably have to parse that line based on the second bold tag you come across. This is why screen scraping is not recommended, but sometimes there is no other choice. If someone takes the bold tags out then your code will not work correctly anymore. Just a chance you gotta take.

Feb 13th, 2006, 9:49 AM	#6
hoffmandirt Hobbyist Programmer Join Date: Jul 2005 Location: PA Posts: 125 Rep Power: 4	Hmmm I never thought about it like that

Feb 15th, 2006, 1:09 PM	#7
Sridhar Newbie Join Date: Feb 2006 Posts: 1 Rep Power: 0	hello everyone, Can anyone help me... I am right now working on a project to create a Microsoft Outlook 2003 Addin... Tell me how to integrate a stand alone application in to Microsoft Outlook 2003

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Programming Forums > Application Development > C#

Screen Scraping