Programming Forums > Script Programming > Python

zem52887 · May 24th, 2006, 11:28 AM

I apologize for being unclear, I've isolated the tables for both Key People and Financial Highlights. In your previous post, you said that while I isolated the tables, I still needed to extract the data in a way that we could import into excel as importing the table is not an option. Thus, my question is how I can further parse for the relevent information out of the tables-- if there aren't constant values (i.e. there's not always a CEO etc.) in these two tables.

Arevos · May 24th, 2006, 11:51 AM

I still don't see how that would affect it. If you have the table, then the location of the information should be fairly predictable. The Key People table, for instance, seems to consist of two cells. One containing the header ("Key People") and the other containing an unordered list of key people. Surely it's just a case of getting the information out of the second cell?

The Finanical Highlights table has a pretty basic structure too. Each row contains the name, or description of the item in the first column (e.g. "Fiscal Year End"), and the value of the item in the second column (e.g. "August").

zem52887 · May 24th, 2006, 11:56 AM

hm I'm definately missing something, not sure what though. If one company has 4 rows of data under "Financial Highlights" and another company has 6 rows of data, then won't I have a problem fetching the cells? Did that make sense? My thought process is that since there aren't a set amount of cells under Financial Highlights, I can't call td[0],td[1], because some have more and some have less... I'm really struggling to verbalize what I'm thinking right now... rough night last night, so let me know if you still don't get what I'm trying to say.

Arevos · May 24th, 2006, 12:11 PM

Why not format all the rows into one string?

(Toggle Plain Text)

lines = []     # create a list to hold some lines of text

# For each row in the table
for row in table.fetch("tr"):
    # Add a line that is "cell[0]: cell[1]"
    # e.g. "Fiscal Year End: August"

    cells = row.fetch("td")
    lines.append(cells[0] + ": " + cells[1])

# Join the list of lines up into one big string
data = "\n".join(lines)

zem52887 · May 24th, 2006, 12:23 PM

Okay I think I need to refer to the python tutorial because I'm at a loss with above code.

Hey Arevos, I hate to ask but I referred to the manual and am still clueless... do you think you could briefly explain what the above code does so that I have a better idea of how to implement it?

Arevos · May 24th, 2006, 12:57 PM

Okay, first I think that you should approach this problem from a different angle. I think you need a better idea of what you're going to put in the CSV file.

Thus, I suggest that you pick a company (or two) at random, and manually create a CSV file that will hold that company's data. Then try importing that CSV file into Excel and seeing what happens.

When you have a CSV file that is imported correctly, then you'll have a better idea of what your program is creating. Put aside your code for the moment, and work on that

zem52887 · May 24th, 2006, 1:00 PM

Okay I'll try that now, but before I do, I posted over at ocforums and someone recommended that a mysql or sqlite database would be better suited rather than an excel spreadsheet... are you familiar with either of these and if so, what are your thoughts?

also, when you say manually, do you mean run a test for get_company_data
#company profile then take that resulting data and put it into a text file and so on and so forth for #contact information and #company name?

wow, so I tried making one and it came out so awful hah. back to the drawing board. I don't know why I magically thought that importing it would remove the <font> tags etc... but I got a mess of everything including tags... Ultimately I need to isolate text alone, without any HTML tags, no?

Arevos · May 24th, 2006, 1:20 PM

Quote:

Originally Posted by zem52887

Okay I'll try that now, but before I do, I posted over at ocforums and someone recommended that a mysql or sqlite database would be better suited rather than an excel spreadsheet... are you familiar with either of these and if so, what are your thoughts?

MySQL and SQLite are relational databases, a little like Microsoft Access, if you're familiar with that piece of software. SQLite is generally for small quantities of data, MySQL for larger quantities.

Yes, they would be a much better option for storing this sort of company information, at least in general. However, since your task was to put this information into an Excel spreadsheet, presumably you don't have the choice of using a database.

Also, relational databases are not as readily accessable as Excel. They're back-end, behind-the-scenes systems, that usually require a user interface to be created before non-technical people can use them. The data in Excel can be viewed by just double-clicking on the file icon.

Quote:

Originally Posted by zem52887

also, when you say manually, do you mean run a test for get_company_data
#company profile then take that resulting data and put it into a text file and so on and so forth for #contact information and #company name?

No, I mean visit the site in a browser, and pull out the information via copy and paste. But instead of putting it straight into Excel, put in a CSV file instead and import it into Excel.

Quote:

Originally Posted by zem52887

wow, so I tried making one and it came out so awful hah. back to the drawing board

Experiment until you get something that looks right.

zem52887 · May 24th, 2006, 1:24 PM

Well the assignment was pretty vague. I chose an excel spreadsheet when I was manually copying and pasting because that's what I was most familiar with at the time. However, I don't think it would be a problem if I wanted to switch over to a database (other than the obvious problem of me not knowing anything about them) but with that aside, if I were to get some aid in creating a database, the script we've been working on would still work for it (maybe with some changes but the bulk of it would still be usable, no?)

zem52887 · May 24th, 2006, 1:33 PM

Quote:

No, I mean visit the site in a browser, and pull out the information via copy and paste. But instead of putting it straight into Excel, put in a CSV file instead and import it into Excel.

I tried this... and I'm running into problems in that they're going into a singular cell, the company description takes up 4 cells etc. Do I need to assign the company description to a value because it doesn't seem like it would work if I were merely copy the whole description and separate it with a comma.

May 24th, 2006, 11:56 AM	#153
zem52887 Hobbyist Programmer Join Date: May 2006 Posts: 127 Rep Power: 3	hm I'm definately missing something, not sure what though. If one company has 4 rows of data under "Financial Highlights" and another company has 6 rows of data, then won't I have a problem fetching the cells? Did that make sense? My thought process is that since there aren't a set amount of cells under Financial Highlights, I can't call td[0],td[1], because some have more and some have less... I'm really struggling to verbalize what I'm thinking right now... rough night last night, so let me know if you still don't get what I'm trying to say. Last edited by zem52887; May 24th, 2006 at 12:07 PM.

May 24th, 2006, 12:11 PM	#154
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 5	Why not format all the rows into one string? (Toggle Plain Text) lines = [] # create a list to hold some lines of text # For each row in the table for row in table.fetch("tr"): # Add a line that is "cell[0]: cell[1]" # e.g. "Fiscal Year End: August" cells = row.fetch("td") lines.append(cells[0] + ": " + cells[1]) # Join the list of lines up into one big string data = "\n".join(lines) lines = [] # create a list to hold some lines of text # For each row in the table for row in table.fetch("tr"): # Add a line that is "cell[0]: cell[1]" # e.g. "Fiscal Year End: August" cells = row.fetch("td") lines.append(cells[0] + ": " + cells[1]) # Join the list of lines up into one big string data = "\n".join(lines)

May 24th, 2006, 1:00 PM	#157
zem52887 Hobbyist Programmer Join Date: May 2006 Posts: 127 Rep Power: 3	Okay I'll try that now, but before I do, I posted over at ocforums and someone recommended that a mysql or sqlite database would be better suited rather than an excel spreadsheet... are you familiar with either of these and if so, what are your thoughts? also, when you say manually, do you mean run a test for get_company_data #company profile then take that resulting data and put it into a text file and so on and so forth for #contact information and #company name? wow, so I tried making one and it came out so awful hah. back to the drawing board. I don't know why I magically thought that importing it would remove the <font> tags etc... but I got a mess of everything including tags... Ultimately I need to isolate text alone, without any HTML tags, no? Last edited by zem52887; May 24th, 2006 at 1:16 PM.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

May 24th, 2006, 11:28 AM	#151
zem52887 Hobbyist Programmer Join Date: May 2006 Posts: 127 Rep Power: 3	I apologize for being unclear, I've isolated the tables for both Key People and Financial Highlights. In your previous post, you said that while I isolated the tables, I still needed to extract the data in a way that we could import into excel as importing the table is not an option. Thus, my question is how I can further parse for the relevent information out of the tables-- if there aren't constant values (i.e. there's not always a CEO etc.) in these two tables.

May 24th, 2006, 11:51 AM	#152
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 5	I still don't see how that would affect it. If you have the table, then the location of the information should be fairly predictable. The Key People table, for instance, seems to consist of two cells. One containing the header ("Key People") and the other containing an unordered list of key people. Surely it's just a case of getting the information out of the second cell? The Finanical Highlights table has a pretty basic structure too. Each row contains the name, or description of the item in the first column (e.g. "Fiscal Year End"), and the value of the item in the second column (e.g. "August").

May 24th, 2006, 12:23 PM	#155
zem52887 Hobbyist Programmer Join Date: May 2006 Posts: 127 Rep Power: 3	Okay I think I need to refer to the python tutorial because I'm at a loss with above code. Hey Arevos, I hate to ask but I referred to the manual and am still clueless... do you think you could briefly explain what the above code does so that I have a better idea of how to implement it?

May 24th, 2006, 12:57 PM	#156
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 5	Okay, first I think that you should approach this problem from a different angle. I think you need a better idea of what you're going to put in the CSV file. Thus, I suggest that you pick a company (or two) at random, and manually create a CSV file that will hold that company's data. Then try importing that CSV file into Excel and seeing what happens. When you have a CSV file that is imported correctly, then you'll have a better idea of what your program is creating. Put aside your code for the moment, and work on that

May 24th, 2006, 1:24 PM	#159
zem52887 Hobbyist Programmer Join Date: May 2006 Posts: 127 Rep Power: 3	Well the assignment was pretty vague. I chose an excel spreadsheet when I was manually copying and pasting because that's what I was most familiar with at the time. However, I don't think it would be a problem if I wanted to switch over to a database (other than the obvious problem of me not knowing anything about them) but with that aside, if I were to get some aid in creating a database, the script we've been working on would still work for it (maybe with some changes but the bulk of it would still be usable, no?)

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Programming Forums > Script Programming > Python

Never Programmed Before -- Have Task that makes me want to learn or ... die.