Programming Forums > Script Programming > Python

nytrokiss · Nov 16th, 2006, 11:31 PM

def find_all_items(site):
    site = urllib.urlopen(site).read()
    all_items = re.findall(r'watch\.asp\?\w+\=\w*\&\w*\=\w+',site)
    next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site)
    try:
        find_all_items(urllib.basejoin("<some url>",next_page[0]))
    except IndexError:
        pass
    return remove_dups(all_items) # ,<---- this removes all the duplicate items

How do i keep the value of all_items that i have within the first function call and return a list of all the items at once also from the second function call

Now the list is being overwritten

DaWei · Nov 16th, 2006, 11:34 PM

Is this a tutorial or a question? It doesn't seem to say, anywhere.

titaniumdecoy · Nov 17th, 2006, 1:06 AM

It never fails to amaze me how so many of the newcomers to this forum have such unbelievably bad (written) communication skills.

Arevos · Nov 17th, 2006, 2:34 AM

(Toggle Plain Text)

def find_all_items(site):
    site = urllib.urlopen(site).read()
    all_items = re.findall(r'watch\.asp\?\w+\=\w*\&\w*\=\w+',site)
    next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site)
    try:
        all_items.extend(find_all_items(urllib.basejoin("<some url>",next_page[0])))
    except IndexError:
        pass
    return remove_dups(all_items) # ,<---- this removes all the duplicate items

Also, you may want to think about using the set() data type, which contains only unique values and doesn't retain order.

Perhaps:

(Toggle Plain Text)

def find_all_items(site):
    site = urllib.urlopen(site).read()
    all_items = set(re.findall(r'watch\.asp\?\w+\=\w*\&\w*\=\w+',site))
    next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site)
    try:
        all_items.union(find_all_items(urllib.basejoin("<some url>",next_page[0])))
    except IndexError:
        pass
    return all_items

And I can't see a Python HTML parsing problem without mentioning Beautiful Soup. Though your problem might be simple enough that it needs only regexs.

nytrokiss · Nov 17th, 2006, 3:01 AM

I tried both the examples the issue is that the list all_items is cleared when the function is called again!

Arevos · Nov 17th, 2006, 3:20 AM

Hm. It might help if you showed all your code, as it's difficult to understand exactly what you're asking for. Do you want some sort of persistence? Then you either need a functor:

 python Syntax (Toggle Plain Text) 
class FindAllItemsFunctor
    def __init__(self):
        self.all_items = []
    def __call__(self, site):
        # Your find_all_items code using self.all_items instead of all_items
 
find_all_items = FindAllItemsFunctor()
class FindAllItemsFunctor
    def __init__(self):
        self.all_items = []
    def __call__(self, site):
        # Your find_all_items code using self.all_items instead of all_items

find_all_items = FindAllItemsFunctor()

Or you need to use some functions within functions:

 python Syntax (Toggle Plain Text) 
def find_all_items_creater():
    all_items = []
    def find_all_items(site):
        # ...
    return find_all_items
 
find_all_items = find_all_items_creater()
def find_all_items_creater():
    all_items = []
    def find_all_items(site):
        # ...
    return find_all_items

find_all_items = find_all_items_creater()

Or you could use a global variable:

 python Syntax (Toggle Plain Text) 
all_items = []
def find_all_items(site):
    global all_items
    # ...
all_items = []
def find_all_items(site):
    global all_items
    # ...

Or even do something clever with the function's parameters:

 python Syntax (Toggle Plain Text) 
def find_all_items(site, all_items = [])
    # ...
def find_all_items(site, all_items = [])
    # ...

Nov 16th, 2006, 11:31 PM	#1
nytrokiss Newbie Join Date: Oct 2006 Posts: 23 Rep Power: 0	Retaining the contents of a list during a recursive function (Toggle Plain Text) def find_all_items(site): site = urllib.urlopen(site).read() all_items = re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: find_all_items(urllib.basejoin("<some url>",next_page[0])) except IndexError: pass return remove_dups(all_items) # ,<---- this removes all the duplicate items def find_all_items(site): site = urllib.urlopen(site).read() all_items = re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: find_all_items(urllib.basejoin("<some url>",next_page[0])) except IndexError: pass return remove_dups(all_items) # ,<---- this removes all the duplicate items How do i keep the value of all_items that i have within the first function call and return a list of all the items at once also from the second function call Now the list is being overwritten

Nov 16th, 2006, 11:34 PM	#2
DaWei Resident Grouch Join Date: Jun 2005 Posts: 6,453 Rep Power: 10	Is this a tutorial or a question? It doesn't seem to say, anywhere. __________________ Abstraction doesn't make it impossible to write bad code; it makes it possible to write superior code. Contributor's Corner: Grumpy on C++ Exceptions DaWei on Pointers

Nov 17th, 2006, 2:34 AM	#4
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 5	(Toggle Plain Text) def find_all_items(site): site = urllib.urlopen(site).read() all_items = re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: all_items.extend(find_all_items(urllib.basejoin("<some url>",next_page[0]))) except IndexError: pass return remove_dups(all_items) # ,<---- this removes all the duplicate items def find_all_items(site): site = urllib.urlopen(site).read() all_items = re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: all_items.extend(find_all_items(urllib.basejoin("<some url>",next_page[0]))) except IndexError: pass return remove_dups(all_items) # ,<---- this removes all the duplicate items Also, you may want to think about using the set() data type, which contains only unique values and doesn't retain order. Perhaps: (Toggle Plain Text) def find_all_items(site): site = urllib.urlopen(site).read() all_items = set(re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site)) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: all_items.union(find_all_items(urllib.basejoin("<some url>",next_page[0]))) except IndexError: pass return all_items def find_all_items(site): site = urllib.urlopen(site).read() all_items = set(re.findall(r'watch\.asp\?\w+\=\w\&\w\=\w+',site)) next_page = re.findall(r'Watches\.asp\?\w+\=[0-9]+\&pg\=\w+',site) try: all_items.union(find_all_items(urllib.basejoin("<some url>",next_page[0]))) except IndexError: pass return all_items And I can't see a Python HTML parsing problem without mentioning Beautiful Soup. Though your problem might be simple enough that it needs only regexs.

Nov 17th, 2006, 3:20 AM	#6
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 5	Hm. It might help if you showed all your code, as it's difficult to understand exactly what you're asking for. Do you want some sort of persistence? Then you either need a functor: python Syntax (Toggle Plain Text) class FindAllItemsFunctor def __init__(self): self.all_items = [] def __call__(self, site): # Your find_all_items code using self.all_items instead of all_items find_all_items = FindAllItemsFunctor() class FindAllItemsFunctor def __init__(self): self.all_items = [] def __call__(self, site): # Your find_all_items code using self.all_items instead of all_items find_all_items = FindAllItemsFunctor() Or you need to use some functions within functions: python Syntax (Toggle Plain Text) def find_all_items_creater(): all_items = [] def find_all_items(site): # ... return find_all_items find_all_items = find_all_items_creater() def find_all_items_creater(): all_items = [] def find_all_items(site): # ... return find_all_items find_all_items = find_all_items_creater() Or you could use a global variable: python Syntax (Toggle Plain Text) all_items = [] def find_all_items(site): global all_items # ... all_items = [] def find_all_items(site): global all_items # ... Or even do something clever with the function's parameters: python Syntax (Toggle Plain Text) def find_all_items(site, all_items = []) # ... def find_all_items(site, all_items = []) # ...

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Nov 17th, 2006, 1:06 AM	#3
titaniumdecoy Expert Programmer Join Date: Nov 2005 Posts: 855 Rep Power: 3	It never fails to amaze me how so many of the newcomers to this forum have such unbelievably bad (written) communication skills.

Nov 17th, 2006, 3:01 AM	#5
nytrokiss Newbie Join Date: Oct 2006 Posts: 23 Rep Power: 0	I tried both the examples the issue is that the list all_items is cleared when the function is called again!

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
dev c++ software, template problem	cairo	C++	11	Jun 2nd, 2006 12:42 PM
libraries	matko	C	1	Jan 22nd, 2006 2:12 PM
Jackpot game	zorin	Visual Basic	3	Jun 10th, 2005 1:19 PM
User-defined creatNode and deleteNode functions for a doubly-linked list	jgs	C	2	Apr 28th, 2005 8:53 AM
airport Log program using 3D linked List : problem reading from file	gemini_shooter	C++	0	Mar 2nd, 2005 4:12 PM

Programming Forums > Script Programming > Python

Retaining the contents of a list during a recursive function