Programming Forums > Script Programming > Python

titaniumdecoy · Jun 27th, 2006, 2:29 PM

I wrote a method which is supposed to search a directory (and subdirectories up to 'recurl' levels) for the first instance of a file named 'name', and return the full path to that file.

(Toggle Plain Text)

def findfileswithname(path, name, recurl=4, i=0):
    if not os.path.isdir(path): return
    for item in os.listdir(path):
        itempath = os.path.join(path, item)
        if os.path.isdir(itempath) and i < recurl:
            findfileswithname(itempath, name, recurl, i+1)
        elif item == name:
            print 'found it %s' % os.path.join(path, item),
            return os.path.join(path, item)
    print '.',

print findfileswithname('T:\Test', 'schedule.xls', recurl=2)

Quote:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . found it T:\Test\Folder\schedule.xls . . . . . . . . . . . . . . . . . . . . . . . None

After the method finds the file and returns it, it keeps going, eventually returning none. Is there a way to 'break out' of the method's recursive calls when the file is located? (The last thing printed, "None", is the return value of the call to the function.)

titaniumdecoy · Jun 27th, 2006, 5:00 PM

I managed to sidestep this problem by using os.walk:

(Toggle Plain Text)

filename = 'schedule.xls'
searchdir = 'T:\Test'

for root, dirs, files in os.walk(searchdir):
    if filename in files:
        result = root + name
        break

print result

I am still interested in how the code in my last post could be fixed. Also, is there a way to specify the maximum number of subdirectories os.walk will look inside?

Arevos · Jun 27th, 2006, 5:06 PM

(Toggle Plain Text)

def findfileswithname(path, name, recurl=4, i=0):
    if i >= recurl: return
    for item in os.listdir(path):
        itempath = os.path.join(path, item)
        if os.path.isdir(itempath):
            found = findfileswithname(itempath, name, recurl, i+1)
            if found: return found
        elif item == name:
            print 'found it %s' % os.path.join(path, item),
            return os.path.join(path, item)
    print '.',

The code in red demonstrates how to return the file. If the file has been found by the recursive subfunction, it returns the result. Otherwise, it carries on checking the other directories.

The code in blue I added/changed to fix a small bug (or what I suspect is a bug). The original if statement read, "if itempath is a directory, and the recusive limit has not been met, then do this". This means that the elif would have read: "if itempath is not a directory, or if it is a directory and the recursive limit was not met, and the item is the item we're searching for, then do this."

Thus, if the recursive limit was met, then the function would return directories as well as files. The blue code should fix this up.

Also, I removed this line of code:

(Toggle Plain Text)

if not os.path.isdir(path): return

You should not ignore invalid input like this. Instead, raise an exception. Fortunately, you don't have to add any extra code to do this, since os.listdir will raise the exception for you.

titaniumdecoy · Jun 27th, 2006, 5:38 PM

Thanks, Arevos. I'm using the code you fixed rather than os.walk() because I can't specify a maximum recursive limit, as far as I know.

titaniumdecoy · Jun 28th, 2006, 11:37 AM

I have another question. I based the function in my first post on the code for another similar function I wrote, which returns a list of all files with an appropriate extension. However, in this function I don't return the list of files until the very end. Why does this work here when it doesn't in the function I posted previously? Or have I done something wrong here as well?

(Toggle Plain Text)

def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]):
    """Return a list of filepaths to files with an extension in exts
    Recurl is the level of recursion, or depth of subdirectories to search"""
    if not os.path.isdir(path): return
    for item in os.listdir(path):
        itempath = os.path.join(path, item)
        if os.path.isdir(itempath) and i < recurl:
            findfileswithext(itempath, exts, recurl, i+1)
        elif os.path.splitext(item)[1] in exts:
            files.append(itempath)
    return files

Arevos · Jun 28th, 2006, 12:26 PM

It works, because the "files" list is constant throughout the recursion. Take a look at this line:

(Toggle Plain Text)

def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]):

The highlighted code assigns an empty array to the "files" argument. It does only once, when the function is defined. Each time it is called, it uses the same list object.

I'll give you a quick example of this:

(Toggle Plain Text)

>>> def foobar(x = []):
...     print x
...     x.append(len(x))
...
>>> foobar()
[]
>>> foobar()
[0]
>>> foobar()
[0, 1]
>>> foobar()
[0, 1, 2]

This works because lists are mutable; the data contained within them can be altered. Thus, with your original function, the matching files are all contained in a single array, referenced by the "files" argument.

It usually isn't a good idea to use this functionality, however, as it's somewhat confusing.

Arevos · Jun 28th, 2006, 2:19 PM

As an aside, there's a lot of common functionality in your two functions that can be abstracted out. If there was a function similar to os.walk, but with a recursive limit, then your functions would be a lot simpler.

Maybe something like this:

(Toggle Plain Text)

def limited_walk(path, limit, n = 0):
    if n > limit: return
                    
    for file in os.listdir(path):
        file_path = os.path.join(path, file)
            
        if os.path.isdir(file_path):
            for item in limited_walk(file_path, limit, n + 1):
                yield item
        else:
            yield file_path

# Example of use:
for filename in limited_walk("some_directory", 4):
    if os.path.basename(filename) == name:
        print filename

Jun 27th, 2006, 5:00 PM	#2
titaniumdecoy Expert Programmer Join Date: Nov 2005 Posts: 837 Rep Power: 3	I managed to sidestep this problem by using os.walk: (Toggle Plain Text) filename = 'schedule.xls' searchdir = 'T:\Test' for root, dirs, files in os.walk(searchdir): if filename in files: result = root + name break print result filename = 'schedule.xls' searchdir = 'T:\Test' for root, dirs, files in os.walk(searchdir): if filename in files: result = root + name break print result I am still interested in how the code in my last post could be fixed. Also, is there a way to specify the maximum number of subdirectories os.walk will look inside?

Jun 27th, 2006, 5:06 PM	#3
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 4	(Toggle Plain Text) def findfileswithname(path, name, recurl=4, i=0): if i >= recurl: return for item in os.listdir(path): itempath = os.path.join(path, item) if os.path.isdir(itempath): found = findfileswithname(itempath, name, recurl, i+1) if found: return found elif item == name: print 'found it %s' % os.path.join(path, item), return os.path.join(path, item) print '.', def findfileswithname(path, name, recurl=4, i=0): if i >= recurl: return for item in os.listdir(path): itempath = os.path.join(path, item) if os.path.isdir(itempath): found = findfileswithname(itempath, name, recurl, i+1) if found: return found elif item == name: print 'found it %s' % os.path.join(path, item), return os.path.join(path, item) print '.', The code in red demonstrates how to return the file. If the file has been found by the recursive subfunction, it returns the result. Otherwise, it carries on checking the other directories. The code in blue I added/changed to fix a small bug (or what I suspect is a bug). The original if statement read, "if itempath is a directory, and the recusive limit has not been met, then do this". This means that the elif would have read: "if itempath is not a directory, or if it is a directory and the recursive limit was not met, and the item is the item we're searching for, then do this." Thus, if the recursive limit was met, then the function would return directories as well as files. The blue code should fix this up. Also, I removed this line of code: (Toggle Plain Text) if not os.path.isdir(path): return if not os.path.isdir(path): return You should not ignore invalid input like this. Instead, raise an exception. Fortunately, you don't have to add any extra code to do this, since os.listdir will raise the exception for you.

Jun 28th, 2006, 11:37 AM	#5
titaniumdecoy Expert Programmer Join Date: Nov 2005 Posts: 837 Rep Power: 3	I have another question. I based the function in my first post on the code for another similar function I wrote, which returns a list of all files with an appropriate extension. However, in this function I don't return the list of files until the very end. Why does this work here when it doesn't in the function I posted previously? Or have I done something wrong here as well? (Toggle Plain Text) def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]): """Return a list of filepaths to files with an extension in exts Recurl is the level of recursion, or depth of subdirectories to search""" if not os.path.isdir(path): return for item in os.listdir(path): itempath = os.path.join(path, item) if os.path.isdir(itempath) and i < recurl: findfileswithext(itempath, exts, recurl, i+1) elif os.path.splitext(item)[1] in exts: files.append(itempath) return files def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]): """Return a list of filepaths to files with an extension in exts Recurl is the level of recursion, or depth of subdirectories to search""" if not os.path.isdir(path): return for item in os.listdir(path): itempath = os.path.join(path, item) if os.path.isdir(itempath) and i < recurl: findfileswithext(itempath, exts, recurl, i+1) elif os.path.splitext(item)[1] in exts: files.append(itempath) return files

Jun 28th, 2006, 12:26 PM	#6
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 4	It works, because the "files" list is constant throughout the recursion. Take a look at this line: (Toggle Plain Text) def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]): def findfileswithext(path, exts=NC_EXTENSIONS, recurl=3, i=0, files=[]): The highlighted code assigns an empty array to the "files" argument. It does only once, when the function is defined. Each time it is called, it uses the same list object. I'll give you a quick example of this: (Toggle Plain Text) >>> def foobar(x = []): ... print x ... x.append(len(x)) ... >>> foobar() [] >>> foobar() [0] >>> foobar() [0, 1] >>> foobar() [0, 1, 2] >>> def foobar(x = []): ... print x ... x.append(len(x)) ... >>> foobar() [] >>> foobar() [0] >>> foobar() [0, 1] >>> foobar() [0, 1, 2] This works because lists are mutable; the data contained within them can be altered. Thus, with your original function, the matching files are all contained in a single array, referenced by the "files" argument. It usually isn't a good idea to use this functionality, however, as it's somewhat confusing.

Jun 28th, 2006, 2:19 PM	#7
Arevos Programming Guru Join Date: Aug 2005 Location: England Posts: 1,499 Rep Power: 4	As an aside, there's a lot of common functionality in your two functions that can be abstracted out. If there was a function similar to os.walk, but with a recursive limit, then your functions would be a lot simpler. Maybe something like this: (Toggle Plain Text) def limited_walk(path, limit, n = 0): if n > limit: return for file in os.listdir(path): file_path = os.path.join(path, file) if os.path.isdir(file_path): for item in limited_walk(file_path, limit, n + 1): yield item else: yield file_path # Example of use: for filename in limited_walk("some_directory", 4): if os.path.basename(filename) == name: print filename def limited_walk(path, limit, n = 0): if n > limit: return for file in os.listdir(path): file_path = os.path.join(path, file) if os.path.isdir(file_path): for item in limited_walk(file_path, limit, n + 1): yield item else: yield file_path # Example of use: for filename in limited_walk("some_directory", 4): if os.path.basename(filename) == name: print filename

Jun 27th, 2006, 5:38 PM	#4
titaniumdecoy Expert Programmer Join Date: Nov 2005 Posts: 837 Rep Power: 3	Thanks, Arevos. I'm using the code you fixed rather than os.walk() because I can't specify a maximum recursive limit, as far as I know.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Programming Forums > Script Programming > Python

Method not returning?