Programming Forums

Programming Forums (http://www.programmingforums.org/forumindex.php)
-   Visual Basic .NET (http://www.programmingforums.org/forum19.html)
-   -   download images (incrementing url) ? (http://www.programmingforums.org/showthread.php?t=12388)

tristolas Jan 16th, 2007 1:09 AM

download images (incrementing url) ?
 
I'm not sure if this is something Visual Basic's for;
if not, would it be simplest with a scripting language or command prompt or what?

I want to download a bunch of images to harddrive from internet
all images have same URL exept the number increases
for example I would do something like
:

For X = 1 to 172
download http://www.blah.com/lala/imageX to C:\DOCUME~\User\Images\
Next


mrynit Jan 16th, 2007 1:49 AM

if you want a program that will do that you can get a crawler. this is one i have tried http://www.download.com/Advanced-Sit...-10155161.html

tristolas Jan 16th, 2007 3:45 PM

no programming language has the ability to go to a specified url & save it to computer?
The URLs are the URLs of the images; not of the pages that contain the images... is a web crawler necessary for this?

mrynit Jan 16th, 2007 6:32 PM

try scripitng with wget

lectricpharaoh Jan 16th, 2007 6:37 PM

Quote:

Originally Posted by tristolas (Post 122705)
no programming language has the ability to go to a specified url & save it to computer?
The URLs are the URLs of the images; not of the pages that contain the images... is a web crawler necessary for this?

All you need is the URL. You can use the System.Net.WebClient.DownloadData() method to fill a byte array with the file contents. You can then do whatever with this, including saving it to disk, manipulating the data, or whatever you want to do. If you need to do this without blocking on the current thread, you can use the DownloadDataAsync() method instead (don't call the blocking version in multiple threads simultaneously; it will throw an exception). Both links describe the methods; the first one also has code examples. I don't feel like typing a lengthy explanation of asynchronous I/O right now, but if you're familiar with callback functions (or whatever VB calls 'em), then it's pretty easy to use non-blocking I/O (though you really don't need it for this, especially if all the files are fetched from the same server).

tristolas Jan 17th, 2007 6:20 PM

For WebClient.DownloadFile Method, I don't know where to start; a console or windows application?
can I use this method within either one?

If I do windows form,
would I just replace Public Class Form1 with Public Class WebClient or do I have to edit the "meta-file" with
<ComVisibleAttribute(True)> _
Public Class WebClient
Inherits Component


Also the only way I know how to trigger the method(download file) is through a button declared as Private Sub Button1_Click()... but can I declare Public Sub DownloadFile() inside of this^ Private ButtonClick Sub ?

Then would I just stick the method inside of an incrementing loop ?
For example
:

Public Class Form1

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

        Dim adress As String = "http://www.contoso.com/library/homepage/images/"
        Dim number As Integer
        For number = 1 To 174
            Dim fileName As String = "ms-banner.gif" + number
            Dim fulladress As String = Nothing
            ' Create a new WebClient instance.
            Dim myWebClient As New Net.WebClient()
            ' Concatenate the domain with the Web resource filename. Because DownloadFile
            'requires a fully qualified resource name, concatenate the domain with the Web resource file name.
            fulladress = adress + fileName
            Console.WriteLine("Downloading File ""{0}"" from ""{1}"" ......." + ControlChars.Cr + ControlChars.Cr, fileName, fulladress)
            ' The DownloadFile() method downloads the Web resource and saves it into the current file-system folder.
            myWebClient.DownloadFile(fulladress, fileName)
            Console.WriteLine("Successfully Downloaded file ""{0}"" from ""{1}""", fileName, fulladress)
            Console.WriteLine((ControlChars.Cr + "Downloaded file saved in the following file system folder:" + ControlChars.Cr + ControlChars.Tab + Application.StartupPath))
        Next

    End Sub
End Class


Is doing it through console or wget script is simpler (requires less steps) ?

lectricpharaoh Jan 18th, 2007 1:59 AM

Quote:

Originally Posted by tristolas
For WebClient.DownloadFile Method, I don't know where to start; a console or windows application?
can I use this method within either one?

Should be able to, yeah. As for which is best, it depends on your needs. I suspect (from the fact the images are named with sequential strings) that it will function in a batch manner; as such, a console-mode program is just fine. If you want to prompt the user for the base URL or base filename (before the appended number), then a Windows forms app might be better, but you can certainly do this in a console program as well, whether through the use of command-line parameters, or input prompts. Likewise, a forms app can run without any input from the user (besides starting the app, of course).
Quote:

Originally Posted by tristolas
If I do windows form,
would I just replace Public Class Form1 with Public Class WebClient or do I have to edit the "meta-file" with
<ComVisibleAttribute(True)> _
Public Class WebClient
Inherits Component

You don't inherit from the class (well, not unless you're planning on changing its feature set). In this case, inheritance is unnecessary; all you need to do is instantiate a WebClient object, and call the appropriate method. All that other stuff is not needed, and will only serve to confuse matters.
Quote:

Originally Posted by tristolas
Also the only way I know how to trigger the method(download file) is through a button declared as Private Sub Button1_Click()... but can I declare Public Sub DownloadFile() inside of this^ Private ButtonClick Sub ?

If it's a forms app, and you want user input (say, web URL, or base filename), then doing it from a button click handler is fine (preferred, in fact). If you are hard-coding this, reading it from a configuration file, or otherwise determining it without user intervention, then you can do it in the form's load event handler (for a forms app) or in the main method (not sure what VB calls it, as I've not done console apps in VB, but it's probably main).
Quote:

Originally Posted by tristolas
Then would I just stick the method inside of an incrementing loop ?

Yup, much as you've done. All you really need to do in the loop is a) build the URL (URI) string, b) build the local filename string, and c) call the DownloadFile() method. I suggested DownloadData() because I figured you might want to do processing before saving, but if not, DownloadFile() is easier to use (especially since you don't need to determine the file size before starting the download).
Quote:

Originally Posted by tristolas
Is doing it through console or wget script is simpler (requires less steps) ?

Dunno about wget, as I've not used it, but via console, it's pretty simple. See my points above about console vs forms versions.

tristolas Jan 19th, 2007 6:40 PM

:

Dim url As String = "http://www.whatever.com/blah/"
Dim fileName As String = "ring" + times + ".jpg"

brought the error "Conversion from string "ring" to type 'Double' is not valid "
I didn't declare any double variables so don't know how this happens

I also tried converting the incrementing integer("times") to string, but didn't help
:

Module Module1

    Sub Main()
        Dim times As Integer
        For times = 1 To 164
            Dim url As String = "http://www.whatever.com/blah/"
            Dim numba As String = Convert.ToString(times)
            Dim fileName As String = "ring" + times + ".jpg"
            Dim full_path As String = Nothing
            ' Create a new WebClient instance.
            Dim myWebClient As New Net.WebClient()
            full_path = url + fileName

            myWebClient.DownloadFile(full_path, fileName)

            Console.WriteLine((ControlChars.Cr + "Downloaded file saved in" + ControlChars.Cr + ControlChars.Tab))
        Next
    End Sub

End Module


would turning option strict or ecplicit off help ?


thanks

Arevos Jan 19th, 2007 7:01 PM

Quote:

Originally Posted by tristolas (Post 122868)
I also tried converting the incrementing integer("times") to string, but didn't help

Uh, it looks like you're still using the integer "times" to me:
:

Dim times As Integer
        For times = 1 To 164
            Dim url As String = "http://www.whatever.com/blah/"
            Dim numba As String = Convert.ToString(times)
            Dim fileName As String = "ring" + times + ".jpg"

Shouldn't this be:
:

Dim fileName As String = "ring" + numba + ".jpg"

tristolas Jan 20th, 2007 2:57 PM

yeah thanks for pointing out hehe

I just don't know how to execute it ("debug" is disabled),
I did it as a console app (just a *.vb file, nothing else)


All times are GMT -5. The time now is 12:49 AM.

Powered by vBulletin® Version 3.7.0, Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Copyright ©2007 DaniWeb® LLC