I developed an application to download multiple images from a website with a pattern, using a WebClient. When I started with this, my idea was a fast and easy app, so I made a form with:
- WebBrowser (WB)
- Multiline textbox to put multiple urls to crawl
- Listbox to storage the link of each photo
- Textbox to select a targetpath
- Button to start the process
It was easy, I just needed to find the labels of each photos using For + HtmlElement + WB.Document.All. Once I got all links, I used a WebClient to download them very fast (Client.DownloadFile(path, filename)). It works flawlessly.
Now the app is bigger, it has more websites to crawl and I would like to use BackgroundWorker to manage this. I started to learn how BGWorker works, and I noticed that I caot use a Control like WebBrowser, it gives an exception, so I caot use WB.Document.All to get the links.
I found an amazing example in Codeproject of how to download a file with BGWorker, this one concretely:
http://www.codeproject.com/Articles/17979/Downloading-Files-in-NET-With-All-Information-Prog
He uses HttpWebRequest, which seems harder to implement if I first need to get the links.
My question is: Should I recode the app to use HttpWebRequest or can I adapt the current code (WebBrowser implicated) to use a BGWorker? If it's possible to use WebBrowser with BGWorker, which one is the correct way to make it?
My current relevant code (I call this function when I click the button). I put the necessary lines only:
Function ExtractLicensePlate()
Dim Client As New WebClient
Dim totalPhotos As Integer = 0
Dim finalPlate As String = "[no licenseplate found]"
Dim totalCarsList As Integer = txtURLs.Lines.Where(Function(l) Not String.IsNullOrWhiteSpace(l)).Count()
For i = 0 To totalCarsList - 1
WB.Navigate(txtURLs.Lines(i))
WaitForPageLoad()
For Each ele As HtmlElement In WB.Document.All
'ALD
If comboProviders.SelectedIndex = 0 Then
If ele.GetAttribute("src").ToLower.Contains("iddoc") Then
Dim imgsrc As String = ele.GetAttribute("src")
lstPhotos.Items.Add(imgsrc)
End If
'get license plate to make folders
If ele.GetAttribute("action").ToLower.Contains("matr=") Then
Dim fullURLname As String = ele.GetAttribute("action")
finalPlate = fullURLname.Split("=")(3).Substring(0, 7)
End If
End If
Next
If (Not Directory.Exists(txtDirectory.Text & "" & finalPlate)) Then
If Not (finalPlate = "[no licenseplate found]") Then
Directory.CreateDirectory(txtDirectory.Text & "" & finalPlate)
End If
End If
For j = 0 To lstPhotos.Items.Count - 1
Client.DownloadFile(lstPhotos.Items(j).ToString, txtDirectory.Text & "" & finalPlate & "" & finalPlate & j & ".jpg")
Client.Dispose()
Next j
lstPhotos.Items.Clear()
Next i
Return 0
End Function
Any help will be very appreciate.
Thanks to everyone.
