I developed an application to download multiple images from a website with a pattern, using a WebClient. When I started with this, my idea was a fast and easy app, so I made a form with:
- WebBrowser (WB)
- Multiline textbox to put multiple urls to crawl
- Listbox to storage the link of each photo
- Textbox to select a targetpath
- Button to start the process
It was easy, I just needed to find the labels of each photos using For + HtmlElement + WB.Document.All. Once I got all links, I used a WebClient to download them very fast (Client.DownloadFile(path, filename)). It works flawlessly.
Now the app is bigger, it has more websites to crawl and I would like to use BackgroundWorker to manage this. I started to learn how BGWorker works, and I noticed that I cannot use a Control like WebBrowser, it gives an exception, so I cannot use WB.Document.All to get the links.
I found an amazing example in Codeproject of how to download a file with BGWorker, this one concretely:
http://www.codeproject.com/Articles/17979/Downloading-Files-in-NET-With-All-Information-Prog
He uses HttpWebRequest, which seems harder to implement if I first need to get the links.
My question is: Should I recode the app to use HttpWebRequest or can I adapt the current code (WebBrowser implicated) to use a BGWorker? If it's possible to use WebBrowser with BGWorker, which one is the correct way to make it?
My current relevant code (I call this function when I click the button). I put the necessary lines only:
Function ExtractLicensePlate() Dim Client As New WebClient Dim totalPhotos As Integer = 0 Dim finalPlate As String = "[no licenseplate found]" Dim totalCarsList As Integer = txtURLs.Lines.Where(Function(l) Not String.IsNullOrWhiteSpace(l)).Count() For i = 0 To totalCarsList - 1 WB.Navigate(txtURLs.Lines(i)) WaitForPageLoad() For Each ele As HtmlElement In WB.Document.All 'ALD If comboProviders.SelectedIndex = 0 Then If ele.GetAttribute("src").ToLower.Contains("iddoc") Then Dim imgsrc As String = ele.GetAttribute("src") lstPhotos.Items.Add(imgsrc) End If 'get license plate to make folders If ele.GetAttribute("action").ToLower.Contains("matr=") Then Dim fullURLname As String = ele.GetAttribute("action") finalPlate = fullURLname.Split("=")(3).Substring(0, 7) End If End If Next If (Not Directory.Exists(txtDirectory.Text & "" & finalPlate)) Then If Not (finalPlate = "[no licenseplate found]") Then Directory.CreateDirectory(txtDirectory.Text & "" & finalPlate) End If End If For j = 0 To lstPhotos.Items.Count - 1 Client.DownloadFile(lstPhotos.Items(j).ToString, txtDirectory.Text & "" & finalPlate & "" & finalPlate & j & ".jpg") Client.Dispose() Next j lstPhotos.Items.Clear() Next i Return 0
End FunctionAny help will be very appreciate.
Thanks to everyone.