Is there a way to create a specific web Crawler with VBA

Copper Contributor

Hello.

I'm trying to develop a web crawler with VBA which helps me download data from the specific webpage , what i want to do is make in VBA to upload a excel file with product numbers and then to crawl , recently i developed a very easy Crawler which is not a lot specific i have the code , and if someone could change it and make it for specific crawler.

I want from this page : https://www.digikey.com/en/products/detail/3m-tc/3M-1776-12-X-12-6-PK/12144741 to crawl only the Product Attributes.

Below the Photo is the Code.
crawl2.PNG

 

 

 

Sub CountryPopList()

Dim ieObj As InternetExplorer

Dim htmlEle As IHTMLElement

Dim i As Integer

i = 1

Set ieObj = New InternetExplorer

ieObj.Visible = True

ieObj.navigate "https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population"

Application.Wait Now + TimeValue("00:00:05")

For Each htmlEle In ieObj.document.getelementbyclassname("wikitable")(0).getelementsbytagename("tr")

With ActiveSheet

.Range("A" & i).Value = htmlEle.Children(0).textContent

.Range("B" & i).Value = htmlEle.Children(1).textContent

.Range("C" & i).Value = htmlEle.Children(2).textContent

.Range("D" & i).Value = htmlEle.Children(3).textContent

.Range("E" & i).Value = htmlEle.Children(4).textContent

End With

i = i + 1

Next htmlEle

End Sub

 

 

1 Reply

@These are the product codes off manufacturer , and those codes i will add in list that i upload , and then VBA to crawl.




771-9975/006-102
894-8992/023-307
771-8985/006-303
774-9993/007-302
891-8982/016-601
891-8992/015-502
771-9973/017-201
771-9973/017-302
774-9994/016-701
771-9994/016-601
771-8982/017-403
771-9994/005-505
891-8992/006-207
771-9994/005-605
771-9995/006-501
771-8985/006-404
771-9994/006-207
771-6993/006-702
771-9995/006-803
891-8992/006-602
891-8993/006-705
771-8985/007-503
771-8992/007-101