Is there a way to create a specific web Crawler with VBA

%3CLINGO-SUB%20id%3D%22lingo-sub-2127907%22%20slang%3D%22en-US%22%3EIs%20there%20a%20way%20to%20create%20a%20specific%20web%20Crawler%20with%20VBA%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2127907%22%20slang%3D%22en-US%22%3E%3CP%20class%3D%22_1qeIAgB0cPwnLhDF9XSiJM%22%3EHello.%3C%2FP%3E%3CP%20class%3D%22_1qeIAgB0cPwnLhDF9XSiJM%22%3EI'm%20trying%20to%20develop%20a%20web%20crawler%20with%20VBA%20which%20helps%20me%20download%20data%20from%20the%20specific%20webpage%20%2C%20what%20i%20want%20to%20do%20is%20make%20in%20VBA%20to%20upload%20a%20excel%20file%20with%20product%20numbers%20and%20then%20to%20crawl%20%2C%20recently%20i%20developed%20a%20very%20easy%20Crawler%20which%20is%20not%20a%20lot%20specific%20i%20have%20the%20code%20%2C%20and%20if%20someone%20could%20change%20it%20and%20make%20it%20for%20specific%20crawler.%3C%2FP%3E%3CP%20class%3D%22_1qeIAgB0cPwnLhDF9XSiJM%22%3EI%20want%20from%20this%20page%20%3A%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fwww.digikey.com%2Fen%2Fproducts%2Fdetail%2F3m-tc%2F3M-1776-12-X-12-6-PK%2F12144741%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20ugc%20noreferrer%22%3Ehttps%3A%2F%2Fwww.digikey.com%2Fen%2Fproducts%2Fdetail%2F3m-tc%2F3M-1776-12-X-12-6-PK%2F12144741%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3Eto%20crawl%20only%20the%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3EProduct%20Attributes.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22_1qeIAgB0cPwnLhDF9XSiJM%22%3E%3CSTRONG%3EBelow%20the%20Photo%20is%20the%20Code.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22crawl2.PNG%22%20style%3D%22width%3A%20641px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F254454iD0FA83D591E883FC%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22crawl2.PNG%22%20alt%3D%22crawl2.PNG%22%20%2F%3E%3C%2FSPAN%3E%3CBR%20%2F%3E%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22_1qeIAgB0cPwnLhDF9XSiJM%22%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3ESub%20CountryPopList()%0A%0ADim%20ieObj%20As%20InternetExplorer%0A%0ADim%20htmlEle%20As%20IHTMLElement%0A%0ADim%20i%20As%20Integer%0A%0Ai%20%3D%201%0A%0ASet%20ieObj%20%3D%20New%20InternetExplorer%0A%0AieObj.Visible%20%3D%20True%0A%0AieObj.navigate%20%22https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FList_of_countries_and_dependencies_by_population%22%0A%0AApplication.Wait%20Now%20%2B%20TimeValue(%2200%3A00%3A05%22)%0A%0AFor%20Each%20htmlEle%20In%20ieObj.document.getelementbyclassname(%22wikitable%22)(0).getelementsbytagename(%22tr%22)%0A%0AWith%20ActiveSheet%0A%0A.Range(%22A%22%20%26amp%3B%20i).Value%20%3D%20htmlEle.Children(0).textContent%0A%0A.Range(%22B%22%20%26amp%3B%20i).Value%20%3D%20htmlEle.Children(1).textContent%0A%0A.Range(%22C%22%20%26amp%3B%20i).Value%20%3D%20htmlEle.Children(2).textContent%0A%0A.Range(%22D%22%20%26amp%3B%20i).Value%20%3D%20htmlEle.Children(3).textContent%0A%0A.Range(%22E%22%20%26amp%3B%20i).Value%20%3D%20htmlEle.Children(4).textContent%0A%0AEnd%20With%0A%0Ai%20%3D%20i%20%2B%201%0A%0ANext%20htmlEle%0A%0AEnd%20Sub%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2127907%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EExcel%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2127938%22%20slang%3D%22en-US%22%3ERe%3A%20Is%20there%20a%20way%20to%20create%20a%20specific%20web%20Crawler%20with%20VBA%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2127938%22%20slang%3D%22en-US%22%3E%3CP%3E%40T%3CSPAN%3Ehese%20are%20the%20product%20codes%20off%20manufacturer%20%2C%20and%20those%20codes%20i%20will%20add%20in%20list%20that%20i%20upload%20%2C%20and%20then%20VBA%20to%20crawl.%3C%2FSPAN%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%3CDIV%20class%3D%22bbTable%22%3E%3CTABLE%3E%3CTBODY%3E%3CTR%3E%3CTD%3E771-9975%2F006-102%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E894-8992%2F023-307%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-8985%2F006-303%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E774-9993%2F007-302%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E891-8982%2F016-601%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E891-8992%2F015-502%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9973%2F017-201%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9973%2F017-302%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E774-9994%2F016-701%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9994%2F016-601%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-8982%2F017-403%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9994%2F005-505%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E891-8992%2F006-207%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9994%2F005-605%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9995%2F006-501%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-8985%2F006-404%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9994%2F006-207%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-6993%2F006-702%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-9995%2F006-803%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E891-8992%2F006-602%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E891-8993%2F006-705%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-8985%2F007-503%3C%2FTD%3E%3C%2FTR%3E%3CTR%3E%3CTD%3E771-8992%2F007-101%3C%2FTD%3E%3C%2FTR%3E%3C%2FTBODY%3E%3C%2FTABLE%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hello.

I'm trying to develop a web crawler with VBA which helps me download data from the specific webpage , what i want to do is make in VBA to upload a excel file with product numbers and then to crawl , recently i developed a very easy Crawler which is not a lot specific i have the code , and if someone could change it and make it for specific crawler.

I want from this page : https://www.digikey.com/en/products/detail/3m-tc/3M-1776-12-X-12-6-PK/12144741 to crawl only the Product Attributes.

Below the Photo is the Code.
crawl2.PNG

 

 

 

Sub CountryPopList()

Dim ieObj As InternetExplorer

Dim htmlEle As IHTMLElement

Dim i As Integer

i = 1

Set ieObj = New InternetExplorer

ieObj.Visible = True

ieObj.navigate "https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population"

Application.Wait Now + TimeValue("00:00:05")

For Each htmlEle In ieObj.document.getelementbyclassname("wikitable")(0).getelementsbytagename("tr")

With ActiveSheet

.Range("A" & i).Value = htmlEle.Children(0).textContent

.Range("B" & i).Value = htmlEle.Children(1).textContent

.Range("C" & i).Value = htmlEle.Children(2).textContent

.Range("D" & i).Value = htmlEle.Children(3).textContent

.Range("E" & i).Value = htmlEle.Children(4).textContent

End With

i = i + 1

Next htmlEle

End Sub

 

 

1 Reply

@These are the product codes off manufacturer , and those codes i will add in list that i upload , and then VBA to crawl.




771-9975/006-102
894-8992/023-307
771-8985/006-303
774-9993/007-302
891-8982/016-601
891-8992/015-502
771-9973/017-201
771-9973/017-302
774-9994/016-701
771-9994/016-601
771-8982/017-403
771-9994/005-505
891-8992/006-207
771-9994/005-605
771-9995/006-501
771-8985/006-404
771-9994/006-207
771-6993/006-702
771-9995/006-803
891-8992/006-602
891-8993/006-705
771-8985/007-503
771-8992/007-101