Monday, March 12, 2007

Web Scraping Using ASP.NET



Private Function ScrapeHTML(ByVal strURL As String) As String

Dim objWebRequest As System.Net.HttpWebRequest
Dim objWebResponse As System.Net.HttpWebResponse
Dim streamReader As System.IO.StreamReader

Try
objWebRequest = CType(System.Net.WebRequest.Create(strURL), System.Net.HttpWebRequest)
objWebRequest.Method = "GET"
objWebResponse = CType(objWebRequest.GetResponse(), System.Net.HttpWebResponse)
streamReader = New System.IO.StreamReader(objWebResponse.GetResponseStream)
strHTML = streamReader.ReadToEnd
ScrapeHTML = strHTML
Catch ex1 As Exception

ScrapeHTML = strHTML

End Try


End Function

The Return value of the scrapeHTML hold the HTML String output of the given URL.

Do further string comparison to make your own stuffs. :)

No comments: