Screen scraping in C# : CSharp Feeds

Screen scraping in C#

by post@madskristensen.dk (Mads Kristensen) via .NET Slave on 2/13/2007 5:14:25 PM

Some say that screen scraping is a lost art because it is no longer an advanced discipline. That may be right, but there are different ways of doing it. Here are some different ways that all are perfectly acceptable, but can be used for various different purposes.

Old school

It’s old school because this approach has existed since .NET 1.0. It is highly flexible and lets you make the request asynchronously.

public static string ScreenScrape(string url)
{
System.Net.WebRequest request = System.Net.WebRequest.Create(url);
// set properties of the request
using (System.Net.WebResponse response = request.GetResponse())
{
using (System.IO.StreamReader reader =new System.IO.StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
}

Modern

In .NET 2.0 we can use the WebClient class, which is a cleaner way of solving the same problem. It is equally as flexible and can also work asynchronous.

publicstaticstring ScreenScrape(string url)
{
using (System.Net.WebClient client =new System.Net.WebClient())
{
// set properties of the client
return client.DownloadString(url);
}
}

The one-liner

This is a short version of the Modern approach, but it deserves to be on the list because it is a one-liner. Tell a nineteen ninetees developer that you can do screen scraping in one line of code and he wont believe you. The approach is not flexible in any way and cannot be used asynchronously.

publicstaticstring ScreenScrape(string url)
{
returnnew System.Net.WebClient().DownloadString(url);
}

That concludes the medley of screen scraping approaches. Pick the one you find best for the given situation.