c# - How to view the headers sent by HttpWebRequest

Question

I'm scraping a site with HttpWebRequest, but the site is returning an error. The page works fine when I hit it from my browser. I'd like to compare them to see what may be causing the error. I know how to intercept the request from my browser to inspect the headers, but how do I view the data sent by HttpWebRequest?

Answer 1

In order to compare what you do in code, and what the browser does, I am sure a HTTP debugging tool such as Fiddler, would be the easiest solution.

Fiddler acts as a proxy between client and server, and displays all information sent over the HTTP protocol.

It is possible that you will need to configure your .NET app to use the proxy that Fiddler provides. This blog post provides details on the subject.