c# - HttpWebRequest.GetResponse() hangs the second time it is called

Question

I'm trying to fetch a series of files via HTTP, using HttpWebRequest. The first request goes through fine, but the second time through the same code GetResponse() hangs and times out. WireShark shows that no HTTP traffic is being sent for the second request, so it would appear that it's an API problem.

After some investigation, I've discovered that it has to do with specifying the content-length: if I leave this out, then the code works fine.

My code is:

HttpWebRequest  httpWebRequest = ConfigureRequest();

using (WebResponse webResponse = httpWebRequest.GetResponse())
    // On the second iteration we never get beyond this line
{
    HttpWebResponse httpWebResponse = webResponse as HttpWebResponse;

    using (Stream webResponseStream = httpWebResponse.GetResponseStream())
    {
        if (webResponseStream != null)
        {
            // Read the stream
        }
    }

    statusCode = httpWebResponse.StatusCode;
    httpWebResponse.Close();
}

The symptoms seem very similar to this question and this question, but in both cases the advice given is to dispose of the WebResponse, which I'm already doing.

Edit In response to Gregory, here is ConfigureRequest():

private HttpWebRequest ConfigureRequest()
{
    string          sUrl            = CreateURL(bucket, key);
    HttpWebRequest  httpWebRequest  = WebRequest.Create(sUrl) as HttpWebRequest;

    httpWebRequest.AllowWriteStreamBuffering = false;
    httpWebRequest.AllowAutoRedirect = true;
    httpWebRequest.UserAgent = this.m_sUserAgent;
    httpWebRequest.Method = "GET";
    httpWebRequest.Timeout = this.m_iTimeout;

    // *** NB: This line was left out of my original posting, and turned out to be
    // crucial
    if (m_contentLength > 0)
        httpWebRequest.ContentLength = m_contentLength;

    httpWebRequest.Headers.Add(StaticValues.Amazon_AlternativeDateHeader, timestamp);
    httpWebRequest.Headers.Add(StaticValues.HttpRequestHeader_Authorization, StaticValues.Amazon_AWS + " " + aWSAccessKeyId + ":" + signature);

    return httpWebRequest;
}

Edit: It turns out I committed the cardinal sin of removing code from my question that I hadn't verified was unrelated to the problem. I had removed the following lines:

    if (m_contentLength > 0)
        httpWebRequest.ContentLength = m_contentLength;

because I thought that content-length would never be specified for a GET request. It turns out I was wrong. Removing this line fixes the problem.

The only question I now have is why? I think the content length that is specified is correct, though it's possible it's off by one. Would specifying a content length that is too short prevent the full download from taking place and cause the connection to be left open? I would have expected that Close() and / or Dispose() ought to kill the connection anyway.

Answer 1

httpWebRequest.Abort(); // before you leave

Will solve!

Take a look at the resource.