c# - Json.Net deserialize out of memory issue

Question

I got a Json, which contains among others a data field which stores a base64 encoded string. This Json is serialized and send to a client.

On client side, the newtonsoft json.net deserializer is used to get back the Json. However, if the data field becomes large (~ 400 MB), the deserializer will throw an out of memory exception: Array Dimensions exceeded supported Range. I also see in Task-Manager, that memory consumption really grows fast.

Any ideas why this is? Is there a maximum size for json fields or something?

Code example (simplified):

HttpResponseMessage responseTemp = null;
responseTemp = client.PostAsJsonAsync(client.BaseAddress, message).Result;

string jsonContent = responseTemp.Content.ReadAsStringAsync.Result;
result = JsonConvert.DeserializeObject<Result>(jsonContent);

Result class:

public class Result
{

    public string Message { get; set; }
    public byte[] Data { get; set; }

}

UPDATE:

I think my problem is not the serializer, but just trying to handle such a huge string in memory. At the point where I read the string into memory, the memory consumption of the application explodes. Every operation on that string does the same. At the moment, I think I have to find a way to work with streams and stop reading the whole stuff into memory at once.

Answer 1

To read large JSON string with use of JsonConvert.DeserializeObject will consume your lots of memory. So One of the ways to over come from this issue, you can create an instance of JsonSerializer as given below.

 using (StreamReader r = new StreamReader(filePath))
 {
          using (JsonReader reader = new JsonTextReader(r))
         {
                JsonSerializer serializer = new JsonSerializer();
                T lstObjects = serializer.Deserialize<T>(reader);
        }
}

Here filePath :- is your current Json file and T :- is your Generic type object.