Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

I have come across a precision issue with double in .NET I thought this only applied to floats but now I see that double is a float.

double test = 278.97 - 90.46;
Debug.WriteLine(test) //188.51000000000005

//correct answer is 188.51

What is the correct way to handle this? Round? Lop off the unneeded decimal places?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
323 views
Welcome To Ask or Share your Answers For Others

1 Answer

Use the decimal data type. "The Decimal value type is appropriate for financial calculations requiring large numbers of significant integral and fractional digits and no round-off errors."


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
...