Visual studio 中的简单减法结果不正确
simple subtraction in Visual studio with incorrect result
像这样做一个简单的减法:
Dim a, b As Double
a = 81.4
b = 76.0
MsgBox(a - b)
你希望它的结果是 5.4,但你得到的却是 5.4000000000000001
这是怎么回事?
浮点数,即使是双精度浮点数,也不是 'real' 具有无限精度的数字。
见What Every Computer Scientist Should Know About Floating-Point Arithmetic
像这样做一个简单的减法:
Dim a, b As Double
a = 81.4
b = 76.0
MsgBox(a - b)
你希望它的结果是 5.4,但你得到的却是 5.4000000000000001 这是怎么回事?
浮点数,即使是双精度浮点数,也不是 'real' 具有无限精度的数字。
见What Every Computer Scientist Should Know About Floating-Point Arithmetic