Python 四舍五入问题需要将每个数字四舍五入
Python rounding issue need to round half up every single digit
我不确定是否有人对以下场景有任何好主意:
import math
from decimal import *
def apply_precision_scale(value, precision, scale):
# supported formats:
# time: precision: 7, scale 4
# amplitude: precision: 7, scale: 5
getcontext().rounding = ROUND_HALF_UP
value_int = math.trunc(value)
rounded_input = None
# supports negative numbers
if precision == 7 and scale == 5:
if (value_int >= 0 and value_int <= 99) or \
(value_int >= -99 and value_int < 0):
places = scale
elif (value_int >= 100 and value_int <= 999) or \
(value_int <= -100 and value_int >= -999):
places = scale - 1
elif (value_int >= 1000 and value_int <= 9999) or \
(value_int >= -1000 and value_int >= -9999):
places = scale - 2
elif (value_int >= 10000 and value_int <= 99999) or \
(value_int >= -10000 and value_int >= -99999):
places = scale - 3
elif (value_int >= 100000 and value_int <= 999999) or \
(value_int >= -100000 and value_int >= -999999):
places = scale - 4
elif (value_int >= 1000000 and value_int <= 9999999) or \
(value_int >= -1000000 and value_int >= -9999999):
places = scale - 5
else:
print("Error: The value exceeds allowed precision")
return None
elif precision == 7 and scale == 4:
if (value_int >= 0 and value_int <= 999) or \
(value_int <= -100 and value_int >= -999):
places = scale
elif (value_int >= 1000 and value_int <= 9999) or \
(value_int >= -1000 and value_int >= -9999):
places = scale - 1
elif (value_int >= 10000 and value_int <= 99999) or \
(value_int >= -10000 and value_int >= -99999):
places = scale - 2
elif (value_int >= 100000 and value_int <= 999999) or \
(value_int >= -100000 and value_int >= -999999):
places = scale - 3
elif (value_int >= 1000000 and value_int <= 9999999) or \
(value_int >= -1000000 and value_int >= -9999999):
places = scale - 4
else:
print("Error: The value exceeds allowed precision")
return None
else:
print("Error: Unrecognized precision and scale format")
return None
return float(round(Decimal(value), places))
def verify_time_increments(voltages):
i = 2
failures = 0
keys = list(voltages)
increment = 0.001953125
increment_list = [(apply_precision_scale(increment * i, 7, 4)) for i in range(len(keys) + 1) if i != 0]
for index, time in enumerate(voltages):
try:
expected_time = increment_list[index]
i += 1
except IndexError: # mismatch between number of voltages and values in increment list
print(index)
print(len(increment_list))
数据采用以下格式并持续到时间数据点结束:
Voltages 是一个字典,格式如下 [0.002:-34.54, 0.0039:-35.65, etc]
出于某种原因,我在大数字上遇到这样的错误:
预期:64.0605
实际:64.0606
预期:64.123
实际:64.1231
在第一个例子中,我处理的值是 64.060546875,第二个例子是 64.123046785。我试图将它们缩减为 4 位有效数字,但将每个数字向上舍入,这意味着 46875 部分将四舍五入为 50000,因此 64.060546875 将四舍五入为 64.0606。
if anyone has any bright ideas
这个怎么样?从数学上讲,您实际上做错了事,试图将单个数字四舍五入到远远超过舍入点。
四舍五入只检查答案中您想要的最后一个数字之后的数字是有原因的。
想法是尽量减少误差,根据你想要的位数选择最接近的值。最接近64.060546875
的four-decimal-place值为64.0605
,如下图table:
Value: 64.050546875 64.050546875
Round: 64.0505 64.0506
Error: 0.000046875 0.000053125
因此 64.0505
四舍五入后的值更接近实际值。
我想没人明白我是在尝试 ROUND_UP 而不是 ROUND_HALF_UP。这会解决我的问题。
我不确定是否有人对以下场景有任何好主意:
import math
from decimal import *
def apply_precision_scale(value, precision, scale):
# supported formats:
# time: precision: 7, scale 4
# amplitude: precision: 7, scale: 5
getcontext().rounding = ROUND_HALF_UP
value_int = math.trunc(value)
rounded_input = None
# supports negative numbers
if precision == 7 and scale == 5:
if (value_int >= 0 and value_int <= 99) or \
(value_int >= -99 and value_int < 0):
places = scale
elif (value_int >= 100 and value_int <= 999) or \
(value_int <= -100 and value_int >= -999):
places = scale - 1
elif (value_int >= 1000 and value_int <= 9999) or \
(value_int >= -1000 and value_int >= -9999):
places = scale - 2
elif (value_int >= 10000 and value_int <= 99999) or \
(value_int >= -10000 and value_int >= -99999):
places = scale - 3
elif (value_int >= 100000 and value_int <= 999999) or \
(value_int >= -100000 and value_int >= -999999):
places = scale - 4
elif (value_int >= 1000000 and value_int <= 9999999) or \
(value_int >= -1000000 and value_int >= -9999999):
places = scale - 5
else:
print("Error: The value exceeds allowed precision")
return None
elif precision == 7 and scale == 4:
if (value_int >= 0 and value_int <= 999) or \
(value_int <= -100 and value_int >= -999):
places = scale
elif (value_int >= 1000 and value_int <= 9999) or \
(value_int >= -1000 and value_int >= -9999):
places = scale - 1
elif (value_int >= 10000 and value_int <= 99999) or \
(value_int >= -10000 and value_int >= -99999):
places = scale - 2
elif (value_int >= 100000 and value_int <= 999999) or \
(value_int >= -100000 and value_int >= -999999):
places = scale - 3
elif (value_int >= 1000000 and value_int <= 9999999) or \
(value_int >= -1000000 and value_int >= -9999999):
places = scale - 4
else:
print("Error: The value exceeds allowed precision")
return None
else:
print("Error: Unrecognized precision and scale format")
return None
return float(round(Decimal(value), places))
def verify_time_increments(voltages):
i = 2
failures = 0
keys = list(voltages)
increment = 0.001953125
increment_list = [(apply_precision_scale(increment * i, 7, 4)) for i in range(len(keys) + 1) if i != 0]
for index, time in enumerate(voltages):
try:
expected_time = increment_list[index]
i += 1
except IndexError: # mismatch between number of voltages and values in increment list
print(index)
print(len(increment_list))
数据采用以下格式并持续到时间数据点结束:
Voltages 是一个字典,格式如下 [0.002:-34.54, 0.0039:-35.65, etc]
出于某种原因,我在大数字上遇到这样的错误: 预期:64.0605 实际:64.0606 预期:64.123 实际:64.1231
在第一个例子中,我处理的值是 64.060546875,第二个例子是 64.123046785。我试图将它们缩减为 4 位有效数字,但将每个数字向上舍入,这意味着 46875 部分将四舍五入为 50000,因此 64.060546875 将四舍五入为 64.0606。
if anyone has any bright ideas
这个怎么样?从数学上讲,您实际上做错了事,试图将单个数字四舍五入到远远超过舍入点。
四舍五入只检查答案中您想要的最后一个数字之后的数字是有原因的。
想法是尽量减少误差,根据你想要的位数选择最接近的值。最接近64.060546875
的four-decimal-place值为64.0605
,如下图table:
Value: 64.050546875 64.050546875
Round: 64.0505 64.0506
Error: 0.000046875 0.000053125
因此 64.0505
四舍五入后的值更接近实际值。
我想没人明白我是在尝试 ROUND_UP 而不是 ROUND_HALF_UP。这会解决我的问题。