[DOC] Fix granularity calculation
The granularity is calculated as 10 ** ndigits.abs rather than ndigits.abs * 10. For example, if ndigits is -2, it should be 10 ** 2 == 100 rather than 2 * 10 == 20.
10 ** ndigits.abs
ndigits.abs * 10
ndigits
-2
10 ** 2 == 100
2 * 10 == 20
Added by
[DOC] Fix granularity calculation
The granularity is calculated as
10 ** ndigits.absrather thanndigits.abs * 10. For example, ifndigitsis-2, it should be10 ** 2 == 100rather than2 * 10 == 20.