From: "headius (Charles Nutter)" <headius@...>
Date: 2012-09-20T09:58:37+09:00
Subject: [ruby-core:47605] [ruby-trunk - Bug #7037] float formatting inconsistently rounds half to even


Issue #7037 has been updated by headius (Charles Nutter).


If we want to go by the letter of the spec (and I may have an older copy...please confirm), it says "if the underlying platform of a conforming processor supports IEC 60559:1989" the representation shall be that representation. I need to check the document for definitions of "platform" and "processor", but an argument can be made that our platform is the JVM...in which case, the JVM's representation of floating-point would be on with the spec.

The spec also says rounding due to arithmentic operations is implementation-defined.

The spec also says nothing about String#% or Kernel#sprintf.
----------------------------------------
Bug #7037: float formatting inconsistently rounds half to even
https://bugs.ruby-lang.org/issues/7037#change-29558

Author: headius (Charles Nutter)
Status: Closed
Priority: Normal
Assignee: 
Category: core
Target version: 
ruby -v: 2.0.0dev


MRI does not appear to consistently round half to even. I'm not sure what rounding strategy this is, but it rounds xx05 and xx15 to odd for xxx, and other values to even:

irb(main):001:0> "%1.1f" % 1.05
=> "1.1"
irb(main):002:0> "%1.1f" % 1.15
=> "1.1"
irb(main):003:0> "%1.1f" % 1.25
=> "1.2"
irb(main):004:0> "%1.1f" % 1.35
=> "1.4"
irb(main):005:0> "%1.1f" % 1.45
=> "1.4"
irb(main):006:0> "%1.1f" % 1.55
=> "1.6"

None of the tie-breaking strategies I could find (http://en.wikipedia.org/wiki/Rounding#Tie-breaking) seem to support MRI's model.

If MRI is indeed using "half even", xx05 should round to xx0 and xx15 should round to xx2. An example with Java's BigDecimal appears to support this:

irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.0"
irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.4"

We would like clarification about the proper rounding tie-breaker strategy to use so we can fix this JRuby issue properly: http://jira.codehaus.org/browse/JRUBY-6889


-- 
http://bugs.ruby-lang.org/