Ruby on Rails | Screencasts | Download | Documentation | Weblog | Community | Source

Ticket #8275 (closed defect: fixed)

Opened 2 years ago

Last modified 8 months ago

[PATCH] test_number_with_precision fails

Reported by: lau Assigned to: core
Priority: high Milestone: 2.x
Component: ActiveRecord Version: edge
Severity: major Keywords: test
Cc:

Description

Certain numbers are not correctly rounded in number_with_precision. For instance: 31.125 with precision 2 should be 31.13, but number_with_precision returns 31.12.

These test are added and fail on a system with ruby 1.8.4 (2005-12-24) [i686-darwin8.7.1]:

assert_equal("31.13", number_with_precision(31.125, 2)) assert_equal("8.13", number_with_precision(8.125, 2))

This is seemingly due to a problem with sprintf in Ruby. Calling printf "%01.2f" 31.825 in the OSX Terminal correctly returns 31.83 On the other hand in irb: sprintf "%01.2f", 31.825 returns "31.82"

The same erroneous behavior has been observed on a windows machine in cygwin when calling printf in a terminal windows with GNU Bash.

This error also causes the bug in ticket #8027.

-- ( copenhagen.rb rails patch day )

Attachments

added_tests_to_test_number_with_precision.diff (0.7 kB) - added by lau on 05/05/07 20:23:05.
Adds asserts to the unit test of test_number_with_precision that fails under at least some compilations of ruby on OSX and a version of cygwin on Windows.
printf_test.c (108 bytes) - added by tobiashm on 05/06/07 21:54:20.
Simple C program to test rounding floats

Change History

05/05/07 20:23:05 changed by lau

  • attachment added_tests_to_test_number_with_precision.diff added.

Adds asserts to the unit test of test_number_with_precision that fails under at least some compilations of ruby on OSX and a version of cygwin on Windows.

05/06/07 15:59:10 changed by tobiashm

Just for reference: I have the same error in the shell on my Mac G4 running OS X 10.4.9

Info: GNU bash, version 2.05b.0(1)-release (powerpc-apple-darwin8.0) 

$ printf "%01.2f\n" 31.825
31.82
$ printf "%01.2f\n" 32.825
32.83

irb(main):001:0> sprintf "%01.2f", 31.825
=> "31.82"
irb(main):002:0> sprintf "%01.2f", 32.825
=> "32.83"

This might have something to do with the C compiler used?

05/06/07 21:46:39 changed by mortenrt

My MacBook Pro running OS X 10.4.9 

$ printf "%01.2f\n" 31.825
31.83
$ irb
irb(main):001:0> sprintf "%01.2f", 31.825
=> "31.82"

Wierd indeed. I used i686-apple-darwin8-gcc-4.0.1 to compile Ruby.

05/06/07 21:54:20 changed by tobiashm

  • attachment printf_test.c added.

Simple C program to test rounding floats

05/06/07 21:56:41 changed by tobiashm

This C program has the error when compiled with powerpc-apple-darwin8-gcc-4.0.0 on my machine

05/24/07 01:13:42 changed by josh

  • keywords set to test.
  • summary changed from [TEST] test_number_with_precision fails to [PATCH] test_number_with_precision fails.

07/04/07 15:26:17 changed by tomafro

Isn't this discrepancy because 31.825 is a float, not a decimal? 

>> "%01.2f" % 31.825
=> "31.82"
>> "%01.20f" % 31.825
=> "31.82499999999999900000"

>> "%01.2f" % BigDecimal.new("31.825")
=> "31.83"
>> "%01.20f" % BigDecimal.new("31.825")
=> "31.82500000000000300000"

03/24/08 21:12:59 changed by david

  • status changed from new to closed.
  • resolution set to fixed.

(In [9086]) Fixed NumberHelper#number_with_precision to properly round in a way that works equally on Mac, Windows, Linux (closes #11409, #8275, #10090, #8027) [zhangyuanyi]