Math.floor issue

[mako]

I’m modifying numbers to have 2 decimal places after calculations for cash values - so for values like 0.075 it would be $0.08 (rounded up). I’m using math.floor and it works for the most part but I’m seeing a weird result running the code below:

0.075 —> rounds correctly to 0.08

0.575 —> rounds INCORRECTLY to 0.57 (should be 0.58)

1.075 —> rounds correctly to 1.08

it looks like math.floor(58) = 57 when calculated from the values above it, but it’s correct math.floor(58)=58 in the print statement.

Is there something I’m missing or doing incorrect here?

This is in Corona Version 2013.1251 (2013.11.1):

local function round(val, decimal)

  if (decimal) then

    local part1 = (val * 10^decimal)

    local part2 = part1+0.50

    local part3 = math.floor(part2)

    print("-----------------------------------------------")

    print(“multiplying it =”,part1)

    print(“adding 0.5 =”,part2)

    print("** math.floor to above =",part3)

    print(“math.floor(58)=”,math.floor(58))

    print("------------------------------------------------")

    local finalval = part3 / (10^decimal)

    return finalval

  else

    return math.floor(val+0.5)

  end

end

print(“rounded value 0.075=”,round(0.075,2))

print(“rounded value 0.575=”,round(0.575,2))

print(“rounded value 1.075=”,round(1.075,2))

[rob]

You probably should add the 0.5 before you change the exponent.

Rob

[mako]

Thanks Rob,

I tried playing with that but it seems to not like multiplying by 100 specifically which seems weird. When I run the code below, it shows 57on the lines marked ‘>>>>>>error>’ all the others show 58.

print(">>>>>>",math.floor( 0.5 + (0.0575 * 1000)) )

print("------------------------------------------------")

print(">>>>> error >",math.floor( 0.5 + (0.575 * 100)) )

print(">>>>> error >",math.floor( 0.5 + (0.575 * 10^2)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (0.575 * 10 * 10)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (5.75 * 10)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (57.5)) )

[AlanPlantPot]

I just tried this:

print( math.round(0.575 \* 100))

I would expect the answer to be 58:

0.575 * 100 = 57.5 —> should round up to 58

but it prints out as 57. 

Have I overlooked something?

I use this as my rounding to X decimal place function, which is similar to yours but I’ve never needed to use math.floor or add 0.5:

local function roundDec(num, dec) local d = 10 ^ dec return math.round(num \* d) / d end

It’s always seemed to give me correct results until I just tried this value.

[rob]

I’ll check with the engineers.

Rob

[AlanPlantPot]

Well that’s rubbish.  

I get it in the case of (1/3) * 3 ~= 1 since 1/3 will be 0.3333333333… and that will cause issues when rounding.

But surely (0.575 * 100) is 57.5? Where is the rounding error introduced in that calculation?

[mako]

Thanks for the workaround alex81, it seems to work for me.

Mako

[rob]

Actually the engineers provided me with a boat load of information on this.   It’s not a Corona SDK issue and it’s technically not even a Lua issue since Lua is just calling the C-Lib’s math.floor function directly.   The problem is floating point number tend to have these issues.  While we might think 0.575 is 0.575 internally to the hardware it might be represented by 0.5749999999999 and it’s very possible that it’s getting rounded up a bit in hardware to make it look like 0.575.  Since you know your exponent (in the the original post), why not do a 1/1^number and add that to your value to make sure you cover for this hardware issue.

Rob

[piotrz55]

To explain how it works for interested

To represent fractions computers can only use binary data, so
0.5 becomes 0.1 in binary (we have 1*2^(-1))
0.25 is 0.01 (0*2^(-1) + 1*2^(-2))
0.75 is 0.11 (1*2^(-1) + 1*2^(-2))

But to represent 0.575 computer have to add more 0/1s (ie smaller negative powers of 2: 0.5, 0.25, 0.125…) to in outcome get 0.575. However computer cannot add 0/1s infinitely - usually up to 32 or 64 times. If it isn’t enough then you loose precision and when rounding it’s still less then 0.575 in human notaton, even 0.574999999999 but for computer still less.

As suggested it’s better to have 575 and when needed divide it because whole numbers are always represented correctly in system.

[jbp1]

Depending on how the ^ (power) operator is called, there could be a compounding issue there – it may technically be doing a float to the power of a float (even though you’re calling it with an integer).  That could be a more computationally expensive operation, and it could lead to further round-off errors.

Unless you need to round to arbitrary precision, you might want to just hard-code:   part1 = (val * 100) and finalval = part3 / 100

(though piotrz55’s and alex81’s suggestions about using integers throughout is an even better idea)

[rob]

You probably should add the 0.5 before you change the exponent.

Rob

[mako]

Thanks Rob,

I tried playing with that but it seems to not like multiplying by 100 specifically which seems weird. When I run the code below, it shows 57on the lines marked ‘>>>>>>error>’ all the others show 58.

print(">>>>>>",math.floor( 0.5 + (0.0575 * 1000)) )

print("------------------------------------------------")

print(">>>>> error >",math.floor( 0.5 + (0.575 * 100)) )

print(">>>>> error >",math.floor( 0.5 + (0.575 * 10^2)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (0.575 * 10 * 10)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (5.75 * 10)) )

print("------------------------------------------------")

print(">>>>>>",math.floor( 0.5 + (57.5)) )

[AlanPlantPot]

I just tried this:

print( math.round(0.575 \* 100))

I would expect the answer to be 58:

0.575 * 100 = 57.5 —> should round up to 58

but it prints out as 57. 

Have I overlooked something?

I use this as my rounding to X decimal place function, which is similar to yours but I’ve never needed to use math.floor or add 0.5:

local function roundDec(num, dec) local d = 10 ^ dec return math.round(num \* d) / d end

It’s always seemed to give me correct results until I just tried this value.

[rob]

I’ll check with the engineers.

Rob

[AlanPlantPot]

Well that’s rubbish.  

I get it in the case of (1/3) * 3 ~= 1 since 1/3 will be 0.3333333333… and that will cause issues when rounding.

But surely (0.575 * 100) is 57.5? Where is the rounding error introduced in that calculation?

[mako]

Thanks for the workaround alex81, it seems to work for me.

Mako

[rob]

Actually the engineers provided me with a boat load of information on this.   It’s not a Corona SDK issue and it’s technically not even a Lua issue since Lua is just calling the C-Lib’s math.floor function directly.   The problem is floating point number tend to have these issues.  While we might think 0.575 is 0.575 internally to the hardware it might be represented by 0.5749999999999 and it’s very possible that it’s getting rounded up a bit in hardware to make it look like 0.575.  Since you know your exponent (in the the original post), why not do a 1/1^number and add that to your value to make sure you cover for this hardware issue.

Rob

[piotrz55]

To explain how it works for interested

To represent fractions computers can only use binary data, so
0.5 becomes 0.1 in binary (we have 1*2^(-1))
0.25 is 0.01 (0*2^(-1) + 1*2^(-2))
0.75 is 0.11 (1*2^(-1) + 1*2^(-2))

But to represent 0.575 computer have to add more 0/1s (ie smaller negative powers of 2: 0.5, 0.25, 0.125…) to in outcome get 0.575. However computer cannot add 0/1s infinitely - usually up to 32 or 64 times. If it isn’t enough then you loose precision and when rounding it’s still less then 0.575 in human notaton, even 0.574999999999 but for computer still less.

As suggested it’s better to have 575 and when needed divide it because whole numbers are always represented correctly in system.

[jbp1]

Depending on how the ^ (power) operator is called, there could be a compounding issue there – it may technically be doing a float to the power of a float (even though you’re calling it with an integer).  That could be a more computationally expensive operation, and it could lead to further round-off errors.

Unless you need to round to arbitrary precision, you might want to just hard-code:   part1 = (val * 100) and finalval = part3 / 100

(though piotrz55’s and alex81’s suggestions about using integers throughout is an even better idea)