View Single Post
03/02/19, 05:46 AM   #10
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
Originally Posted by votan View Post
This function was changed with 4.3.5.
Then it means ZO_LocalizeDecimalNumber has a new bug.

Originally Posted by votan View Post
Maybe it is an ESO Lua specific precision problem.
Here is another number to test: 303.9*100
No it's not ESO or Lua specific. Here Python:
Code:
>>> 308.09 * 100
30808.999999999996
>>> 303.9 * 100
30389.999999999996
It is not possible to store 308.09 in base-2 representation (in a finite number of bits), e.g. the most widely used binary64 (C double, Lua number, Python float). Regardless of whether the language is compiled to machine code, or bytecode, when the compiler translates the source string "308.09" into a number in base-2 representation, the value will inevitably be "something other than 308.09" (whether less or greater may depend on language, rounding mode or computer architecture, but it cannot be exact).

Some numbers simply don't have finite representation in some bases.
For example 1/3 (one third). In base-3, it is simply 0.1; in base-6 it is 0.2; in base-12 it is 0.4; but in base-10 it is 0.333333333... ad infinitum.
  Reply With Quote