I stumbled upon an unexpected precision error in Calc. I understand that floating point numbers are used internally, and can sometimes cause incorrect results in low-significance digits, but this is a really simple case and it surprised me.
I’m starting with a CSV file, which contains a formula. I simplified the issue down to a short 3 line file; the following is the entire file:
11.92
11.29
=A1-A2
Then open that .csv file with LO Calc, checking the “Evaluate formulas” checkbox on the import wizard.
The result in cell A3, which should be simply 0.63, comes up as 0.630000000000001.
I am surprised that such a simple subtraction would generate an error like this; although I expect that someone will probably tell me that floating point number do things like that now and then.
But I would then point out the fact that if you create a new spreadsheet and enter the exact same numbers in cells A1 and A2, and enter the same formula in A3, then Calc displays the correct result in A3, “0.63”. In other words, the behavior is different, despite using the exact same data and formula, depending whether you import from a CSV file, or create the spreadsheet manually by typing in the data into Calc.
I’m not sure if I can call this a “bug”, but I will call it a strange quirk, and a bit of an annoyance.
I am running version 7.6.5.2.