issue with rounding small! values, round to more than 20 decimals,

@ajlittoz: You are right concerning the internal limitations, of course.
However: The questioner tried to round to a decimal order of magnitude actually represented in the IEEE 754 representation of the example number. Am I on error?

@Lupp: i would be quite happy about a critical review of tdf#40528, tdf#128312, tdf#133840 :slight_smile:
@all: i’m aware of the problem that the fp-representation messes up the decimal numbers we deal with on the sheet,
the attempt is to look if “len(number mod 1)” results in the decimals used in the UI rather than those used ‘internally’,
and from that on to find a reasonable! rounding, which does not have to be arbitrarily defined by the developers, but corresponds to the expectations of the user based on the entered / visible values, and common math rules,
some of it works, i stumbled over the boundary conditions ‘scientific string’, and over rounding to more than 20 decimal places as calc processes much smaller values and i would like to (try to) correct rounding errors of 0.1E-33 + 0.2E-33 as well,
the whole thing is not a solution for the real application, just an impulse for the developers that better results are! possible,
and on that way … whereever i lug … new bug …

which does not have to be arbitrarily defined by the developers,

There is no rounding by developers. They don’t do any rounding and they most probably never will implement any rounding because the performance cost for a spreadsheet application will be too high. Spreadsheet mathematics fully relies on the machine/processor arithmetic of a 64-bit processors.

And @Lupp’s conclusion in his answer about your issue being a bug is purely formal, since the standard does not define an upper limit for digit and therefore ROUND() should accept any digit, even if it is far beyond of anything making sense for the current processor’s arithmetic (That’s how I understood the conclusion)

Concerning the very old bug: I also stumbled over related problems when I once tried to sort by formulas. In an example I specifically had tuned, there were 138 out of 1000 numbers that were judged by COUNTIF() to be greater than themselves. Insofar the situation has improved. A recent test with the old example didn’t show a single case of the kind. However: Programming cannot resolve the underlying problem finally as long as not either
-Users edit and read numbers (comunicate with the software) in IEEE 754 Double (or any similar rep.).
-Spreadsheet software returns to BCD arithmetics ignoring all the modern hardware features.
-Processors shift to decimal.
In addition there is the fact that most numbers neither have an exact ordinary decimal nor an exact dyadic representation.
There cannot be exact involution between finite decimal and dyadic representations of numbers!
Attached to my answer by editing you will find an example exemplifying this to more detail.

Quoting @anon73440385: “There is no rounding by developers. …”
I cannot find a plausible understnding of this. In addition there is a setting (I always strongly dissuade from enabling) named Precision as shown which in simple cases (no time, no fractional formats) actually seems to do as the name claims.
Quoting @anon73440385: “And _Lupp’s conclusion in his answer being a bug is purely formal, …”
There always is an implicit limitation like ...if not impossible due to logical or hardware limitations... This does not apply to the given case. Even =ROUND(1;-400) should clearly return 0 e.g. according to the specification.
For evidence see the above mentioned attachment, e.g.

@anon73440385: i’m quite sure i’d read plenty comments about rounding in bugs, and discussions which level to use, might be it’s only used for UI display?

@Lupp - if there was a rounding implemented in Calc then all discussions about about “Why is 0.043 - 0.042 = 0.000999999999999994 and not 0.001” would not occur and in that sense I made the statement. If there would be some (heuristic) rounding the value shown would be 0.001 (expected by many people excluding me).

Regarding your second quote: Isn’t that what I tried to express? It’s formal and it doesn’t refer to processor arithmetic and machine epsilon.

@anon73440385: reg. your statement ‘0.043 - 0.042 = 0.000999999999999994 and not 0.001" would not occur’
imho it shouldn’t occur!!,
off from the question who’s right with his assumptions about the basics of calc’s behaviour …
important is the question:
!!! why should / do we nag plenty people with results neither fullfilling their expectations nor simple school math? !!!
first question ‘do we want that?’ - no,
second question ‘do we have to?’ - different opinions,
third question ‘if yes why?’ - different opinions,
‘unavoidable’? - no, i’ll attach a sample to my answer, see blue cells there, calc can! produce better results,
‘performance’? - yes!, you can’t handle big projects like that,
but!

  • most sheets are ‘small’,
  • there are plenty other and faster possibilities to implement even better decimal math,
  • hardware is improving,
  • for plenty tasks precision is more important than speed,
  • it’s a pity how much time is lost in discussions and for irritated users …

@anon73440385
There was plenty of discussion, and there are misled expectations by the zillions.
The one way Calc tries to get rid of useless criticism was to create the mentioned option - and I also cannot propose another one. (I only dissuade from this one, and tell users to apply explicit rounding where rounding is wanted.) The fundamental limitations of “machine arithmetic” are mathematically inavoidable - given a fix length of RAM to be used for a number.
Concerning our different usage of the term “formal” I insist on mine. The mentioned bug actually is a (substantial) bug, and my mentioned advice to use explicit rounding relies -in principle- on the availabilty of correct (bug-free) rounding functions. Fortunately next to nobody will need orders of magnitude outside [-20…20].
We should discuss the rest sitting together with a glass of beer (each). After all, what do I have a motorbike for?

Last comment: The main reason for all these types of discussions seems to be an unclear mixture of expectations, opinions on how it should be and how it is. Finally I accept the invitation for a beer :slight_smile:

@anon73440385:

There is no rounding by developers. They don’t do any rounding and they most probably never will implement any rounding

That’s not true. You will notice for example that the calculation

=0.3-0.2-0.1

indeed produces 0.0 which it wouldn’t without some rounding, compare with

=RAWSUBTRACT(0.3;0.2;0.1)

@erAck - ok,obviously I misunderstood everything @mikekaganski ever told about what developers do and wrote into bug reports on these issue(s). You know the code and I’m quiet now.

@ajlittoz: sorry, sometimes i’m slow in thinking … :frowning:
‘You have a misconception about the real number of digits in your number.’
no - i have a quite good knowledge about ‘my number’, what’s irritating is what ‘fp-machines’ and calc make of it. i accept that it’s neccessary, and know something about how and why, but i’d really like better results and less questions,
and to play in that direction i needed the rounding of small values, and - surprised myself - it works, as well the rounding (not perfect in that sheet) as the correction of fp-artefacts, as of now better than expected … see sheet attached to my answer for @anon73440385

@newbie-02: I also apologise because my answer was too fast and didn’t address correctly your question.

FP numbers are OK as long as you don’t meet the limits where it is impossible to have an exact representation of the number both in bases 1/2 and 1/10. There is a common subset between N and FP and it is adequate for accounting provided you opt for an adequate base unit (¢ vs €) and your numbers are not too large. If you take the precaution to always used integer arithmetic (within FP range) for accounting, you’ll have no problem.

I don’t think Calc is the right tool for “scientific” computing. Your 6.234e-27 hinted me in this direction. Numerical analysis tells me there is enough inaccuracy with method and computing (round/truncate) that I should not try to interfere with hardware FP result lest I add more inaccuracy and degrade the result.

This is perhaps where I did not understand your question.

I can confirm the described behavior of the ROUND() function with LibreOffice 7.0.1.2 Calc.
The respective specification OpenDocument-v1.3-cs01-part4-formula.odt (#6.17.5 ROUND) [10^−Digits badly rendered in the pdf version] gives no related constraint. Conclusion: It’s a bug.

Bugs should be reported to bugs.documentfoundation.org .

===Edit: The announced attachment===
ask266009roundingToVerySmallModulus.ods

@Lupp: bug is filed, see here:
if you have time i’d like a recheck …
:slight_smile:

For the specific case in the question there’s the ROUNDSIG() function, round to significant digits, here

=ROUNDSIG(6.23455E-027;4)

=> 6.235E-27

@erAck: nice to hear from you, but i’m in question if that helps,
i want to truncate the artefacts from fp-rounding (fp-conversion), and thus round to a calculated number of ‘decimal digits’, digits after! the decimal separator,
roundsig will spread the accuracy over all digits, including those before the “.” or “,”, and thus be too ‘harsh’ with the decimal part … ?
acc. to a suggestion from @Lupp that rounding should ‘go away from zero’ i adapted my substitute-formula to:
function round_e (ByVal darg1 as Double, darg2 as Double)
round_e = sgn(darg1) * Int((abs(darg1) * 10^darg2) + .5) / 10^darg2
and hope that ‘holds’

@erAck: sorry, took some time to think about,
roundsig doesn’t help for people who want a limited number of decimals,
but as i’m targetting the general precision roundsig could be better,
will investigate, but am stuck with anther problem, ‘cInt("+308")’ resulting in ‘0’, while ‘cInt(“308”)’ gives ‘308’ and ‘cInt("-308")’ ‘-308’ ???
thus later …
for big numbers the situation reg. ‘visible content’ / ‘influence of rounding artefacts’ is a little different?

[edit: sheet attached for @anon73440385]
sample for 0.043 - 0.042, and other errors - click do download
[/edit]

tried to help myself, and hope that

function round_e (ByVal darg1 as Double, iarg2 as Integer)
'own rounding routine, careful with neagtive values, ".5" rounded 'up' - towards "+ infinite"
round_e = int((darg1 * 10^iarg2) +.5) / 10^iarg2 
end function

will make it …