issue with rounding small! values, round to more than 20 decimals,

[ajlittoz]

@newbie-02: I also apologise because my answer was too fast and didn’t address correctly your question.

FP numbers are OK as long as you don’t meet the limits where it is impossible to have an exact representation of the number both in bases 1/2 and 1/10. There is a common subset between N and FP and it is adequate for accounting provided you opt for an adequate base unit (¢ vs €) and your numbers are not too large. If you take the precaution to always used integer arithmetic (within FP range) for accounting, you’ll have no problem.

I don’t think Calc is the right tool for “scientific” computing. Your 6.234e-27 hinted me in this direction. Numerical analysis tells me there is enough inaccuracy with method and computing (round/truncate) that I should not try to interfere with hardware FP result lest I add more inaccuracy and degrade the result.

This is perhaps where I did not understand your question.

[Lupp]

I can confirm the described behavior of the ROUND() function with LibreOffice 7.0.1.2 Calc.
The respective specification OpenDocument-v1.3-cs01-part4-formula.odt (#6.17.5 ROUND) [10^−Digits badly rendered in the pdf version] gives no related constraint. Conclusion: It’s a bug.

Bugs should be reported to bugs.documentfoundation.org .

===Edit: The announced attachment===
ask266009roundingToVerySmallModulus.ods

[newbie-02]

@Lupp: bug is filed, see here:
if you have time i’d like a recheck …

[erAck]

For the specific case in the question there’s the ROUNDSIG() function, round to significant digits, here

=ROUNDSIG(6.23455E-027;4)

=> 6.235E-27

[newbie-02]

@erAck: nice to hear from you, but i’m in question if that helps,
i want to truncate the artefacts from fp-rounding (fp-conversion), and thus round to a calculated number of ‘decimal digits’, digits after! the decimal separator,
roundsig will spread the accuracy over all digits, including those before the “.” or “,”, and thus be too ‘harsh’ with the decimal part … ?
acc. to a suggestion from @Lupp that rounding should ‘go away from zero’ i adapted my substitute-formula to:
function round_e (ByVal darg1 as Double, darg2 as Double)
round_e = sgn(darg1) * Int((abs(darg1) * 10^darg2) + .5) / 10^darg2
and hope that ‘holds’

[newbie-02]

@erAck: sorry, took some time to think about,
roundsig doesn’t help for people who want a limited number of decimals,
but as i’m targetting the general precision roundsig could be better,
will investigate, but am stuck with anther problem, ‘cInt("+308")’ resulting in ‘0’, while ‘cInt(“308”)’ gives ‘308’ and ‘cInt("-308")’ ‘-308’ ???
thus later …
for big numbers the situation reg. ‘visible content’ / ‘influence of rounding artefacts’ is a little different?

[newbie-02]

[edit: sheet attached for @anon73440385]
sample for 0.043 - 0.042, and other errors - click do download
[/edit]

tried to help myself, and hope that

function round_e (ByVal darg1 as Double, iarg2 as Integer)
'own rounding routine, careful with neagtive values, ".5" rounded 'up' - towards "+ infinite"
round_e = int((darg1 * 10^iarg2) +.5) / 10^iarg2 
end function

will make it …