[edit II by b. 2021-06-09]
had some success with correction for addition roundoff/-on fails, the classic ‘=0.1 + 0.2’ → 0.30000000000000004, hope it holds for other values, see attached screenshot ‘repaAdd’ - click to open, cells B34, C34,
would like / need some more testcases for roundon / roundoff fails in additions, who has some ready? (they normally don’t come into user sight as calc rounds and displays only 15 digits and the errors are very small, one ULP at max?, thus out of sight. '=rawsubtract(x, value(left(x,5))) helped me to make some cancellation fails visible,
and does someone have a code pointer for the preparation / rounding of values for display on screen and write to file? would like to try to give that one more digit …
[edit by b. 2021-06-07]
code found, credits go to @mikekaganski, that was absolutely fantastically hidden …
achieved some success, see attached screenshot ‘coroSub’ - click to open, left standard calc, right patched ver.
who would like to help for a clean implementation and further improvement(s)?
@mikekaganski: the project contradicts some theses about fp inaccuracy you made and spreaded in the past, may i ask you to try to find errors in it? ‘advocatus diaboli’?
@all: with these patches calc will deviate from excel in some results … in direction ‘mathematically correct’. how to handle?
A: say ‘correct is better, the problem goes to … Redmont?’
B: make precise calculations optional? a switch / option ‘excel compatible’?
C: leave calc as it is and continue to declare mathematical accuracy impossible, superfluous and undesirable?
(if i got lost somewhere … be lenient, the intention was good, the work unfamiliar and difficult, and even if i hacked together cruel code and other bullshit … it shows that more is possible than calc delivers so far. (This is what gnumeric has been showing for decades, but this is a point that even the! precision fanatics par excellence haven’t solved yet?) Calc could possibly overtake Excel and! gnumeric? i like that idea … )
original question was:
i think i can put some better precision in some calculations, esp. improving subtractions which suffer from cancellation, but … i’m not a coder and not used to calc’s programming structure …
(did some tries replacing approxSub, as it’s defined in a ‘hxx’ file it didn’t let me define functions, tried to put it in math.cxx and ran into ‘not defined’ and missing access to other standards and plenty other errors … ),
can anyone give me a pointer where the calculation for a simple subtraction starts, and which path it’s supposed to make through the modules and procedures? would be very nice …
(P.S. i know it’s more a ‘dev-question’, but at the moment not connected to a special bug, asked in tdf#124286, not yet answered, thus asking here too)