calc: calculations: code: 'assume' integers or 'clearly identifiable'?

[newbie-02]

fresh new year to all, may it be better than the last …

‘integers’ are guesses? or clearly identifiable in IEEE 754 doubles,

i had some discussions about floats and doubles lately and thought about ‘improving’ the calculation and rounding errors of them, the distinction between ‘integers’ and ‘fractions’ is important for that (values where - decimally - other digits than ‘0’ appear after the decimal point, or not), and i vaguely remember that @erAck had written somewhere ‘assumed to be integers’ or something like that … but couldn’t find the place right away …

my question - it’s more a question for developers - is

what is the decision in the calculation routines ‘integer’ or ‘fraction’? or is it not clearly visible in the bit-salad?

thanks for any help, it won’t directly solve problems of other users but could help to improve calc and IEEE 754 massively … if … i’m right with my guessing …

[erAck]

All double values >= 2^52 by definition of IEEE 754 are integer values. Values < 2^52 are integers if when casted to integer and then casted back to double are equal to the original double value, alternatively the mantissa and exponent could be inspected whether they represent an integer value by calculating 1.mantissa*2^exponent and see if the result is integer. There’s nothing to be “assumed to be integers”.

[newbie-02]

yes, could add: there is something like the decimal point in IEEE floats, shall we call it a ‘binpoint’? it’s between the implicit 1 and the mantissa for range / exponent ‘0’, and shifted right by one place for positive / left acc. negative exponents, bits right of the binpoint being ‘1’ → fraction, all bits right ‘0’ → integer,

and just besides as it came up on looking for theese values: the behaviour of calc displaying
‘=2^53’ as 4503599627370496,00 and
‘=2^52-0,5’ as 4503599627370500,00 with a number format like “0,00” is … funny? … but only when you know why and have learnt to deal with it, for a normal user a correct 16th digit would be better …

[erAck]

You keep repeating yourself over several threads and bugzilla entries. Yes, the display string conversion should do better, and yes, I already said that a dozen times.