Calc Basic macro integer arithmetic problem

[Ajohn]

I’m dealing with prime number fractions and have noticed that Calc Basic isn’t really using integer. It’s rounding if I do a divide for instance rather than truncation. It’s using some sort of rounding process. I also then noticed that some integer functions are stated to return double so it looks like pure integer is never used under the hood.

I wonder if there are any compiler options to prevent the rounding or even better force true integer. Mod seems to work ok but as there is no quotient that I am aware of so I have to use divide.

I chose the integer way as the macro has to loop through 4,000 numbers and I can’t check all of them. If done in pure integer all numbers have to be ok just on the basis of one being correct. Float has been known to cause problems with what I am doing.

John

[Ajohn]

Well I didn’t expect an answer but feel the lack of true integer variables is disgusting.

[Lupp]

As opposed to the spreadsheet cells where any numerical value is IEEE Double, LibreOffice BASIC has integer types. What you may grumble about is that integer arithmetic isn’t supported to the degree you would like. Btw: the operator @hasklo mentioned should be equivalent to Quotient(Numerator, Denominator).
Generally machine arithmetic does not support unlimited precision for integer numbers. If you need it, you have to resort to mathematical software. Try the free ‘Maxima’ e.g.

[hasklo]

I know it’s been really long since you asked this question… but maybe this can help someone with similar question.

Andrew Pitonyak answered this question in his book “OpenOffice.org Macros Explained” under the section “Integer division”. You can download a PDF version here: [http://www.pitonyak.org/OOME_3_0.pdf](http://www.pitonyak.org/OOME_3_0.pdf

Basically, the “/” operator returns double. If you set it to an integer, it gets rounded as you have found. There is another operator specifically for integer division: “” (from left to right slash), which will truncate the result.

dim iMe as Integer
iMe = 3\4    ' --> iMe is 0

[CaveMan]

thank you!! I thought I was going nuts.