Ask LibreOffice - RSS feedhttps://ask.libreoffice.org/en/questions/Questions and answers for LibreOfficeenSun, 21 Jun 2020 13:54:34 +0200Calc Basic macro integer arithmetic problemhttps://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/ I'm dealing with prime number fractions and have noticed that Calc Basic isn't really using integer. It's rounding if I do a divide for instance rather than truncation. It's using some sort of rounding process. I also then noticed that some integer functions are stated to return double so it looks like pure integer is never used under the hood.
I wonder if there are any compiler options to prevent the rounding or even better force true integer. Mod seems to work ok but as there is no quotient that I am aware of so I have to use divide.
I chose the integer way as the macro has to loop through 4,000 numbers and I can't check all of them. If done in pure integer all numbers have to be ok just on the basis of one being correct. Float has been known to cause problems with what I am doing.
John
-Tue, 07 Jun 2016 15:41:41 +0200https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/Comment by Lupp for <p>I'm dealing with prime number fractions and have noticed that Calc Basic isn't really using integer. It's rounding if I do a divide for instance rather than truncation. It's using some sort of rounding process. I also then noticed that some integer functions are stated to return double so it looks like pure integer is never used under the hood.</p>
<p>I wonder if there are any compiler options to prevent the rounding or even better force true integer. Mod seems to work ok but as there is no quotient that I am aware of so I have to use divide.</p>
<p>I chose the integer way as the macro has to loop through 4,000 numbers and I can't check all of them. If done in pure integer all numbers have to be ok just on the basis of one being correct. Float has been known to cause problems with what I am doing.</p>
<h2>John</h2>
https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=145697#post-id-145697As opposed to the spreadsheet cells where any numerical value is IEEE Double, LibreOffice BASIC **has** integer types. What you may grumble about is that integer arithmetic isn't supported to the degree you would like. Btw: the operator @hasklo mentioned should be equivalent to Quotient(Numerator, Denominator).
Generally machine arithmetic does not support unlimited precision for integer numbers. If you need it, you have to resort to mathematical software. Try the free 'Maxima' e.g.Thu, 08 Feb 2018 12:15:35 +0100https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=145697#post-id-145697Comment by Ajohn for <p>I'm dealing with prime number fractions and have noticed that Calc Basic isn't really using integer. It's rounding if I do a divide for instance rather than truncation. It's using some sort of rounding process. I also then noticed that some integer functions are stated to return double so it looks like pure integer is never used under the hood.</p>
<p>I wonder if there are any compiler options to prevent the rounding or even better force true integer. Mod seems to work ok but as there is no quotient that I am aware of so I have to use divide.</p>
<p>I chose the integer way as the macro has to loop through 4,000 numbers and I can't check all of them. If done in pure integer all numbers have to be ok just on the basis of one being correct. Float has been known to cause problems with what I am doing.</p>
<h2>John</h2>
https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=71449#post-id-71449Well I didn't expect an answer but feel the lack of true integer variables is disgusting.Sat, 11 Jun 2016 17:13:21 +0200https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=71449#post-id-71449Answer by hasklo for <p>I'm dealing with prime number fractions and have noticed that Calc Basic isn't really using integer. It's rounding if I do a divide for instance rather than truncation. It's using some sort of rounding process. I also then noticed that some integer functions are stated to return double so it looks like pure integer is never used under the hood.</p>
<p>I wonder if there are any compiler options to prevent the rounding or even better force true integer. Mod seems to work ok but as there is no quotient that I am aware of so I have to use divide.</p>
<p>I chose the integer way as the macro has to loop through 4,000 numbers and I can't check all of them. If done in pure integer all numbers have to be ok just on the basis of one being correct. Float has been known to cause problems with what I am doing.</p>
<h2>John</h2>
https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?answer=145691#post-id-145691I know it's been really long since you asked this question... but maybe this can help someone with similar question.
Andrew Pitonyak answered this question in his book "OpenOffice.org Macros Explained" under the section "Integer division". You can download a PDF version here: [http://www.pitonyak.org/OOME_3_0.pdf](http://www.pitonyak.org/OOME_3_0.pdf
Basically, the "/" operator returns double. If you set it to an integer, it gets rounded as you have found. There is another operator specifically for integer division: "\" (from left to right slash), which will truncate the result.
dim iMe as Integer
iMe = 3\4 ' --> iMe is 0Thu, 08 Feb 2018 11:37:33 +0100https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?answer=145691#post-id-145691Comment by CaveMan for <p>I know it's been really long since you asked this question... but maybe this can help someone with similar question.</p>
<p>Andrew Pitonyak answered this question in his book "<a href="http://OpenOffice.org">OpenOffice.org</a> Macros Explained" under the section "Integer division". You can download a PDF version here: [<a href="http://www.pitonyak.org/OOME_3_0.pdf](http://www.pitonyak.org/OOME_3_0.pdf">http://www.pitonyak.org/OOME_3_0.pdf]...</a></p>
<p>Basically, the "/" operator returns double. If you set it to an integer, it gets rounded as you have found. There is another operator specifically for integer division: "\" (from left to right slash), which will truncate the result.</p>
<pre><code>dim iMe as Integer
iMe = 3\4 ' --> iMe is 0
</code></pre>
https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=251558#post-id-251558thank you!! I thought I was going nuts.Sun, 21 Jun 2020 13:54:34 +0200https://ask.libreoffice.org/en/question/71180/calc-basic-macro-integer-arithmetic-problem/?comment=251558#post-id-251558