BITAND Producing Incorrect Output

Version: 24.8.2.1 (X86_64) / LibreOffice Community
Build ID: 0f794b6e29741098670a3b95d60478a65d05ef13
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: en-CA (en_GB); UI: en-GB
Calc: CL threaded

Using BIN2DEC, BITAND, DEC2BIN to isolate first decimal digit of a number, to then determine its value.

Found BITANDing target number with 0000001111 is producing an oddball result after decimal 3. At target number decimal 4 onwards, the seventh bit is turned into a 1 eg result: BITAND 0000000100(4) with 0000001111 > 0001000100, not the expected 0000000100.

Any ideas/remedies ?

…especially about what you trying to solve? …NO!

Somewhat odd comment…

It was not odd: it highlighted the omissions in your question. E.g., I had to guess the exact formula you used. And that’s only about BITAND, without any hint what “Using BIN2DEC, BITAND, DEC2BIN to isolate first decimal digit of a number, to then determine its value” could mean in formula terms, or in any other way to get the idea of the task. So even if my answer could resolve the specific misunderstanding around that function, it still may be a case of XY problem…

2 Likes

You obviously call it like

=BITAND(1111;100)

or maybe

=BITAND("00001111";"00000100")

and assume that BITAND takes numbers 15 (1111b) and 4 (100b). No, either way you are passing one thousand one hundred eleven (10001010111b), and one hundred (1100100b). There is no “binary” format, which you may pass to functions taking numbers. You may call it like

=BITAND(BIN2DEC("00001111");BIN2DEC("00000100"))
1 Like

Elaborating on the XY problem as mentioned by @mikekaganski, in an attempt to determine the whY, so as to enable a rethinking of eXecution.

Background

From what you write I assume that by “first decimal digit” you refer to the “one’s position”, i.e. the rightmost digit. Also I assume that you are working with integers.
AFAIK no function of Calc uses BCD encoding (at least not in userland scope), so extracting a decimal digit from a binary representation is not straightforward. Why do you do it this way?


E.g.: Using the binary masking you seem to suggest, an arbitrary decimal number ending in 1 can return any odd value from 1 - 15.


Suggested solution

The first (lowest order, rightmost) decimal digit of an integer can be easily extracted by a few different approaches. I see two basic strategies:

  • Numerically as a remainder of division: =MOD(<sourcenumber>;10)
    Returns a number in range 0 - 9
  • Textually from a digit string: =RIGHT(<sourcenumber>;1)
    The function expects a string first parameter, so it will implicitly convert numeric to string. Then it returns a single character in range “0” - “9”

Note: My suggested formulas are really just basic display of the approach to task as stated. You may need to adjust the formulas for robustness, e.g. accounting for trailing spaces in string, or when you have calculated numbers which look like integers but aren’t exactly integer.

1 Like