LibreOffice Basic error in convertig HEX to DEC

81337 – BASIC: Long integer support
147379 – Implement LongLong data type in Basic

Try use CLng instead of hexadecimal literals:

Msgbox CLng("&H8000") ' 32768
1 Like

This is a typical case where it’s indispensable to know the version of LibO.
With Version 25.2.2.2 there isn’t any related problem. HEX2DEC() works correctly up to FFFFFF and even FFFFFFFF (8 halfbytes) is correctly converted.
In Basic an automatic conversion like q = &HFFFFFF works up to the needed 6 “hex” digits and so does CDbl(). In one link, @nobu has already pointed out that the problem should be solved for versions V 7.3 or higher.
The Basic command rgbYellow = CDbl(&HFFFF00) even works in the very first version of LibO (3.3) and the result can be assigned to myCell.CellBackColor with the expected effect.
(The Basic type Long accepts assignments of up to &H7FFFFFFF as the upper limit of positive values. The extreme negative value of type Long is to get by &H&H80000000.)

Thank you for your response. Yes this does fix the condition. The line now becomes:
RGBLime - CLng("&HFF00").
There was another response from KamilLanda to add the Type declaration character (&) at the end. As in:
RGBLime = &HFF00&
Although both work, it does not resolve the underlying issue. That is that the assignment of a large range of HEX values will NOT give produce the correct results. The CALC function HEX2DEC has no problem. The 2 suggestions are “workarounds” not solutions.
Since there were 2 solutions, I am not sure if I can indicate that both were the solution.
Thanks again
John

Note that this “issue” can’t be resolved. Compatibility requires that the literals without explicit type specifier are treated as signed integral types of the minimal size that can contain all bits. In fact, your chosen variant is definitely a workaround (requiring a call to a function, passing a string into it, and doing the string-to-number conversion), while the use of type character is the correct solution (using the proper language function, avoiding the non-obvious, but existing and consistent across implementations, type detection rules of the language).

1 Like

Obviously I still don’t understand what you see as your actual issue.
Might the attached example with “macros” help?
disask121803colorCodes.ods (18.0 KB)

Sorry, The version is 25.2.2.2
I don’t know why you introduced HEX2DEC. I know it works correctly.

But it does not work for &HFF00. That is the whole issue.

RGBYellow is treated as a long integer, so it works. In my example RGBLime &HFF00 when assigned to the variable is turned into RGBYellow &HFFFF00. Also, I don’t see why you were convertinng to Double and not Long
John

It worked for me in every version since V 3.3.

@Lupp Note tdf#62326 - the handling of the hex literals changed in 6.4. Since then,

MsgBox &HFF00

produces -256.

I will try to explain. Forget colours. That was just how I found the problem. Let’s say I assign the HEX value &H8432 to a variable. No error, and unless I check this value I will assume that I have entered 33842. However the value is -31694. Perhaps this value is being used to prove the grand unified theory. Poof there goes my Nobel prize, and forever being remembered.
Except some bug report being found, or someone stumbling upon this post no one else will know about this problem with LibreOffice Basic.
I hope this clears up what the issue is.
John

Lupp:
Sorry I added the response in the wrong place
John

I have no problem with that. It is the assignment that causes this problem.
John

How do you get it to work?
John

Having read your replies and the comment by @mikekaganski, I am confused.
Though I thought my tests were thoroughly performed, there must have been a mistake.
I must have used explicit conversion with CDbl() or CLng() in the Basic code in all the tests with recent versions.
The Example I attached actually contains such a conversion.
A next, now really thorough test showed the issue.
Sorry for having wasted your time.

Nothing that teaches is a waste of time. I have learned many things from you and all responders. However it only highlights how much I still do not know. I feel often that I am wasting your time to try to get something into my thick skull.
I appreciate your patience.
John

1 Like

I have chosen to use the Type Declaration character.

Thank you for this. It explains a lot.
John

Add still one & to the end :slight_smile:
msgbox(&hFFFF&)

https://bugs.documentfoundation.org/show_bug.cgi?id=151171

3 Likes

… i.e., use the type-declaration character. Just note that use of these characters in literals was implemented in version 7.0.

that’s wrong cause

  • n(hex)=00 00 00…ff ff ff or
  • n(dec)=000 000 000…255 255 255
    so your colour is defined as n(hex)=‘00 ff 00’.

Leading “0” are not required. Only significant HEXits (don’t know if this is a word, but it means a single character representing a value from 0 to 15) are required. Assigning
FF00, or 00FF00, or N’0’sFF00 produces the same results.
John