LibreOffice Basic error in convertig HEX to DEC

English
,
#1

The Calc application that I created uses various colours in the sheets to colour cells as well as charts. I am using HEX codes for the colour values. All colours map to the HTML colour palette in LibreOfice.
One of the colour is Lime (HEX FF00, dec 65280). When the app runs the Lime has become Yellow (HEX FFFF00). It seems that the system is acting like an integer value (2 bytes) not a long (4 bytes). There us no unsigned long, so I cannot force it to ignore the sign bit.
I did some testing with various values. Attached is a macro that shows that a large block of HEX values are not correctly converted to DEC values.
'=================================================
Sub HEXValuesError
dim loGood as long
dim loBad as long
dim hiBad as long
dim hiGood as long
dim RGBLime as long

loGood = &H7FFF
loBad = &H8000
hiBad = &HFFFF
hiGood = &H10000
RGBLime = &HFF00

Dim oService As Object
'This runs the Calc HEX2DEC function
'It shows the correct values
Set oService = CreateUnoService(“com.sun.star.sheet.FunctionAccess”)
msgbox oService.callFunction(“HEX2DEC”, Array(“7FFF”))
msgbox oService.callFunction(“HEX2DEC”, Array(“8000”))
msgbox oService.callFunction(“HEX2DEC”, Array(“FFFF”))
msgbox oService.callFunction(“HEX2DEC”, Array(“10000”))
msgbox oService.callFunction(“HEX2DEC”, Array(“FF00”)) '65280 Green

'This shows the block of values that do not translate (32768 - 65535 inclusive) 
msgbox "logood " & logood		'* 32767
msgbox "loBad " & loBad			'* -32768 (32768)
msgbox "hiBad " & hiBad			'* -1     (65535)
msgbox "hiGood " & hiGood		'* 65536
msgbox "RGBLime " & RGBLime		'* -256		Yellow

End Sub
'===================================================

I would love to be proven wrong in this. Perhaps some setting that will correct this, or just something weird about my setup.
If it is a problem, how do I pass it on?
Any feedback would be appreciated
Thank you
John

#2

https://bugs.documentfoundation.org/show_bug.cgi?id=130426
https://bugs.documentfoundation.org/show_bug.cgi?id=130476

#3

81337 – BASIC: Long integer support
147379 – Implement LongLong data type in Basic

#4

Try use CLng instead of hexadecimal literals:

Msgbox CLng("&H8000") ' 32768
1 Like
#7

This is a typical case where it’s indispensable to know the version of LibO.
With Version 25.2.2.2 there isn’t any related problem. HEX2DEC() works correctly up to FFFFFF and even FFFFFFFF (8 halfbytes) is correctly converted.
In Basic an automatic conversion like q = &HFFFFFF works up to the needed 6 “hex” digits and so does CDbl(). In one link, @nobu has already pointed out that the problem should be solved for versions V 7.3 or higher.
The Basic command rgbYellow = CDbl(&HFFFF00) even works in the very first version of LibO (3.3) and the result can be assigned to myCell.CellBackColor with the expected effect.
(The Basic type Long accepts assignments of up to &H7FFFFFFF as the upper limit of positive values. The extreme negative value of type Long is to get by &H&H80000000.)

#9

Thank you for your response. Yes this does fix the condition. The line now becomes:
RGBLime - CLng("&HFF00").
There was another response from KamilLanda to add the Type declaration character (&) at the end. As in:
RGBLime = &HFF00&
Although both work, it does not resolve the underlying issue. That is that the assignment of a large range of HEX values will NOT give produce the correct results. The CALC function HEX2DEC has no problem. The 2 suggestions are “workarounds” not solutions.
Since there were 2 solutions, I am not sure if I can indicate that both were the solution.
Thanks again
John

#10

Note that this “issue” can’t be resolved. Compatibility requires that the literals without explicit type specifier are treated as signed integral types of the minimal size that can contain all bits. In fact, your chosen variant is definitely a workaround (requiring a call to a function, passing a string into it, and doing the string-to-number conversion), while the use of type character is the correct solution (using the proper language function, avoiding the non-obvious, but existing and consistent across implementations, type detection rules of the language).

1 Like
#11

Obviously I still don’t understand what you see as your actual issue.
Might the attached example with “macros” help?
disask121803colorCodes.ods (18.0 KB)

#12

Sorry, The version is 25.2.2.2
I don’t know why you introduced HEX2DEC. I know it works correctly.

But it does not work for &HFF00. That is the whole issue.

RGBYellow is treated as a long integer, so it works. In my example RGBLime &HFF00 when assigned to the variable is turned into RGBYellow &HFFFF00. Also, I don’t see why you were convertinng to Double and not Long
John

#15

It worked for me in every version since V 3.3.

#16

@Lupp Note tdf#62326 - the handling of the hex literals changed in 6.4. Since then,

MsgBox &HFF00

produces -256.

#17

I will try to explain. Forget colours. That was just how I found the problem. Let’s say I assign the HEX value &H8432 to a variable. No error, and unless I check this value I will assume that I have entered 33842. However the value is -31694. Perhaps this value is being used to prove the grand unified theory. Poof there goes my Nobel prize, and forever being remembered.
Except some bug report being found, or someone stumbling upon this post no one else will know about this problem with LibreOffice Basic.
I hope this clears up what the issue is.
John

#18

Lupp:
Sorry I added the response in the wrong place
John

#19

I have no problem with that. It is the assignment that causes this problem.
John

#20

How do you get it to work?
John

#21

Having read your replies and the comment by @mikekaganski, I am confused.
Though I thought my tests were thoroughly performed, there must have been a mistake.
I must have used explicit conversion with CDbl() or CLng() in the Basic code in all the tests with recent versions.
The Example I attached actually contains such a conversion.
A next, now really thorough test showed the issue.
Sorry for having wasted your time.

#22

Nothing that teaches is a waste of time. I have learned many things from you and all responders. However it only highlights how much I still do not know. I feel often that I am wasting your time to try to get something into my thick skull.
I appreciate your patience.
John

1 Like
#23

I have chosen to use the Type Declaration character.

Thank you for this. It explains a lot.
John

#5

Add still one & to the end :slight_smile:
msgbox(&hFFFF&)

https://bugs.documentfoundation.org/show_bug.cgi?id=151171

3 Likes
#8

… i.e., use the type-declaration character. Just note that use of these characters in literals was implemented in version 7.0.

Impressum (Legal Info) | Datenschutzerklärung (Privacy Policy)   Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.