I made a test in LibO 4.3.5.2 on an XP machine:
The formua =VLOOKUP(A2,$Sheet1.$A$2:$C$17,2,0) works correct even when I copy the formula down to the end of the column.
As you can see I selected as the area 3 columns A, B, C; the entire array is A2 to C17.
What could be gone wrong is that Calc does not automatically make the corner points of the 3 column array A2 and C17 as absolute points $A$2 and $C$17. If Google doc does this automatically than you have the reason for not working in LibO.
If
=VLOOKUP(B8,Sheet1!B:D,3,0)
is your formula in Calc, then make B to $B$1 and D to $D$ end value of your array (bottom most right cell of your array).
If
=VLOOKUP(B8,Sheet1!B:D,3,0)
is your working formula in Google doc then there is a small difference between Google doc and LibO in the way that Google docs allows you to select an entire column just by using the column names, B, C, D