You can use a matrix formula. Think the data as column vectors, where you add vectors. If your data is from B1 to E99, then you can use AVERAGE(B1:B99+C1:C99+D1:D99+E1:E99), entered as matrix formula with CTRL+SHIFT+ENTER. But I cannot test, whether LibreOffice can handle 500'000 rows in this way.

"Pulling down" is the wrong way for filling so many rows. Other methods:

(1) In case the target column is adjacent to a data column, you can double-click the drag-fill-square. That will fill the target column as far as there are filled cells on the left side of it.

(2) First mark the target area and then fill it. For marking such large area use the name field, which is left-top of the table grid. Add the range address and then hit ENTER-key. If the formula is already in the clipboard, then you can simple paste it. For a new formula make sure the active cell (with the black border) is the first cell in the target area, then mark the target area via name field, then enter the formula into the input line and finish with ALT+ENTER.