Using Conditional Format on Large Files

Choochoo22 · January 31, 2021, 11:21pm

I’m new to Libre Office and just trying it out for the first time.

I work with some very large .csv files of logged data. The largest at the moment is roughly 30 columns and 500k+ lines. This file opens OK. Finding anomalies in this data is like finding needles in a haystack. It is greatly helped, made possible really, by using conditional formatting to make the anomalies stand out.

To do this I select a range of columns and apply conditional formats. When done in LibreCalc the program hangs and crashes. Even selecting one column causes the program to crash when trying to apply conditional formats. Is there a way to fix this?

Rick

mariosv · February 1, 2021, 9:52pm

Try to select a defined range instead of select the whole column.

Choochoo22 · February 2, 2021, 8:17pm

Thank you for responding.

On a whim last night I tried it again and it worked OK (whole column). This morning it does not work whether the whole column is selected or only the needed range.

When it is working there is a side effect that scrolling through the file while conditional formatting is active is much slower. This is important in such a large file.

Rick

mariosv · February 4, 2021, 6:40pm

Are there functions in the conditions?

Choochoo22 · February 5, 2021, 8:27am

I have used functions in conditional formatting but generally no. For this task I typically use 3 shades, under-equal-over an expected “normal” value.

Most of the time this is irrelevant as LC crashes when clicking on the conditional formatting button after making the selection.

mariosv · February 5, 2021, 5:55pm

Please test with a clean profile Menu/Help/Restart in Safe Mode

Choochoo22 · February 5, 2021, 10:48pm

I went to safe-mode, selected factory-reset, checked both boxes and continue in safe mode.
With these settings and the sample data it works when selecting color-scale but locks up and crashes if selecting condition…
Although color-scale is what I was looking for, it turns out not to be useful even when it works. Scrolling through the one hour of sample data to view the highlighted anomalies takes three hours of holding down the mouse button while staring at the screen. By comparison; it takes about 25min to scroll through the same data without the highlighting.
The “find” function seems potentially much more convenient but has its own problem as it can only find = and can’t find <>. I asked about this in a separate question here: Is there a way to find a value that is different
I would like to request that the find function be expanded to include <>, <, >, between, etc. similar to the filter function. John is apparently working on it.