Big spreadsheet - how much RAM?

[eteb3]

I have a spreadsheet of two sheets, each with 400k rows, and around 10 columns. Around 75MB.

It started as a CSV so I’m confident there’s no formatting.

Copying and pasting a column of formulas uses 100% of my CPU. It takes 2-3 minutes to copy one column at a time, and I have to force-quit when I ask it to do more than that.

What sort of RAM/storage/etc do I need to work with something this size?
(I’m not techie so I don’t actually know what the limiting factor is likely to be)

Mac OS 12.7
1.6 GHz Dual-Core Intel Core i5
8 GB 1600 MHz DDR3

Thanks.

[Steph1]

Which version of Libre Office do you use?

[fpy]

@eteb3

What

difficult to say in general.

@eteb3

400k

try wtih 4k, 40k, … to make your own idea.

some other ideas …

• Calc - Low performance on large spreadsheet
• How do I set memory (RAM) allocation in newer versions of Libre Office?

[Villeroy]

What is the purpose of loading database data into a spreadsheet? If it is a one-off job, import, sort, filter format, analyse and throw away the source, you can live with the shortcomings and delays.
If this data set is expanding and used on a daily basis, you should not store the stock data on the grid of an arithmetic calculator.

[eteb3]

@Villeroy

If it is a one-off job, import, sort, filter format, analyse and throw away the source, you can live with the shortcomings and delays.

The problem is, I can’t: LO is literally unusable with a file this size.

@Villeroy

If this data set is expanding and used on a daily basis, you should not store the stock data on the grid of an arithmetic calculator.

Ok, I’ve had a similar comment before. Makes sense, I guess; I just don’t know what the alternative is.

[eteb3]

@Steph1

version

It’s 24.8.2.1

[eteb3]

@fpy

try wtih 4k, 40k, … to make your own idea

So I look at the RAM requirements for 4000 rows and then multiply by 400?

[Hrbrgr]

@eteb3

RAM requirements for 4000 rows and then multiply by 400?

You cannot calculate a memory location by specifying a number of rows.
The decisive factor is the maximum number of characters in a line.
1 character corresponds to 1 byte (8 bits).
Add an overhead of approx. 10%.
Now you can do the math.

[Villeroy]

@eteb3

Ok, I’ve had a similar comment before. Makes sense, I guess; I just don’t know what the alternative is.

Developing a small database is the one and only alternative. See Why does LibreOffice Calc take so long to save changes to a file?

[eteb3]

@Villeroy

Developing a small database is the one and only alternative

Thanks. If I have all this data in csv format, can I use Base to create a database from the data?

(Although in natural language I would say the csv is a “database” already.)

I’ve had a quick look at the help page and it looks more complicated than this.

My data source will also provide the data as json, in case that’s easier.

[Villeroy]

@eteb3

I’m not techie

No, I’m afraid, you can’t. This is for definitively for techies and Base can’t help you with that. In that other thread, it took me one hour to store 3.6 million csv records in a database that is fairly usable with Base.
P.S: with Base and with Calc and Writer as well, if you boil down the 3.6 million records to the ones you are interested in right now. Base is a bridge between databases and office documents.