does a maximum number of rows per sheet exist?

I tried to read data for a database with calc. I could not read the (huge) dataset (the csv dataset of world trade data had a size of about 1 Gb) , and got the following message: “the data could not be loaded because the maximum number of rows per sheet was exceeded.”
Searching the internet for similar messages, I read one suggestion to use libreoffice-base, as this would not have this problem. Doing so I got an identical message: “the data could not be loaded because the maximum number of rows per sheet was exceeded.” Does this mean datasets of this size cannot be opened in libreoffice? My present version of libreoffice is 6.3.3.2. If you would like to receive a copy of the csv-dataset, I’ll be happy to send it to you.
Thank you
Julien Michielsen

FAQ

Thanks Mike: this does answer my question as far as calc is concerned. However, this does not answer why the same happened in base: does base have the same maximum of rows? This seems unprobable to me as databases should not have a maximum.

You haven’t mentioned anything about how you used Base, or which suggestion have you followed, so I can only assume that you also somehow used Calc (even if indirectly) when got the warning second time.

I retried , making sure no other part of libreoffice (calc or word) was started. No succes, same error message “the data could not be loaded because the maximum number of records per sheet was exceeded”. The fact that the message mentions “sheet” suggests that calc is used for opening csv files, even though the file was opened from libreoffice base.
Thanks for your help
Julien

Please follow @keme’s advise, which is the proper way of opening something in Base. In general, LibreOffice is a monolithic application, and it doesn’t take into account which module you use at the moment when you are opening a file; it uses universal detection method to decide which filter/module to use based on the file content. So yes, when you try simply opening CSV from inside Base, that wouldn’t avoid opening it in Calc, and that’s expected. Base connects to databases, and only opens its own filetype.

Most likely the Base approach will work.

The message you got indicates that LibreOffice tried to open the file as a spreadsheet, which may happen also when you are in a Base window. (Base does not relate to the “sheet” concept which is mentioned in the message.) If you use File - Open to open the csv, the suite’s preferred tool is Calc, regardless of which application you are in when you select the menu item. Base should Connect to the file, not Open it.

  • Make a dedicated folder for your database, and move the csv file into it. The folder should only contain the data file(s).
  • In the first page of the Base dialog, select Connect to existing and then type Text.
  • In the next page you browse to the database folder you made, and adjust settings to suit the format of your file.
  • When you complete the setup, you will be asked to save the database file. This file is just a “settings file” to tell Base how to connect to your data.

Working on a csv dataset of this size may be slow, but it should work. To speed things up, you may consider importing data to a proper database and create index on key fields.

If the above is what you did, and it still does not work, make a note of that here. If you also can share the data, post a link to the share in that note. Post the note as a comment to your question or to this answer, as appropriate. Do not use the “Suggest answer” button unless you have a solution for the issue.