Large CSV import crashes Base

Hello, I am new to LO. I have a large spreadsheet (90K rows, 30 columns) which includes long data strings in one column. It opens in Calc, but I cannot import the data into Base. When I try to, I get the following error:

Runtime Error!

Program: C:…\soffice.bin

This application has requested the Runtime to terminate it in an unusual way. Please contact the application’s support team for more information.

LO then crashes and has to recover the Calc and Base files before proceeding (recovery has not caused issues).

I can import the column headers and the first 100 or so rows (haven’t tried to see how far down I can import), but cannot import the entire dataset. Same is true if I delete the column with the long data strings.

To import, I am copying all of the data in Calc, pressing the “Table” button in Base, and selecting Edit / Paste.

Is there anything I need to be doing to be able to import all of the data in my spreadsheet into Base?

Thanks in advance!

-Sam

Have you tried to open CSV in Base?

Menu/File/New/Database
Connect to an existing database
Select: Text
and follow the wizard.

That is a large data file to try and import into Base. You will have performance issues with 90K tuplets in an embedded hsqldb ODB file. Just imagine how big the array is, that you are asking LO to parse for you and insert into your table.

Either split up your CSV file into smaller, manageable chunks, or else look up the hsqldb documentation about accessing TEXT TABLES directly from the SQL command line. You can issues SQL commands from within your ODB file once it is opened in LO via the TOOLS > SQL menu entry. Even then, you will probably experience performance issues.

If you setup your ODB file to point to hsqldb file source instead of embedding it in the default mode, you might find that things improve. Look up “Using hsqldb in file mode” on the OpenOffice.org forum.