csv freezing Calc

[limaunion]

Hello all! newbie in this forum. I’m trying to open a csv file and Calc simply freezes with one of my cpu cores @100% usage.

Here’s all the related information:

$ libreoffice --version
LibreOffice 6.0.3.2 00m0(Build:2)

$ cat stat_archive.csv | wc
30721 30721 20755246

Here’s the first line as an example:

{"_id":{"$oid":“5a4aca64e4b0d5a04c07861a”},“assoc_time”:{"$numberLong":“1544851372”},“disassoc_time”:{"$numberLong":“1514841628”},“duration”:{"$numberLong":“256”},“tx_bytes”:{"$numberLong":“0”},“rx_bytes”:{"$numberLong":“0”},“bytes”:{"$num
berLong":“0”},“user_id”:{"$oid":“5a46c524e4b0d5a05a04dad6”},“mac”:“b8:44:d9:78:a8:e4”,“name”:null,“hostname”:“iPhone-de-Ariel”,“is_guest”:true,“site_id”:“53b5a52ee4b06025fe091cad”,“ip”:“192.168.88.29”,“is_wired”:false,“ap_mac”:“04:18:d6:a
8:43:aa”,“radio”:“ng”,“channel”:1,“roam_count”:0,“o”:“session”,“oid”:“53b5a52bb4b06025fe091cad”,“time”:{"$numberLong":“1514351372000”},“datetime”:{"$date":“2018-01-02T00:02:52.000Z”}}

Any ideas what to do?
Thanks.

[erAck]

Are you sure it’s freezing and not just taking a looong time? It’s 20MB data. That looks more like a JSON file than a CSV file; anyway, nothing to tell without the full file. If it doesn’t contain sensitive data (or replace that with xxx) best submit a bug report and attach the file. If possible trim the file down to an amount that still exhibits the hang.

[tvincent]

Calc is preloading the data with the current CSV config, that you last used to open a CSV file.
Using a separator that doesn’t fit your csv may result in long loading time.

Workaround can be to create a file with only the first line of the big CSV file, and load it using the proper separator.
Then when opening the big version again, Calc will use the proper separator to load the preview.

In fact, Calc should ask the CSV reading parameters before trying to load the preview.

[SM_Riga]

Hello @limaunion

The sample provided has only one common thing with standard comma separated values structure, it is file extension. So the main problem here I believe is trying to import as CSV file which is not a CSV, but contains many (a lot of in 20MB) commas and double quotes, which presumably are used as value and string delimiters in import dialog, but which do not correlate with expected CSV file structure. You can try to change string delimiter in import dialogue to single quote (’), but anyway you will not get the correct structure: as @erAck mentioned, the text string given is a JSON encoded string, representing some object and it is a 2-dimensional array (contains nested arrays), so it’s structure is not reproducible within plain CSV functionality. Here is more readable structure of the first line in you file.

stdClass Object
(
    [_id] => stdClass Object
        (
            [$oid] => 5a4aca64e4b0d5a04c07861a
        )
    [assoc_time] => stdClass Object
        (
            [$numberLong] => 1544851372
        )
    [disassoc_time] => stdClass Object
        (
            [$numberLong] => 1514841628
        )
    [duration] => stdClass Object
        (
            [$numberLong] => 256
        )
    [tx_bytes] => stdClass Object
        (
            [$numberLong] => 0
        )
    [rx_bytes] => stdClass Object
        (
            [$numberLong] => 0
        )
    [bytes] => stdClass Object
        (
            [$num berLong] => 0
        )
    [user_id] => stdClass Object
        (
            [$oid] => 5a46c524e4b0d5a05a04dad6
        )
    [mac] => b8:44:d9:78:a8:e4
    [name] => 
    [hostname] => iPhone-de-Ariel
    [is_guest] => 1
    [site_id] => 53b5a52ee4b06025fe091cad
    [ip] => 192.168.88.29
    [is_wired] => 
    [ap_mac] => 04:18:d6:a 8:43:aa
    [radio] => ng
    [channel] => 1
    [roam_count] => 0
    [o] => session
    [oid] => 53b5a52bb4b06025fe091cad
    [time] => stdClass Object
        (
            [$numberLong] => 1514351372000
        )

    [datetime] => stdClass Object
        (
            [$date] => 2018-01-02T00:02:52.000Z
        )
)

As you can see, some lines contain nested information. Such a multidimensional structure with nested arrays in not reproducible within CSV file, which is just plain table with values.