Ask Your Question
0

csv freezing Calc

asked 2018-04-19 17:41:09 +0100

limaunion gravatar image

Hello all! newbie in this forum. I'm trying to open a csv file and Calc simply freezes with one of my cpu cores @100% usage.

Here's all the related information:

$ libreoffice --version LibreOffice 6.0.3.2 00m0(Build:2)

$ cat stat_archive.csv | wc 30721 30721 20755246

Here's the first line as an example:

{"_id":{"$oid":"5a4aca64e4b0d5a04c07861a"},"assoc_time":{"$numberLong":"1544851372"},"disassoc_time":{"$numberLong":"1514841628"},"duration":{"$numberLong":"256"},"tx_bytes":{"$numberLong":"0"},"rx_bytes":{"$numberLong":"0"},"bytes":{"$num berLong":"0"},"user_id":{"$oid":"5a46c524e4b0d5a05a04dad6"},"mac":"b8:44:d9:78:a8:e4","name":null,"hostname":"iPhone-de-Ariel","is_guest":true,"site_id":"53b5a52ee4b06025fe091cad","ip":"192.168.88.29","is_wired":false,"ap_mac":"04:18:d6:a 8:43:aa","radio":"ng","channel":1,"roam_count":0,"o":"session","oid":"53b5a52bb4b06025fe091cad","time":{"$numberLong":"1514351372000"},"datetime":{"$date":"2018-01-02T00:02:52.000Z"}}

Any ideas what to do? Thanks.

edit retag flag offensive close merge delete

Comments

Are you sure it's freezing and not just taking a looong time? It's 20MB data. That looks more like a JSON file than a CSV file; anyway, nothing to tell without the full file. If it doesn't contain sensitive data (or replace that with xxx) best submit a bug report and attach the file. If possible trim the file down to an amount that still exhibits the hang.

erAck gravatar imageerAck ( 2018-04-19 20:49:28 +0100 )edit

2 Answers

Sort by » oldest newest most voted
0

answered 2019-02-06 23:26:57 +0100

Hello @limaunion

The sample provided has only one common thing with standard comma separated values structure, it is file extension. So the main problem here I believe is trying to import as CSV file which is not a CSV, but contains many (a lot of in 20MB) commas and double quotes, which presumably are used as value and string delimiters in import dialog, but which do not correlate with expected CSV file structure. You can try to change string delimiter in import dialogue to single quote ('), but anyway you will not get the correct structure: as @erAck mentioned, the text string given is a JSON encoded string, representing some object and it is a 2-dimensional array (contains nested arrays), so it's structure is not reproducible within plain CSV functionality. Here is more readable structure of the first line in you file.

stdClass Object
(
    [_id] => stdClass Object
        (
            [$oid] => 5a4aca64e4b0d5a04c07861a
        )
    [assoc_time] => stdClass Object
        (
            [$numberLong] => 1544851372
        )
    [disassoc_time] => stdClass Object
        (
            [$numberLong] => 1514841628
        )
    [duration] => stdClass Object
        (
            [$numberLong] => 256
        )
    [tx_bytes] => stdClass Object
        (
            [$numberLong] => 0
        )
    [rx_bytes] => stdClass Object
        (
            [$numberLong] => 0
        )
    [bytes] => stdClass Object
        (
            [$num berLong] => 0
        )
    [user_id] => stdClass Object
        (
            [$oid] => 5a46c524e4b0d5a05a04dad6
        )
    [mac] => b8:44:d9:78:a8:e4
    [name] => 
    [hostname] => iPhone-de-Ariel
    [is_guest] => 1
    [site_id] => 53b5a52ee4b06025fe091cad
    [ip] => 192.168.88.29
    [is_wired] => 
    [ap_mac] => 04:18:d6:a 8:43:aa
    [radio] => ng
    [channel] => 1
    [roam_count] => 0
    [o] => session
    [oid] => 53b5a52bb4b06025fe091cad
    [time] => stdClass Object
        (
            [$numberLong] => 1514351372000
        )

    [datetime] => stdClass Object
        (
            [$date] => 2018-01-02T00:02:52.000Z
        )
)

As you can see, some lines contain nested information. Such a multidimensional structure with nested arrays in not reproducible within CSV file, which is just plain table with values.

edit flag offensive delete link more
0

answered 2019-02-06 21:02:59 +0100

Calc is preloading the data with the current CSV config, that you last used to open a CSV file. Using a separator that doesn't fit your csv may result in long loading time.

Workaround can be to create a file with only the first line of the big CSV file, and load it using the proper separator. Then when opening the big version again, Calc will use the proper separator to load the preview.

In fact, Calc should ask the CSV reading parameters before trying to load the preview.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-04-19 17:41:09 +0100

Seen: 63 times

Last updated: Feb 06