best way set up data analysis

English
, , , ,
#1

I use LibreOffice to analyse test data generated in csv files, with a new csv file for each test run.

My process in the past has been to load the csv files into calc, import the calc sheet into base, create a query for the appropriate records, import the query into calc via F4 (now “CTL-SHIFT-F4”) so that I can perform more calculations on it and graph it, then copy the graphs into writer for presentation. The process works, but feels very clunky.

What I desire in the final result is

  1. the ability to review data and weed
    out bad stuff before committing to
    database’
  2. the ability to select data sets with a query
  3. the ability to do calculations on the data from the query
  4. ability to graph the data after analyzed
  5. ability to put the graphs in writer documents
  6. ability to update the graphs as the calculations are refined

I am currently stuck at item 6, because the graphs copied to writer lose any link back to the original calc document. This means the error prone and tedious process of upgrading each and every graph in a document every time it is tweaked, or possibly going back and tweaking the calc document to match the writer document. My rule of the thumb is that if there are two different versions of anything you will invariably get the wrong one, so there should only be one version.

I now realize that I set this up based on experience with Microsoft Office, and suspect there might be a better way to set things up in LibreOffice. I think this workflow and basic requirements are very similar for anyone doing scientific work. Is there a better way to set up LibreOffice to handle this work?

#2

Hello,

There is much stated in your question.

For Item #6 please see my answer in this post → Create an updatable link from within Writer to a Calc Chart?.

I can also visualize possibly eliminating Calc completely. Can bring CSV into Base. Can Create charts in Base (again Macros) - see this post → Charts in Base forms. Can get data from base to Writer.

There are various methods but much depends upon your understanding of LO and somewhat on your ability to code macros.

#3

Thank you for your input! I will attempt to insert an updatable link from Writer to Calc, since updating charts individually is the most odious problem at the moment. I need to create a presentation and keep it up to date as new-better data comes in.

The ability to record macros will be a great help to me in learning to code LO macros “on the fly”. I got reasonably proficient in excel, but never to the point where I would sit down and write a macro “from scratch”–I always started with a recording and then worked from there. Last time I tried to learn LO macros (make that OO macros I think) things were not sufficiently well documented and I did not have time to pursue it. Can you suggest a starting point for me to try again?

I tried the charting ability in base some time back but it was pretty rudimentary, calc was better for one who could not write macros. Would base charts in writer be updatable or would I need macros?

I’m sorry but the editor eats paragraph formatting.

#4

Using recorded macros is not a good method to using or learning macros. There is no easy method to learning this and takes considerable time. All charts mentioned in my answer use macros which require a higher than entry level of understanding.

For documentation, see this post → To learn LibreOffice Base are there introductions or tutorials?. The macro links there apply to all of LO.

#5

Spent last evening playing with linking data into a spreadsheet table. Procedure used was

  1. Open Calc Spreadsheet
  2. Copy Desired Page
  3. Place cursor in writer document
  4. Paste Special;More Options;DDE link
  5. Hilite columns in table
  6. Insert;Chart

Charts could be created and appear to updated when the document is loaded, which is adequate for my needs–no listener required if you only want to update when opening the document. This seems to work well with data about 9 colums by 100 rows. The graphs in write appear crisper than the copied and pasted ones

#6

When I went to my real data (750 rows by 20 columns) I ran into performance problems. These were

  1. Approximately 10 seconds to enter edit mode on any graph
  2. Approximately 5 seconds to leave edit mode on any graph
  3. About 2 minutes to save the writer document (sometimes much more)

My machine uses an I5 cpu at 2.66 GHz, and I have 8 GB of memory, windows 7 64 bit operating system. LO maxes out one core (of four) for these time periods. There is no thrashing of the disk or difference when I save my file on a Solid State Drive. The memory does not appear to be filled.

I also could not figure out how to delete the embedded table—once it was in the document I could not remove it. I’ll start another question on that.

Impressum (Legal Info) | Datenschutzerklärung (Privacy Policy)   Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.