I am aware that it is possible to --convert-to an entire batch of files so that the server/service is not called to life over and over for each file. But I have noticed that my cpu sits far from full usage when the conversions are under way, and I sit on a bottomless pile of docx’s to convert to pdf, so I want to parallellize the batches to make it faster. Is there no way to make this happen? I am on linux, and using the parallel command does not help since the service is only processing one batch at once anyway, and not even multiple terminal instances can run the same command in parallel. Thank you
LibreOffice opens and saves files in a single thread. Additionally, LibreOffice makes sure to only run single process using a given user profile. Hence, whenever you use --convert-to without specifying different user profiles, even if you run these commands in parallel, all these calls will be serialized in one worker process.
Your only solution would be to use several user profiles. Then each profile would mean separate process, and so your CPU utilization (as well as memory load) would increase. See the -env:UserInstallation example in the Starting LibreOffice Software With Parameters.
Note however, that using a separate profile for every converted document for hundreds of files would likely not increase efficiency, but to the contrary, is likely to decrease it. If you would use some /tmp/random_name profile for each file, LibreOffice would need to startup, generate the new profile, do the conversion, and shutdown. The startup, profile creation, and shutdown overhead might cost more than the parallelization could offer. Knowing your CPU, create several directories with files to convert, according to the desired threads number, and issue one --convert-to command per each such directory in parallel.
2 Likes
Thank you very much for the explanation. Could you please elaborate on the last part? Are you suggesting that with something like python multiprocessing it is in fact possible to allow multiple --convert-tos running at the same time? It is my first time working with libreoffice so excuse my ignorance.
- Check your CPU: how many cores has it? Let’s assume 6.
- Have your large set of files split into 6 directories:
dir_to_convert => dir_to_convert
file1.doc subdir1
file2.docx file1.doc
file3.rtf file2.docx
... subdir2
file3.rtf
file4...
...
subdir6
...
- Issue 6 commands:
nohup soffice -env:UserInstallation=file:///tmp/test1 --convert-to odt dir_to_convert/subdir1/* &
nohup soffice -env:UserInstallation=file:///tmp/test2 --convert-to odt dir_to_convert/subdir2/* &
nohup soffice -env:UserInstallation=file:///tmp/test3 --convert-to odt dir_to_convert/subdir3/* &
nohup soffice -env:UserInstallation=file:///tmp/test4 --convert-to odt dir_to_convert/subdir4/* &
nohup soffice -env:UserInstallation=file:///tmp/test5 --convert-to odt dir_to_convert/subdir5/* &
nohup soffice -env:UserInstallation=file:///tmp/test6 --convert-to odt dir_to_convert/subdir6/* &
The /tmp/testN would be the directories with user profiles created separately for each invocation; these invocations will run independently.
3 Likes
hey sorry for the bother, this was working for a while then after a restart, it crashes with the following error:
User installation could not be completed.
javaldx failed!
Warning: failed to read path from javaldx
LibreOffice 7.3 - Fatal Error: The application cannot be started.
User installation could not be completed.
could I have somehow damaged my libre install?
Check if the directories with the temporary profiles are OK. Or better just delete them completely, and let LibreOffice re-create them from scratch.
That worked, thanks for the help. Also, is it not possible to similarly call macros with temporary profiles? I have a custom macro that takes three arguments, which is working as intended when run directly, but with a user profile specified it’s exiting without any error message:
soffice -env:UserInstallation=file:///tmp/test0 "macro:///Standard.Module1.del(0, 125, 0)"
Sorry for the tangent question, I’ve looked quite a bit for solution but it’s apparently quite a niche use case and nothing came up.
of course, with the »-env:UserInstallation=…« switch soffice creates a NEW UserConfigFolder from scratch, why do you expect there is a »…Standard.Module1.del« subroutine??
2 Likes
… and as a workaround for the issue explained by @karolus, you may put the macro into a (blank) document, and load it then call the macro in it with the clean profile:
soffice -env:UserInstallation=file:///tmp/test0 /path/to/my_file_with_macro.odt "macro://./Standard.Module1.del(0, 125, 0)"
note the difference in the macro: URI, with the extra dot meaning “in current document”.
2 Likes