Introducing New LibrePythonista Extension. The power of python now inside of Calc

No, the logfile tells:

@Villeroy use the .deb packages downoaded from http://libreoffice.org/ and install via:

sudo dpkg -i  *.deb   #

this installs into path /opt/libreoffice…/ with its own python-interpreter inside the installation-path, and use by default the same userconfig-path as the packages from ubuntu installed via:

sudo apt-get install libreoffice

@Villeroy please try to install libreoffice from ubuntu.

hmm… thanks, That explains why I can’t get Ubuntu 20.04 to upgrade past LibreOffice 7.6.


BTW. LibrePythonista is mostly developed on Ubuntu 24.04 at this point and works great in that environment on Apt installed version, Snap version, FlatPak Version.

OK, I got LibrePythonista working on Ubuntu 20.04 with LibreOffice 7.6.

There was initially an error with pip installing that seems to be a Ubuntu 20.04 thing.
I had to run:

sudo apt-get install python3.8-distutils

Then LibrePythonista works fine.

Next I will attempt to upgrade to LibreOffice 24.x using the download .dev files and report back.

BTY: The error cause pip install to fail.

2024-10-03 13:11:09,492 - ERROR - libre_pythonista.install.pip_installers.default_installer: Traceback (most recent call last):
  File "/tmp/tmpf7a74q4z/get-pip.py", line 28425, in <module>
    main()
  File "/tmp/tmpf7a74q4z/get-pip.py", line 137, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/tmpf7a74q4z/get-pip.py", line 113, in bootstrap
    monkeypatch_for_cert(tmpdir)
  File "/tmp/tmpf7a74q4z/get-pip.py", line 94, in monkeypatch_for_cert
    from pip._internal.commands.install import InstallCommand
...

ModuleNotFoundError: No module named 'distutils.cmd'

I have isolated the issue. It is acutally embedded python of LibreOffice that is causing the issue.


Numpy installs so files such as _multiarray_tests.cpython-39-x86_64-linux-gnu.so but the embedded python can’t see it. It will only look for _multiarray_tests.cpython-3.9.so.


I ran into the same issue on MacOS and AppImage on Linux.
Never ran into this on a apt install before.


I have to do a bit of work to fix this issue. Will have it working in next release.

@Villeroy Okay, Version 0.3.1 of LibrePythonista is publish on Github. At the moment the LibreOffice Extension website is having issue and won’t show me the login page.


Should be able to just update your extension if LibrePythonist is already installed.


I had no idea that the Download from LibreOffice for Linux had embedded python until today.
I mades several changes to the installer. All my test show it working.

Does not make any difference.

…So lets start from scratch without exceedingly clever handcrafted »pip installers«

Assuming both your LO-python and the python3 from ubuntu runs in version 3.8.

Install the C-dependencies from terminal with:

sudo apt-get install python3-numpy
# update numpy in your user-sitepackages with ( NOT AS SU ! )
pip3 install numpy --user --upgrade --break-system-packages
# test with:
python3
…
import numpy

does it work? If yes…fine! …if no, show the traceback!
does it also work from LO with LibrePythonista Extension? If yes… excellent!
If not, search for some folder named »numpy« elsewhere inside the Extension-folder or …Scripts/python/ or …Scripts/python/pythonpath and remove it! Good luck!

pip3 install numpy --user --upgrade --break-system-packages
Traceback (most recent call last):
File “/home/andreas/.local/bin/pip3”, line 5, in
from pip._internal.cli.main import main
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/cli/main.py”, line 11, in
from pip._internal.cli.autocompletion import autocomplete
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py”, line 10, in
from pip._internal.cli.main_parser import create_main_parser
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py”, line 9, in
from pip._internal.build_env import get_runnable_pip
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/build_env.py”, line 18, in
from pip._internal.cli.spinners import open_spinner
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/cli/spinners.py”, line 9, in
from pip._internal.utils.logging import get_indentation
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/utils/logging.py”, line 29, in
from pip._internal.utils.misc import ensure_dir
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/utils/misc.py”, line 41, in
from pip._internal.locations import get_major_minor_version
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/locations/init.py”, line 66, in
from . import _distutils
File “/home/andreas/.local/lib/python3.8/site-packages/pip/_internal/locations/_distutils.py”, line 20, in
from distutils.cmd import Command as DistutilsCommand
ModuleNotFoundError: No module named ‘distutils.cmd’

ok searching about that spots: stackoverflow
Helps:

sudo apt install python3-distutils

??

OK, first I fulfill distutils, then I install pip:

pip3 install numpy --user --upgrade --break-system-packages
Requirement already satisfied: numpy in ./.local/lib/python3.8/site-packages (1.24.4)

Start Calc, click the Pythonista icon and get the exact same results as before. The button inserts =PY.C(SHEET();CELL("ADDRESS")) returning Err504, none of the menu commands does anything, except “Debug Log Module”, which dumps to the console:

2024-10-05 13:40:08,923 - ERROR - PyModule: Error initializing module
Traceback (most recent call last):
  File "/home/andreas/.config/libreoffice/4/user/uno_packages/cache/uno_packages/lu33326dm92w.tmp_/LibrePythonista(1).oxt/pythonpath/libre_pythonista_lib/code/py_module.py", line 71, in _init_mod
    exec(code, self.mod.__dict__)
  File "<string>", line 3, in <module>
  File "/opt/libreoffice24.2/program/uno.py", line 346, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/home/andreas/.local/lib/python3.8/site-packages/matplotlib/__init__.py", line 124, in <module>
    import numpy
  File "/opt/libreoffice24.2/program/uno.py", line 346, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/home/andreas/.local/lib/python3.8/site-packages/numpy/__init__.py", line 141, in <module>
    from . import core
  File "/opt/libreoffice24.2/program/uno.py", line 425, in _uno_import
    raise uno_import_exc
  File "/opt/libreoffice24.2/program/uno.py", line 346, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/home/andreas/.local/lib/python3.8/site-packages/numpy/core/__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/usr/bin/python3"
  * The NumPy version is: "1.24.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath' (or 'numpy.core._multiarray_umath.add_docstring' is unknown)
 (or '.core' is unknown)

Hm- lets try to tell LO using the python-interpreter of your ubuntu:

take a copy of /opt/libreoffice24.2/program/uno.py , insert as the first lines:

import sys, os
sys.path.append('/opt/libreoffice24.2/program')
os.putenv('URE_BOOTSTRAP', 'vnd.sun.star.pathname:/opt/libreoffice24.2/program/fundamentalrc')

move this copy to /usr/lib/python3/dist-packages/uno.py
also:

sudo cp  /opt/libreoffice24.2/program/unohelper.py /usr/lib/python3/dist-packages/unohelper.py

rename both files in /opt/… to uno.py_backup respectivly unohelper.py_backup

make also a backup of /opt/libreoffice24.2/program/pythonloader.unorc
and edit the content of the original pythonloader.unorc to some like:

[Bootstrap]
PYUNO_LOADER_PYTHONPATH=$ORIGIN
PYTHONHOME=file:///usr/lib/python3.8
PYTHONPATH=$PYTHONHOME $PYTHONHOME/site-packages $PYTHONHOME/lib-dynload $PYTHONHOME/lib-tk $ORIGIN

As I hinted earlier this is a machinery.EXTENSION_SUFFIXES issue. Not a diskutils issue or a Numpy issue. It has to do with Embedded Python in LibreOffice. It is the same for MacOS, and AppImage on Linux.


See LibrePythonista’s cpython_link.py file. It should shed some light on this subject.

Also I would like to note that when you install LibreOffice from the *.deb packages downloaded from the LibreOffice site on Ubuntu 20.04 it will result in two version of LibreOffice, that is if the original apt installed version has not been removed.


The current version of LibreOffice downoload from the LibreOffice website uses Embedded Python version 3.9. However, the default python version for Ubuntu 20.04 is Python 3.8.


So, if you try to pip install Numpy from the terminal you will install the version for Python 3.8 which will not work in Python 3.9 for several reasons. One of these reason is Python 3.8 and 3.9 have different binaries for Numpy.


I had to take all of this into account for LibrePythonista. So, LibrePythonista does a search and finds the correct python version to install Numpy and others with. This means in the embedded version of Python LibrePythonista will use the embedded python to install Pip packages, including Numpy. After the install LibrePythonista checks to see if any *.so file need system links creates so that the embedded python can find the binaries created during the install of packages such as Pandas and Numpy.


All this to say that LibrePythonista, current version, 0.3.1, should install just fine on the version of LibreOffice downloaded from the LibreOffice website. I tested this thoroughly on Ubuntu 20.04 and it works well.


It is also possible to side load numpy and other packages in to an embedded pyhon version of LibreOffice using my Python Path extension.


To sideload one would need to create a virtual environment on Ubuntu 20.04 (in this case) using Python 3.9. This might be possible using the embedded python of the downloaded LibreOffice Python or by using some manager such as pyenv. Once the virtual environment is created it can be then activated and Numpy can be installed. Now the Python Path extension can be used to link to the virtual environment where Numpy is installed. Although, I suspect the *.so issue would sitll need to be addressed for an embedded python version.


See my Pip & Virtual Environments guides for other refrerences.

$ python3 --version
Python 3.8.10
$ /opt/libreoffice24.2/program/python --version
Python 3.8.19

The second variable might be PYUNO_LOADER_PYTHONHOME and I assume that this file remains in the LO program directory.
Now I get this:

$ /opt/libreoffice24.2/program/soffice
Python path configuration:
  PYTHONHOME = '/usr/lib/python3.8'
  PYTHONPATH = '/opt/libreoffice24.2/program/../program'
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/usr/bin/python3'
  sys.base_prefix = '/usr/lib/python3.8'
  sys.base_exec_prefix = '/usr/lib/python3.8'
  sys.executable = '/usr/bin/python3'
  sys.prefix = '/usr/lib/python3.8'
  sys.exec_prefix = '/usr/lib/python3.8'
  sys.path = [
    '/opt/libreoffice24.2/program/../program',
    '/usr/lib/python3.8/lib/python38.zip',
    '/usr/lib/python3.8/lib/python3.8',
    '/usr/lib/python3.8/lib/python3.8/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f8fdaaa7140 (most recent call first):
<no Python frame>

the »pythonloader.unorc« is c&p from my debian-LO7.4.7.2 for arm64, so maybe your PYTHONHOME is different, eg. file:///usr/lib64/python3.8 ??
especially the PYTHONPATH could be complete different in your ubuntu!

Sorry, I’m here on arm64 ( raspberry 4 hardware ) Unfortunately, I have no way of doing this on real or virtual 86-64 hardware with the same environment as yours!

All the paths exist. I give up and revert to factory settings.

Ok, I downgraded LibreOffice to 24.2.6.2 on Ubuntu 20.04.
I installed LibrePythonista. Everything worked fine.
If installing make sure you have the latest versoin of LibrePythonista, currently 0.3.2 which was updated today, installed.


The install log file can be seen here.

Nope. This time the extension does not install the additional Python libraries, possibly because they have not been removed when I uninstalled the extension. The new version behaves exactly as the previous ones. How can I remove all the parts of your extension without resetting my profile?
P.S. Even with a new profile, the extension does not install properly.

For a fresh start without removing the profile, I usually uninstall the extension. This will remove all its settings. Close LibreOffice.


Next I navigate to the users python folder and find the correct python (3.8 in this case) and remove the site-packages directory. This removes all the installed Python packages.
For Ubuntu 20.04 and LibreOffice 24.2 the directory is usually located at ~/.local/lib/python3.8/site-packages. Now Just restart LibreOffice and you should be back.