How to import one Python in-file module from another?

One thing to remember: APSO hides pythonpath. See:

I’m very low-level with all this … but searching on this question I had come across mention of APSO. Are you saying my setup has added or incorporated APSO without me knowing it? Because I haven’t consciously added it…

I see you didn’t actually provide an answer to my question… so does that mean you don’t know of a way (even without APSO)?

@mrodent
It has nothing to do with APSO except for the lack of this capability within.
.
If you go through that entire post you would see I even included a working sample → Using APSO to embed scripts in python, how to import functions from one module to another? - #5 by Ratslinger

1 Like

About 12 years ago I started in a very similar way to embed python in …ods or …odt, but later I decide todo especially the embedding with the help of css.ucb.SimpleFileAccess:

from pathlib import Path
def embed(py_file="embed_function.py"):
    createUnoService = XSCRIPTCONTEXT.ctx.ServiceManager.createInstance
    sfa = createUnoService("com.sun.star.ucb.SimpleFileAccess")
    protocol = "vnd.sun.star.tdoc:/" #transient uri-prefix
    py_folder = "/Scripts/python/"
    p = Path(py_file)
    p_uri = p.absolute().as_uri()
    doc = XSCRIPTCONTEXT.getDocument()
    uid = doc.RuntimeUID
    sfa.createFolder(f"{protocol}{uid}{py_folder}")
    sfa.copy(p_uri, f"{protocol}{uid}{py_folder}{p}")

This way the »bookkeeping« into »manifest.xml« is done by the »sfa-service« automatically!

(Nowadays from an jupyter notebook dev-environment I wrote a patch for ~/.local/lib/python3.9/site-packages/IPython/core/magics/osm.py
with that patch I can use simply the ipython-magic:

%%writefile -d doc example.py

( »doc« is here already the symbol for some (…ods, …odt or …? ) )
into the top-line of a code-cell, to embed the content of the cell into the embedded »doc«…/Scripts/python/example.py

1 Like

@ratslinger

Thanks. I saw your answer first time. Now I’ve run it (i.e. opened the .ods).


Any chance you could show me step by step how you would actually recommend putting together such a zip (.odt, .ods, etc.) file (with a Python module which imports another). In my case this would be an .odb (Base), but an .ods or anything would be fine.


Did you actually have to edit the manifest.xml and/or something else? Obviously I have expanded the file and tried to understand why yours works but mine doesn’t but I don’t quite get it.


Do I need to get hold of APSO and learn how to use it? Having looked at what it is, I’m not quite clear why I need it for these packaging purposes.

@karolus
Thanks for this script. But there’s not quite enough info here for me to understand: this script appears to have to be run in a UNO context: XSCRIPTCONTEXT, etc. Does it run in the file into which you want to import the file embed_function.py? But doesn’t your script itself then have to be embedded? In which case this looks like a sort of puzzling chicken-and-egg situation. I’ll do some experiments.


More significantly, this doesn’t answer my question and appears to be off-topic. It’s nice to be told of a better way of packaging things up, but my specific question is about importing Python modules.

With my comment I wanted to give you an idea how to solve the embedding of a file in a more robust, faster and convinient way.
I have edited the example so that it can also be called from Libreoffice.
Your other question »howto made embedded stuff import-able« was already anwered by @vib. To repeat in my words: »It’s tedious, error-prone and not worth it.«

XSCRIPTCONTEXT is the (python) global Variable into the LO-object-hierachie from inside LO.

From outside (probably in some Development-context) you may create the same NameSpace:
via pre-run:

import uno
from pythonscript import ScriptContext
"""
the following code requires a running instance of soffice with the optional argument:
"--accept=pipe,name=apracadapra;urp;StarOffice.ComponentContext"
"""

PIPENAME = 'apracadapra'
local = uno.getComponentContext()
resolver = local.ServiceManager.createInstance("com.sun.star.bridge.UnoUrlResolver")
ctx = resolver.resolve( "uno:pipe,"
                           f"name={PIPENAME};"
                           "urp;"
                           "StarOffice.ComponentContext")
createUnoService = ctx.ServiceManager.createInstance

XSCRIPTCONTEXT = ScriptContext(ctx, None, None)

@karolus
OK thanks, I understand a bit better now. But did you look at ratslinger’s example? He certainly seems to have cracked the import problem… download and examine Using APSO to embed scripts in python, how to import functions from one module to another? - #5 by Ratslinger

if you look into the topic, you’ll probably see my comment there… :wink:

@mrodent

.
Will try to do so soon. Sorry but currently have some physical issues to deal with. All the links are presented - that is how I got it working although I may have a bit more experience with digging into the internal files.
.
I do suggest you look into the methods presented by @karolus as his knowledge in this and python itself is far greater than mine. I know I will be trying this myself soon.

@karolus
“dirty hacks”. Fair enough. But it also seems foolproof, and the fault is with LibreOffice for not allowing this standard Python behaviour.

But in fact when I talk about importing modules I don’t just mean user-built modules. With all sorts of useful available pip install-type modules one might want to use there is a need to import. The other option presumably being not to bundle up with the file, but to put them somewhere external in the PYTHONPATH. I’m presuming it’s possible to call a non-embedded package from an embedded Python script… but have yet to attempt that and find out what works.

And I haven’t even mentioned virtual environments… could you call an external project which needed a VE? (that’s for another question).

I imagine the justification is that this is a specialised environment meant primarily for fairly simple macro manipulations of Base (or Writer, Calc, etc.) objects …

Turns out it is much easier to work with python, LibreOffice and Vritual Environment for more complex (multi-script) projects then python macros in many cases.

Once the Virtual Environment is set up, which is a bit tricky because uno.py must be added to the environment in some way. On windows this usually means tricking the virtual environment into thinking it is running under LibreOffice. In Linux it is a simple matter of creating a link to uno. I wrote a short guide on this for OOO Development Tools (OooDev), see dev guide

I also wrote a tool OOOENV to aid in managing ‘uno’ and virtual environments.

Once all of that is done it is super simple to run python scripts for LibreOffice using OooDev.

There are many examples of this in the LibreOffice Python UNO Examples repository.

It basically only take a few lines of code to start LibreOffice.

Example:

loader = Lo.load_office(Lo.ConnectSocket())
doc = Calc.open_doc(fnm=self._fnm)
GUI.set_visible(is_visible=True)

See the Garlic Secrets as an example.

1 Like

Wow, that’s a really exciting way of doing things (if I’ve understood correctly what you’re describing there). Amazing. I have so much to learn! :upside_down_face:

I assume it still won’t enable me to control the font size in the header row for the columns of data grids in Base forms (only joking: of course this is beyond the power of the gods themselves).

Haha, You never know.
I just recently added hundreds on new classes to OooDev for the purpose of formatting.
I am busy documenting all these new classes. You can see an example for the Calc Borders.

Basically anything you can do in a format dialog you can accomplish using these new classes.

@vib
I’m having a bit of trouble understanding what to do with this Python VE approach. NB this is W10.

I got as far as > oooenv env -u. This does not give the sought output but

Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = ‘D:\apps\Python\PyQt5;’
program name = ‘D:\My documents\software projects\EclipseWorkspace\lo_base_automation.venv\Scripts\python.exe’
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = ‘D:\My documents\software projects\EclipseWorkspace\lo_base_automation\.venv\Scripts\python.exe’
sys.base_prefix = ‘’
sys.base_exec_prefix = ‘’
sys.executable = ‘D:\My documents\software projects\EclipseWorkspace\lo_base_automation\.venv\Scripts\python.exe’
sys.prefix = ‘’
sys.exec_prefix = ‘’
sys.path = [
‘D:\apps\Python\PyQt5’,
‘’,
‘D:\My documents\software projects\EclipseWorkspace\lo_base_automation\.venv\Scripts\python38.zip’,
‘D:\apps\Python\Python3.8.5\Lib\’,
‘D:\apps\Python\Python3.8.5\DLLs\’,
‘D:\apps\Python\Python3.8.5\Lib\’,
‘D:\apps\Python\Python3.8.5\DLLs\’,
‘D:\apps\LibreOffice\LibreOffice_7.4.6\program’,
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named ‘encodings’

Current thread 0x00003de0 (most recent call first):

… which doesn’t look too promising. (And I then find that > python fails with the same error…).

Incidentally, I don’t know why Python 3.8.5 is being used there: LO 7.4.6 is using 3.8.16.


And as it happens, pyenv-win caused a security error in PowerShell (which I know nothing about), so I used a quick and dirty method of downloading 3.8.16 for W10 and then copying the files python38.dll and pythonw.exe to the location under …\LibreOffice_7.4.6\program so that I could create and activate a VE. Could this have something to do with it?


My OS is using 3.10.5 so I’m mystified about how 3.8.5 crept in there. Incidentally I know nothing whatsoever about poetry.


Could we maybe chat in a separate discussion location as it doesn’t seem to relate to the original question here?

Poetry is simalar to a Package managar linke pip and much more. There is a small learning curve to getting started. After that I find way easier to manage packages, publish packages, install virtual environmen autmotically etc.

Yes you should send me direct message on this subject and we can report back here in some form if we come up with anything interesting.

I need to know step by step how you have created your virtual environment.

LibreOffice python macros do not work well with importing multiple files.
It will work best if you precompile your python scripts into a single file and then use it as a macro.

I created LibreOffice Script Compiler for this purpose. I use it heavily with the OOO Development Tools (OooDev) project.

It is capable of compiling many, many scripts into a single file and automatically embeding the single script into a LibreOffice document.

Here is a quick example.

1 Like

Thanks… you seem to know quite a bit about this.

Did you try at any point to add an entry to sys.path, as it stands when a script file starts to run? I just printed it out at the beginning of my macro script file, and there is indeed no path entry there which would permit that script to import a “sibling” entry in the Scripts/python path inside the .odb file (which is a zip file, obviously).

I also just expanded my doctored .odb and found that it had two “manifest.xml” files which it needed to unpack, in the same location (/META-INF). That’s not the case with the original .odb, which suggests to me that I’m not packing things up properly with the above utility script.

You probably know that in the Java world there is such a thing as an “executable .jar”, whereby the normal structure of modules and imports works, in a compressed file context. I’ve heard of “egg” and “wheel” files in Python … might these be relevant to this issue?

I remember I tried many different hacks to get multi scripts working.
IIRC LibreOffice changes that paths internally, so, manipulating the sys.path did not work out.

I tired so many hacks and nothing I could come up with was even close to satisfactory so I created the Script Compiler.

If nothing else I would LibreOffice Script Compiler to at least compress you many scripts into one script. Having it automattically embed into a LibreOffice document is optional.

If you find a satisfactory work around I would really like to hear about it.

Will try out Script Compiler, definitely.