Calling Python urllib functions in Basic

fcsw · September 3, 2025, 4:49am

Microsoft Windows Version 22H2 (Win10)

Version: 25.2.5.2 (X86_64) / LibreOffice Community
Build ID: 03d19516eb2e1dd5d4ccd751a0d6f35f35e08022
CPU threads: 2; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded

I’m setting up an app that makes http HEAD requests. The http client that comes with Windows 10 and the http client in the OAuth2OOo extension are blocked or banned at some URLs. Curl gets around these blocks & bans somehow, but has to be called in Shell(), which creates other problems. I’ve tried changing the User-Agent header in the Win & OAuth2OOo clients to “curl/8.13.0” but that doesn’t create any improvement.

So I’m trying to test the Python urllib library. According to the comments in the request.py file, “The simplest way to use this module is to call the urlopen function, which accepts a string containing a URL or a Request object…” Since I know nothing about Python, I’m interested in simple things.

So…

Sub TestUrllibUrlopenFunction
	Dim objScriptProvider	As Object
	Dim objRequest			As Object
	Dim objResponse			As Object
	Dim strUri				As String
	
	objScriptProvider = ThisComponent.getScriptProvider
	strUri = "vnd.sun.star.script:urllib/request.py$urlopen?language=Python&location=user"
	objRequest = objScriptProvider.getScript(strUri)
	objResponse = objRequest.invoke(array("https://atun-rsia.org/resources"), array(), array())
End Sub

This results in

BASIC runtime error.
An exception occurred 
Type: com.sun.star.uno.RuntimeException
Message: Error during invoking function urlopen in module file:///C:/Users/user/AppData/Roaming/LibreOffice/4/user/Scripts/python/urllib/request.py (<class 'AttributeError'>: module 'http.client' has no attribute '_create_https_context'
  File "C:\Program Files\LibreOffice\program\pythonscript.py", line 921, in invoke
    ret = self.func( *args )
  File "C:\Users\user\AppData\Roaming\LibreOffice\4\user\Scripts\python\urllib\request.py", line 184, in urlopen
    _opener = opener = build_opener()
  File "C:\Users\user\AppData\Roaming\LibreOffice\4\user\Scripts\python\urllib\request.py", line 565, in build_opener
    opener.add_handler(klass())
  File "C:\Users\user\AppData\Roaming\LibreOffice\4\user\Scripts\python\urllib\request.py", line 1363, in __init__
    context = http.client._create_https_context(http_version)
).

OK, try the other simple thing.

Sub TestUrllibRequestObject
	Dim objScriptProvider	As Object
	Dim objRequest			As Object
	Dim objResponse			As Object
	Dim strHeaders			As String
	Dim strUri				As String
	
	objScriptProvider = ThisComponent.getScriptProvider
	strUri = "vnd.sun.star.script:urllib/request.py$Request?language=Python&location=user"
	objRequest = objScriptProvider.getScript(strUri)
	strHeaders = "If-Modified-Since: Mon, 01 Sep 2025 00:00:00 GMT"
	objResponse = objRequest.invoke(array("https://atun-rsia.org/resources"), _
		array("headers={" & strHeaders & "}"), array("method=HEAD"))
End Sub

Result:

BASIC runtime error.
An exception occurred 
Type: com.sun.star.uno.RuntimeException
Message: Couldn't convert <ooo_script_framework.Request object at 0x000002D5C93CAEF0> to a UNO type; caught exception: <class 'AttributeError'>: 'Request' object has no attribute 'getTypes', traceback follows
no traceback available
.

What am I doing wrong?

karolus · September 3, 2025, 5:55am

the whole urllib is part of the the python-standard-libraries, why do you try to copy&paste the whole folder-tree into …/Scripts/python?
why do you »sophisticate« your workflow with basic -calls via ScriptProvider.invoke(…) ?

karolus · September 3, 2025, 5:41am

sounds like, YOUR »request.py« try to import http.client… … but there is some »http.py« nearby YOUR request.py which shadows the original which should be somewhere in ~~…/lib/python…/http/~~ ? sorry, in your case propably …/libreoffice/program/pythonx.y/http/
So find the wrong »http.py« and remove or rename it.

fcsw · September 3, 2025, 8:44am

Partial solution: wrong version. The Python core that is part of LO is 3.10. I installed the 3.13 version of urllib. When I installed 3.10, TestUrllibUrlopenFunction stopped producing errors. So that part is fixed. But TestUrllibRequestObject is still producing the same error.

It’s 4:30 AM. I’m fine but my computer is getting sleepy.

sokol92 · September 3, 2025, 10:32am

I propose a slightly more complex training setup.
Cell B3 of the attached file contains the request URI.
Cell B4 contains text in JSON format.
When you click the Run Macro button, a macro is launched that performs a POST request of the “application/json” type and places the result (a JSON format string) in cell B5.
Basic:

Sub HttpPost()
  Dim oSheet as Object, pyScript As Object
  Dim retval As String, url As String, jsonData As String
  pyScript = ThisComponent.getScriptProvider().getScript("vnd.sun.star.script:" & _
             "utils.py$http_post?language=Python&location=document") 
  oSheet = ThisComponent.Sheets(0)
  url = oSheet.getCellbyPosition(1, 2).String
  jsonData = oSheet.getCellbyPosition(1, 3).String            
  
  retval=pyScript.invoke(Array(url, jsonData), Array(), Array())
  oSheet.getCellbyPosition(1, 4).String = retval
End Sub

Python:

# coding: utf-8
import urllib.request
import urllib.parse
import json

def http_post(url, str_json):

    try:
        data_json = json.loads(str_json)
    except Exception as e:
        return f"** Error: {e}"
    data_encoded = json.dumps(data_json).encode("utf-8")
    req = urllib.request.Request(url, data=data_encoded, method="POST")
    req.add_header("Content-Type", "application/json")

    try:
        with urllib.request.urlopen(req) as response:
            return response.read().decode("utf-8")
    except Exception as e:
            return f"** Error: {e}"

HttpPostJson.ods (16.3 KB)

karolus · September 3, 2025, 11:18am

why do you so?

sokol92 · September 3, 2025, 11:21am

Fixed the error (confused strings and JSON objects), restored the message.

fcsw · September 4, 2025, 4:35am

(post deleted by author)

fcsw · September 4, 2025, 4:37am

@Sokol92 Is this utils.py script documented anywhere?

karolus · September 4, 2025, 4:52am

yes, its just the python-snippet from @sokol92 above! its also embedded in his example:

HttpPostJson.ods (16.3 KB)

fcsw · September 4, 2025, 8:54am

@Sokol92 I didn’t realize that the python code was the function called in the Basic code. So I pulled the utils file out of the .ods file & …oh. OK, got it.

I rewrote your python function thusly:

import urllib.request

def http_head(url, mod_date):

    req = urllib.request.Request(url, method="HEAD")
    req.add_header('If-Modified-Since', mod_date)

    try:
        with urllib.request.urlopen(req) as response:
            return response.read()
    except Exception as e:
            return f"** Error: {e}"

I rewrote my Basic sub thusly:

Sub TestUrllibUrlopenFunction
	Dim objScriptProvider	As Object
	Dim objRequest			As Object
	Dim strUri				As String
	Dim strResponse			As String
	
	objScriptProvider = ThisComponent.getScriptProvider
	strUri = "vnd.sun.star.script:httpHead.py$http_head?language=Python&location=user"
	objRequest = objScriptProvider.getScript( strUri )
	strResponse = objRequest.invoke( array( "https://wiki.documentfoundation.org/Macros/Python_Guide/Introduction", "Mon, 01 Sep 2025 00:00:00 GMT" ), array(), array() )
MsgBox strResponse
End Sub

It runs w/out errors, but it isn’t returning anything. I should be getting a “304 Not Modified” status. What am I missing?

Edit: Never mind. I changed a couple of double quotes to single quotes & got the 304. Not bad for hacking in notepad.
I’m crediting my own post w/ the solution, because that’s the post w/ the solution, but the the credit should go to Sokol92 for supplying a python function that someone who knows nothing about python could easily modify.