Issues when using loop to open file

English
, , ,
#1

Good Morning,

I have an issue when opening OGD files from a folder using a loop. There is nothing wrong with the files. I can open them manually/individually with no problem. However when i make a macro to open the files, analyze them and close them, then inevitable some problem occurs, like LO unable to read the file due to XML problem, or even LO crashes. This is true whether I write the code in Basic or Python. Here is my basic code:

Option Explicit
Dim dtp As Object
Dim doc As object
Sub Main
Dim strPath As String, strFileName As String

	dtp= GetProcessServiceManager().createInstance("com.sun.star.frame.Desktop")
	strPath ="K:\folderpath\odg\"
	strFileName = Dir(strPath, 0)
	Do While (strFileName <> "")
		strFileName = Dir()
		If strFileName Like "*odg" Then
			doc = getDoc(strPath & strFileName)
			closeIt(doc)
		End if
	Loop
End Sub

Function getDoc(fName As String) As Object
Dim _doc As Object
Dim url As String
Dim docs As Object
On Error GoTo check_window:
	url = ConvertToUrl(fName)
	_doc =dtp.loadComponentFromURL(url, "_default", 0, Array()) 'Crash/error occurs here
	'Sometimes the problem (which does not raise an error) is that the document is open, but _doc variable is not set.
	'When this happen then i check for which DrawingDocuments are open and set the 
	'_doc variable to that instance
	If isDrawing(_doc) then
		getDoc=_doc
		Exit function		
	Else
check_window:
	'If an error is raised, then again i go through the same procedure of check which if 
	'any drawing documents are open and set the _doc variable to that instance
		docs = dtp.getComponents().createEnumeration()
		While docs.hasMoreElements()
			_doc = docs.nextElement()
			If isDrawing(_doc) Then
				getDoc=_doc
				Exit function
			End if
		Wend
	End If

End Function
Function isDrawing(doc)
	If doc Is Nothing Then
		isDrawing = false
	Else
		isDrawing = doc.SupportsService("com.sun.star.drawing.DrawingDocument")
	End if
End Function

Sub closeIt(_doc As Object)
	If isDrawing(_doc) Then
		_doc.close(false)
		_doc.dispose()
		_doc=Nothing
	End If
End sub

The version I use is:
Version: 7.3.1.3 (x86) / LibreOffice Community
Build ID: a69ca51ded25f3eefd52d7bf9a5fad8c90b87951
CPU threads: 8; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: CL

#2

Without actual testing, I would immediately suspect some OS/filesystem caching artifacts - i.e., your following Dir sees not-yet-removed lockfile (but that shouldn’t match the mask: its name is like .~lock.orig_filename.odg#, so strFileName Like "*odg" should fail); but trying to read it fails, because the file is being destroyed … ?

#3

Hello,

The files are not destroyed. When I exit LO and reopen files individually, there is no problem.

#4

I wrote about “destroyed temporary lockfiles”, not destroying normal drawing files.

But as I mentioned, my theory doesn’t explain why such files match the mask.

#5

Well it does not touch the lock files, since the name of the lock files do not match the mask like “*odg”

#7

Indeed - that was what I mentioned already twice.

Yet, I would still think about some issues with the dynamically changing directory content; and my first test would be creating file list (array of names) using Dir first, then using that list to iterate, not Dir between open/close.

#9

Okay I will check that.

#6

Can’t understand the two consecutive assignments. Did you probably intend
strPath = strPath & "01-Projects\prognosis_picks\preprocessed\odg\"
in the second place?

#8

okay sorry for that double entry. there should be only one line for assignment to strPath.

strPath ="K:\folderpath\odg\"
#10
  • Well, does the code work now as expected?
  • Was the other line not actually contained in the code you tried to run?
  • Did you try to debug your code in the Basic IDE?
  • Did you use the Watch panel to make sure the used variables have the expected values (point to the intended objects) at a specific step?
  • Please clarify your comment above.
#11
  • the code has the problems as I described
  • the line strPath =“K:\folderpath\odg” is a dummy that I am using because i am not at liberty to use my real file path when posting problems on t his forum
  • I regularly use the debugger
  • I regularly use the watch panel
#12

There is an obvious issue with your Sub closeIt(_doc). However, it wouldn’t crash the application when run from the IDE in single-step mode, but report an error.
Since you claim, to be familiar with the debugger and the watch, you will also know that you should report any crash or error precisely identifying the command last given when the error occurred, and all the messages you got.
I will not continue with an analysis while I’m still stabbing in the dark.

#13

Hello,

This is the latest modification with the code:

REM  *****  BASIC  *****
Option Explicit
Dim dtp As Object
Const 	strPath="K:/folderpath/odg/"


Sub Main
Dim fList() As Variant
Dim fInd As Integer
Dim docs As Variant
	setupLibs()

	SF_Exception.Console(false)
	SF_Exception.ConsoleClear(0)

	dtp= GetProcessServiceManager().createInstance("com.sun.star.frame.Desktop")	
	fList=fileList()
	For fInd=lbound(fList) To ubound(fList)
			docs = getDocs(fList(fInd))
			closeIt(docs)
			whatever()
	Next fInd
	dtp=Nothing

End Sub

sub whatever()
Dim time1, time2

time1 = Now
time2 = Now + TimeValue("0:00:01")
    Do Until time1 >= time2
        DoEvents
        time1 = Now()
    Loop

End sub

Function fileList() As Variant
Dim arr As Variant : arr = Array()
Dim strFileName As String

	strFileName = Dir(strPath, 0)
	Do While (strFileName <> "")
		strFileName = Dir()
		If strFileName Like "*odg" Then
			arr = SF_Array.Append(arr, strPath & strFileName)
		End If
	Loop
	fileList= arr
End Function


Sub loadLibrary(daLibs As Variant)
Dim fIndex As Integer
Dim daLib As String
	with GlobalScope.BasicLibraries
		For fIndex = lbound(daLibs) To ubound(daLibs)
			daLib= daLibs(fIndex)
			if not .isLibraryLoaded(daLib) Then
				.loadLibrary(daLib)
			End If
		Next fIndex
	End With 
End Sub

sub setupLibs()
Dim libs As Variant
	libs = Array("ScriptForge","Access2Base")
	loadLibrary(libs)	
End Sub

Function getDocsFromDtComponent() As Variant
Dim docs As Object, _doc As Variant : _doc = Array()
Dim tDoc As Object
	docs = dtp.getComponents().createEnumeration()
	While docs.hasMoreElements()
		tDoc = docs.nextElement()
		If isDrawing(tDoc) Then
			_doc = SF_Array.Append(_doc, tDoc)
		End If
	Wend
	docs = Nothing
	getDocsFromDtComponent=_doc
End function


Function getDocs(fName As String) As Variant
Dim _doc As Variant : _doc = Array()
Dim tDoc As Object
Dim url As String
On Error GoTo get_window
	url = ConvertToUrl(fName)
	SF_Exception.DebugPrint("attempting to open file", fName)
	SF_Exception.ConsoleToFile("D:\temp\temp.log")	
	tDoc = dtp.loadComponentFromURL(url, "_default", 0, Array()) 'Crash/error occurs here
	If tDoc Is Nothing Then
		getDocs = getDocsFromDtComponent()
		SF_Exception.DebugPrint("opening file indirectly", fName)
		SF_Exception.ConsoleToFile("D:\temp\temp.log")	
	else
		_doc = SF_Array.Append(_doc, tDoc)
		getDocs = _doc
		SF_Exception.DebugPrint("file opened directly", fName)
		SF_Exception.ConsoleToFile("D:\temp\temp.log")	
	End If
	Exit Function
get_window:
	SF_Exception.DebugPrint("Error ", Err , ": " , Error$ , "At line : " , Erl ,  Now )
	SF_Exception.ConsoleToFile("D:\temp\temp.log")	
	'getDocs = getDocsFromDtComponent()
End Function

Function isDrawing(doc)
	If doc Is Nothing Then
		isDrawing = false
	Else
		isDrawing = doc.SupportsService("com.sun.star.drawing.DrawingDocument")
	End if
End Function

Sub closeIt(_docs As Variant)
Dim _doc As Object
Dim fInd As Integer
	For fInd=lbound(_docs) To ubound(_docs)
		_doc = _docs(fInd)
		_doc.close(false)
		_doc.dispose()
		_doc=Nothing
	Next fInd
End Sub

I have made changes which should result in some kind of error message, however at the key point where i know an err would occur, LibreOffice will crash. It is on

tDoc = dtp.loadComponentFromURL(url, "_default", 0, Array()) 'Crash/error occurs here

It happens intermittently, and at one point i did manage to see this:

std::bad_alloc

So I guess I will leave this issue alone. I will check this link:

QA/BugReport/Debug Information - The Document Foundation Wiki
for more information.

Thank you for your help

#14
  • How can it “happen intermittently” if the error causes a crash?

From my limited understanding: Such an error isn’t thrown by the error handling system of LibO but by the core code directly. It tells that an attempt to allocate (reserve) memory (RAM) failed. It shouldn’t be suppposed to be related to the Loop or to the running Sub at all. See std::bad_alloc - cppreference.com

  • What happens if you open the respective file directly (FileOpen dialog or doubleclick in the respective folder)?
  • How much RAM does it need when opened directly (not from the Sub)?
#15

I just ran the code, and by the time it reaches the ninth file, which is 4891 kilobytes, it crashed again. However I am able to open the file from the LO interface, without any problem.

From the windows task manager here is the screenshot:

image
image899×314 52 KB

#16

It looks as if the other files didn’t get unloaded, and so the out-of-memory resulted.

But use of 32-bit LibreOffice itself looks odd on a 64-bit system (which is an own question, not saying that 32-bit should crash - just it is more easily crashed when much data is loaded, which shouldn’t happen here anyway). I see that it’s already mentioned in the initial post. Any specific reason that you use 32-bit, not 64-bit one?

Note that you loose the first file this way :wink: - move the strFileName = Dir() to the end of the loop, right before Loop.

FTR: I tried this code using Version: 7.4.0.1 (x64) / LibreOffice Community
Build ID: 43e5fcfbbadd18fccee5a6f42ddd533e40151bcf
CPU threads: 12; OS: Windows 10.0 Build 19044; UI render: default; VCL: win
Locale: en-US (ru_RU); UI: en-US
Calc: CL

on a directory with 128 copies of an ODG. It run fine, without any errors, with task manager fluctuating soffice.bin memory use between ~160 MB and ~190 MB.

Any reason that you use 7.3.1.3, which was obsoleted already at the beginning of March, when 7.3.2 bugfix release appeared; current bugfix release is 7.3.5?

#17

Okay. Long story. Yes I do have a Windows 64 bit workstation. Company IT security policies are very strict so I can not install software which require admin privileges without prior approval, and the default answer from IT security has been, “don’t ask us anything, we tell you everything”. Since this policy does not extend to software which not requiring admin privileges, I installed the portable-app version of LO, which as far as I can tell, exists as only 32-bit.

Regarding the files not unloading, here is my code:

Sub closeIt(_docs As Variant)
Dim _doc As Object
Dim fInd As Integer
	For fInd=lbound(_docs) To ubound(_docs)
		_doc = _docs(fInd)
		_doc.close(false)
		_doc.dispose()
		_doc=Nothing
	Next fInd
End Sub

So theoretically they SHOULD unload.

I will try to download the latest and greatest. Thank you for your advice.

#18

Indeed; the point was, that it could be a bug.

Impressum (Legal Info) | Datenschutzerklärung (Privacy Policy)   Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.