Sharing data between queries and macros

Hi,

the LibreOffice Base program I am writing will be used on a shared server with multiple users. It is important therefore that no data specific to any one user gets stored in the permanent tables in the database. That said, I need to understand how to pass data from one query to another and to various macros, all within the same form, all relative to a specific user’s context.

For the implementation so far, just to get it working, I have setup a specific permanent table (tbl_SrchValues) that I read from the queries and write from the macros. This is okay for a stand-alone solution, but I now need something like a global variable that can be passed around.

I have read that I could implement Temporary Tables to share this data.

In the attached database, the main table is tbl_ASME, which contains data about collections of data called “volumes”, identified by VOL in the table. This table is read-only. (In the real table I have about 1000 volumes). The other fields contain data about the specific volume, such as whether it has been photographed (FOTO), what kind of data (TIPO) is contained (baptisms, weddings, deaths etc), which church (CHIESA) was involved, date span etc.

A number of concatenated queries are run on tbl_ASME, and the user can input values for these queries. For the logic of the queries I need to pass information between the queries and between a number of macros. This is accomplished in this example by using table tbl_SrchValues, which is written and read and keeps track of where we are.

You can ignore the other tables e.g. tbl_1, tbl_26 etc, as they are used in the second part of the program to visualise photographed data within an identified volume. You can also ignore tables tbl_Tipo and tbl_Chiesa.

I now need to eliminate tbl_SrchValues and find some other mechanism to share the data.

(Additional information, added 11 Jan 2019)

Hi, I have come back to this problem again. I have tried to use the temporary tables handled by Firebird. This would have the advantage of letting me keep all the present table queries and updates, I just have to substitute my tables with temporary ones. This is easier said than done!

I have been unable in fact to get them to work, even though I have managed to create a temporary table, but, despite defining its structure in the SQL CREATE request, when it is created there is no structure, the defined fields do not exist.

The Firefird description of how these tables work is a bit cryptic to me, they also talk about “commit”, which I believe only happens when the .odb files are closed.

Anyway, can you help with some simple SQL code that I can use in a macro that will let me Create a temporary structure, write and read some data into it, and then get rid of it when the user finishes? This is the code I am presently using, but although the table is created, its structure is not.

Sub create_temporary_table
Dim oDatasource As Object
Dim oConnection as Object
Dim oSQL_Statement As Object
Dim strSQL As String
oDatasource = thisDatabaseDocument.CurrentController
IF NOT (oDatasource.isConnected()) THEN oDatasource.connect()
oConnection = oDatasource.ActiveConnection()
oSQL_Statement = oConnection.createStatement()
strSQL = "CREATE GLOBAL TEMPORARY TABLE ""tbl_test"" "
strSQL = strSQL & "(""ID"" INTEGER, ""Field"" INTEGER)"
oSQL_Statement.execute(strSQL)

End sub

test.odb

Hello,

You do not specify the database used on the server. For example, with Firebird server see this page → Global Temporary Tables (GTTs) as a possibility.

Have I missed something fundamental here? (Excuse my ignorance!) I was hoping to simply port a LibreOffice test.odb from Windows where it is now working on embedded Firebird, to a Lubuntu system, still with embedded Firebird. I have not yet got Firebird working, but does you question imply something more complex than I have imagined? By the way, your link to the GlobalTemporary Tables seems a perfect solution (if I could only get Firebird to work!)

Probably. Embedded databases are not multi-user capable. They are complete within the .odb of Base.

You have stated this is to be on a shared server with multiple users accessing the data and presuming one location for the data accessed by all. This means the database must operate in server mode. Not possible with embedded databases.

In a multi-user environment, each user would either have their own copy of the Base file or it would be accessible by them through something like a document server.

You are right, my idea was for each user to have their own copy of the base file, but so far I have been unable to implement this in LibreOffice, as I had to use fixed tables in the database to share information between queries and macros. I had mistakenly thought that I could still use an embedded database, being as it was read-only, but you are right that I need for that data to be on a server, which means I have to structure the LIbreOffice program differently to connect to the database of the Lubuntu server. Then hopefully I can set up a temporary shared area that is local to each user’s context, so that the queries and macros can share information. The Firebird temporary tables link you sent would probably enable this, but if I have to use a database server, then it would make more sense for me to use the MariaDB database on the Lubuntu machine. I need to think about this some more!

Personally I would use Firebird or PostgreSQL server. Both appear to have Temp tables and available in 32-bit for Linux:

https://www.postgresql.org/download/linux/ubuntu/

https://www.postgresqltutorial.com/postgresql-temporary-table/

But this is a choice you will need to make. I have a few DB servers installed & use PostgreSQL for personal applications - others for answering questions & testing here.

Should also note, do not have any recent experience with the 32-bit versions. Did work with PostgreSQL in 32-bit many years ago. My current systems are 64-bit. As you have seen with Java, 32-bit can present other problems.

I have added more information in the original text, I cannot get the temporary tables to create properly

Hello,

My tests (first use of temp table) shows this to work quite well except for one small glitch (describe later).

It appears your test was done using an embedded database and also don’t see a primary key or commit line in your SQL.

As noted in the link provided, a temporary table holds the data for the session in place - the immediate connection being used. In an embedded DB there can only be the one session.

Edit:

Note that I have been able to do the same in a Firebird embedded DB. Main difference is that it only displays the current session and does not actually show the different records when there are multiple sessions. It does demonstrate when closing the session the data is removed. Also the table name is listed but cannot be accessed - same as in server (but no table name even listed).

End Edit

So in order to test with this, Firebird server needs to be installed and I use FlameRobin as a GUI front end for maintenance. With this in place you can execute multiple sessions to see the effects of the temporary table.

Now I don’t see the value of the macro to create this table. Like any other table you create it once (at least in Firebird server) - it is the data for each session which becomes unique. This is explained in the first paragraph of the link provided.

Used this to create the table (can do from Base or FlameRobin):

CREATE GLOBAL TEMPORARY TABLE “tbl_test” (“ID” int not null primary key, “Field” int) on commit preserve rows;

Then added a record in FlameRobin:

image description

Next opened Base connected to this DB and queried “tbl_test” and it was empty as it should be. It is a different session.

image description

So added a transaction using:

INSERT INTO “tbl_test” (ID, “Field”) VALUES (‘1’, ‘6384679’);

Resulting in:

image description

On the left is FlameRobin session and the Base session on the right - each with its’ own data for the table. If a session is closed and the a new one started the data is gone from the previous session. The data remains only while the session is open.

Now the glitch (probably a bug). While there is no problem with SQL to enter/modify data, the temporary table is NOT listed in the Table section of Base. Have not discovered any method to display. Should not be a concern when using macros as it seems you have, and this is easily accessed through the server where it resides. The user (of the Base file) should not need this.

What may present a problem in this instance is if you need to use this tables fields in a form - seems not accessible but again some short macro code will overcome this.

Edit #2:

Went another step. Modified your sample to use a temp table. Changes made:

Deleted table “tbl_SrchValues” .

Create temp table:

CREATE GLOBAL TEMPORARY TABLE "tbl_SrchValues" (
      "ID" int not null primary key,
      "SrchVolValue" BIGINT,
      "FoundVolValue" BIGINT,
      "SrchPhotoValue" INT,
      "SrchDateValue" INT,
      "SrchImageValue" VARCHAR(20),
      "SrchTypeValue" VARCHAR(255),
      "SrchChurchValue" VARCHAR(255))
 on commit preserve rows;

Attached macro to Open Document event of the Base file to set initial record in temp table. Basically used your macro and simply changed the CREATE statement to an UPDATE statement.

Copied your form and pasted with 1 attached. This is the working form and uses the temp table.

Added query for temp table and is used in subForm _SrchValues

Sample ----- TempTableTest.odb

A quick look at the PostgreSQL documentation seems to indicate a different process there. Will try testing this later.

Edit:

A quick test in PostgreSQL shows the need to create the temporary table each time a new session is started. The table structure is also not available under Tables nor in the Form the same as with Firebird.

Also on the ‘commit’ comment, that is only for Firebird embedded. Other DB’s don’t function that way. And it occurs anytime you save Base or issue a ‘commit’ command - not when the Base file is closed.

this is amazing Ratslinger, thankyou so much for taking the time to sort this out for me. LibreOffice has been a difficult learning curve for me, but thanks to you, I am getting there!
The modified version you have sent me solves the immediate problem of using temporary tables to enable data sharing between queries and macros. So although my question is effectively solved, I will leave it open for a short time while I try out the Firebird Server and FlameRobin, just in case I need to ask you something else, and being as you have been so kind as to give examples of how they work. Thankyou again for your time and patience.

@nexturejohn,

You are welcome.

Please understand. This is a question and answer site. Since this question has been answered, if you have another question ask as a new question. You can always reference another posting when asking a question. Questions asked within other questions get buried from others and make answers even more difficult to find.

Thank you.