The following Basic function inserts csv data into a HSQL database table. The same (or similar) code should work with any database engine supporting text tables linked to csv files.
The code performs 4 steps:
- Disconnect any existing csv file from the text table
- Overwrite the linked csv file with a given new csv file, for instance with a downloaded csv picked by a file picker dialog.
- Reconnect the text table with the csv file.
- Run
INSERT INTO "DataTable" (SELECT * FROM "View)
where “View” refers to a view (a SELECT statement) which does all the conversion work.
The view
- arranges the column order to match the data table’s column order.
- converts comma decimals to point decimals and date/time strings into ISO date/time strings.
- eliminates any duplicates already existing in the data table.
- whatever needs to be done to fulfill data integrity requirements.
The function returns the count of inserted rows.
Arguments:
- Database connection
- URL of the file to be imported.
- Name of the text file in the database directory.
- Name of the text table linked to the text file.
- Name of the view.
- Name of the target table.
Function ImportCSV(oConnection, sURL$, sTextFile$, sTextTable$, sView$, sDataTable$) As Long
sqlSET ="SET TABLE """& sTextTable & """ SOURCE "
oStmt1 = oConnection.prepareStatement(sqlSET & "OFF")
oStmt2 = oConnection.prepareStatement(sqlSET & "ON")
sqlINSERT = "INSERT INTO """& sDataTable &""" (SELECT """& sView &""".* FROM """& sView &""")"
'print sqlINSERT
oStmt3 = oConnection.prepareStatement(sqlINSERT)
b = oStmt1.execute()
filecopy sURL, cDatabase_Path & sTextFile
b = oStmt2.execute()
b = oStmt3.executeUpdate()
ImportCSV = b
End Function
A macro calling the above function may look like this:
Sub my_import(ev)
Const cCaption = "Import my csv" 'file picker caption
Const cDownloadPath = "file:///home/villeroy/Downloads/" 'file picker path
Const cDatabase_Path = "file:///home/villeroy/hsql/database/" 'database path
Const cLabel = "Text table" 'label of file picker filter
Const cPattern = "*accounting*.csv" ' file picker filter
Const cCSV = "My_Import.csv" 'name of text file
Const cTextTable = "My_Linked_TextTable"
Const cView = "My_Import_View"
Const cDataTable = "Database Table"
sURL = pickFile(cCaption, cDownloadPath, cLabel, cPattern)'file picker function
REM get connection from calling form control:
frm = ev.Source.Model.Parent
conn = frm.ActiveConnection
x = ImportCSV( _
oConnection:=conn, _
sURL:=sURL, _
sTextFile:=cCSV, _
sTextTable:=cTextTable, _
sView:=cView, _
sDataTable:=cDataTable _
)
Msgbox x &" records imported."
End Sub
P.S. This is how I used to do the same thing without macro code:
- Close the office suite.
- Replace the text file with the new text file.
- Open the database document.
- Copy the view icon over the data table icon and confirm the import dialog. All the details are handled by the view.
P.P.S. A demo: Apache OpenOffice Community Forum - [HSQL, Base, Basic] Macro Driven CSV Import - (View topic)
Download csv_import.odb and import.csv, open the database, the form, click the button. Click the same button a second time. It will not import the same data twice.