LibreOffice Calc 24.2.7.2 won't ask for password when opening encrypted XLSX

[kniffte]

Hello,
using Java 21/POI, we’re creating encrypted XLSX files. The encryption settings were taken from POI’s tutorial at Apache POI™ - Encryption support - section " XML-based formats - Encryption":

try (POIFSFileSystem fs = new POIFSFileSystem()) {
    EncryptionInfo info = new EncryptionInfo(EncryptionMode.agile, CipherAlgorithm.aes192, HashAlgorithm.sha384, -1, -1, null);
    Encryptor enc = info.getEncryptor();
    enc.confirmPassword("foobaa"); // of course, we're using a different password, in this case, it has 6 characters

    // Read in an existing OOXML file and write to encrypted output stream
    // don't forget to close the output stream otherwise the padding bytes aren't added
    try (OPCPackage opc = OPCPackage.open(new File("..."), PackageAccess.READ_WRITE);
        OutputStream os = enc.getDataStream(fs)) {
        opc.save(os);
    }

    // Write out the encrypted version
    try (FileOutputStream fos = new FileOutputStream("...")) {
        fs.writeFilesystem(fos);
    }
}

The file is then send to me via email (usually, it’s for external users, but I’m currently developing something here).
Now, this file can opened using Excel 365 on multiple machines w/o any issues.
But LibreOffice doesn’t even dare to ask for password, it handles it as if it was a CSV file - asking for details on how to open it:

Any ideas, why it is failing? Before those changes, we could open encrypted XLS files.

Version: 24.2.7.2 (X86_64) / LibreOffice Community
Build ID: 420(Build:2)
CPU threads: 12; OS: Linux 6.8; UI render: default; VCL: kf5 (cairo+xcb)
Locale: de-DE (de_DE.UTF-8); UI: en-US
Ubuntu package version: 4:24.2.7-0ubuntu0.24.04.3
Calc: threaded

Thanks a lot.
kniffte

Update: here’s a sample file created using above code:
audit_28.xlsx (8.5 KB)

[mikekaganski]

I couldn’t download the sample XLSX file that you attached to your question, to check locally

[kniffte]

Hello @mikekaganski ,

sorry, I had to organize a sample data first.

I’ve attached the file to the original post.

Thanks + Sorry.
kniffte

[mikekaganski]

Thanks; but the password foobaa that you mentioned in the code in the question doesn’t fit the file. Additionally, the file is an XLS, but for some reason, it’s called XLSX - is that intentional?

@kniffte

Before those changes, we could open encrypted XLS files.

Which changes were meant? I have tested this file using LibreOffice up to version 4.0, and all are handled the same way.

[kniffte]

Hello @mikekaganski ,
you’re right, I’m sorry.
First, here’s the password (verified it using Excel): 03530
Second: we used to provide XLS (Excel 97-2003) files, that were encrypted using RC4. The change(s) I was talking about:

1. switch to XLSX format (using XSSFWorkbook rather than HSSFWorkbook as before)
2. switch from RC4 encryption to EncryptionInfo(EncryptionMode.agile)

Based on your comments, I wanted to provide a couple examples for you - and now I found a working solution using EncryptionMode.agile:

    public Path generateFileAndGetPath() throws IOException {
        Path outputFile = FileService.createRandomTempFile("xlsx");
        try (XSSFWorkbook wb = new XSSFWorkbook();
            java.io.ByteArrayOutputStream bos = new java.io.ByteArrayOutputStream();
            java.io.FileOutputStream fos = new java.io.FileOutputStream(outputFile.toFile())) {

            /* create content */
            Sheet sheet = wb.createSheet("Vehicles");
            Row titleRow = sheet.createRow(0);
            titleRow.createCell(0).setCellValue("My Content");
            wb.write(bos);

            /* encrypt workbook */
            ByteArrayOutputStream encryptedWorkbook = encryptWorkbook(bos.toByteArray(), "ABC");

            /* write content to tmp file */
            fos.write(encryptedWorkbook.toByteArray());
        } catch (Exception e) {
            log.error("Error while creating Excel file", e);
            throw new IOException("Error while creating Excel file", e);
        }
        return outputFile;
    }
    
    private ByteArrayOutputStream encryptWorkbook(byte[] workbookData, String password) throws Exception {
        try (ByteArrayOutputStream bos = new ByteArrayOutputStream();) {
            /* Create a new POIFS file system */
            POIFSFileSystem fs = new POIFSFileSystem();

            /* Initialize encryption information with agile mode */
            EncryptionInfo info = new EncryptionInfo(EncryptionMode.agile);

            // Create an encryptor and set the password
            Encryptor encryptor = info.getEncryptor();
            encryptor.confirmPassword(password);

            /* Write the encrypted workbook data to the POIFS file system */
            try (OutputStream os = encryptor.getDataStream(fs)) {
                os.write(workbookData);
            }

            /* Save the encrypted file system to the specified file path */
            fs.writeFilesystem(bos);
            return bos;
        }
    }

The problem: originally I had the EncryptionInfo set to:

EncryptionInfo info = new EncryptionInfo(EncryptionMode.agile, CipherAlgorithm.aes192, HashAlgorithm.sha384, -1, -1, null);

and that seems to be invalid. Changing it to

EncryptionInfo info = new EncryptionInfo(EncryptionMode.agile);

solved the issue.
Maybe somebody else has an issue with encrypted XLSX files.

Thanks.
kniffte

[mikekaganski]

Thanks; filed tdf#166241 for the original file.