How can I import from Excel without the cruft that slows down Calc?

I have an .xlsm with about 10,000 rows, many of which have colors. The formulas are very simple - SUM and SUMIFS. No conditional formatting.

If I import that into an ODS, it’s so slow I filed a bug against Calc. I’ve removed all macros, but the imported file is still very slow to open.

Is there a way to import that Excel spreadsheet without whatever cruft causes the slowness? If I copy/paste just the values, without formatting, the ODS opens fast, but I need to keep the colors.