Using v6.1.5.2(x64) Writer and the Insert > Envelope wizard, I select the #10 envelope size in the Format tab. When I start the print dialog the Print Preview shows the page size as a C5 Envelope, 6.38 in by 9.01 in. This is the format information that is sent to the printer which is incorrect, of course. I have found other mismatches in the Format tab of the Insert > Envelope wizard.
On Bugzilla you can see if there is already a bug entry. Otherwise, you can also enter a bug there yourself. Here on ask.libreoffice users help other users.