Is this a geometrical/mechanical constraint with your printer tray or a procedural question with LibreOffice?
Edit your question (don’t use an answer for something which is not an answer) to provide more information: printer brand and type, OS, LO version is a bare minimum.
If your tray can accommodate slightly larger format than A4, you can try defining custom size pages. Another suggestion: as long as stationery can travel through the printer undamaged, don’t bother yourself with the exact dimensions; use A4 in Writer and trust the printer for imaging your formatted document in an A4 window on the envelope (you may need to test where this window is “anchored” on the envelope).