Just for documentation: @caolan answered this to me via #libreoffice-dev (06.12.2021, 10:57):
it is a gtk4 GtkWindow in --enable-gtk4, it doesn’t have client side decorations, except in RTL mode where it does actually use a GtkHeaderBar, but makes it looks like it doesn’t
so vcl/unx/gtk3/gtkframe.cxx has a gtk_header_bar_new new in it, but guarded with some conditions, remove the conditions and play around with that area
and there is no .ui file yet.
Will investigate from there on and see how far I can get