Here are a couple links from Filemaker for version 13 specifically. It appears .csv and dBase are export options. I can verify that both formats are compatible with LibreOffice as I recently had to do a similar migration.
Supported import/export file formats
Exporting data from FileMaker Pro
I found the easiest way to get data into Base is to first import into Calc and go from there. However, your process will vary depending on which database format you choose. Base can either use its internal database format (not highly recommended), or you can hook it up to a separate backend like MySQL, Maria or PostrGres (highly recommended). I recommend the separate backend because an internal database is more likely to get corrupted if LibreOffice experiences some kind of error. I am currently working on a fairly sophisticated MySQL backend database system and am enjoying the benefits of such. Of course this information will probably just open more questions for you if you aren’t already familiar with how to do all that stuff.
I’m not well-versed with Filemaker, but I recently did something even crazier… I just exported ANCIENT files from Q&A for DOS! Ha ha. It was actually kind of fun but challenging due to the crazy old formats Q&A uses.