In LibreOffice Calc (Version 7.3.7.2), the font size in the formula bar is set by the default system settings for your operating system. You can change this in your operating system settings for Windows, Mac, or Linux. You may need to restart LibreOffice for the font change to take hold.
In an X-desktop environment linux based system like mine (running Xubuntu) this is changed in Settings: Settings Editor: xsettings: Font Name.