This key combination is a toggle that “Displays or hides the formulas instead of the values in all cells.” (Search for the back-tick on that page to see the instructions/explanation.)
This is a known bug: fdo#103388. At least since January it has been registered as an issue for OSX. (And I can confirm!) [ Update - As noted by @vsfoote, this has been fixed, and will be available when you get v. 6.0.3+ for OSX. I’m still on 6.0.1, though.]
There is a workaround, but it is very much USE AT YOUR OWN RISK
Open LibO Preferences (
Open the first category,
LibreOffice > Advanced,
Open Expert Configuration.
in the “search” bar in the new pop-up, type: “toggleFormula” and hit
ENTER. This will bring up a function that includes
Double-click that line, and clear the value
.uno:ToggleFormula from the box that pops up.
OK (for each of the three pop-up windows).
Now ⌘` will work on the system level, and you’ll be able to use that key combination to switch between Calc windows.
It is a simple matter to reverse this operation, of course, but you now need to search on
QUOTELEFT_MOD1 to find that function.