[Calc] I can't select non-adjacent cells when using solver. Am I doing something wrong?

I can select multiple cells that are next to each other by clicking and dragging or even shift clicking between them, but I can’t use ctrl+click to select cells that aren’t next to each other. I need to in order to select the cells that I want changed.

What you tried cannot work - on a superficial level simply because the first cell address in the respective field is selected text which will be overwritten on the next input. However, if you remove the selection by (e.g.) ‘CursorRight’ or a click behind the input before adding another cell, you will create invalid input: The needed delimiter (semicolon) is missing. The used type of input filed isn’t prepared to understand the Ctrl+Click method you can use in the sheet itself.
You need to enter something via the keyboard. The mouse isn’t the only input/pointing device after all.
Similar situations occur with Chart DataRanges and in additional cases.
You may look for a related bug of type ENHANCEMENT on bugs.documentfoundation.org , and probably file one yourself.
Amendment: I didn’t read the bug discussion yet, but tdf#53300 may be losely related (though I don’t feel sure about what an ‘expand button’ is).
That NEW bug is about 8 years old now. You should probably not be too confident concerning a quick reaction .