There is some complicated behavior related to this. Here’s what I’ve identified on my system (4.1.4.2-4.fc19 on Fedora). You should be able to get the results you want using the following description, but it is clumsy and non-intuitive.
First, here’s the official description: Importing and Exporting CSV Files - LibreOffice Help
In my experience, the first time I save a file as CSV, I am given the “Export Text File” prompt, and I can select whatever delimiter I want (confirmed with gEdit). However, if I open the file to edit it, I do not see that prompt the next time that I save. For subsequent saves, tabs are used, even if I had used commas originally.
I think this may be connected to the “Text Import” dialog choices. If I import the text with multiple delimiters selected, then the next time it is saved with tabs. However, if I deselect the non-comma delimiters, then it saves with comma delimiters. My impression is that the default is to import with comma, tabs, and semicolons as delimiters.
If I want to specify the delimiters (again), I’ve found that I can select the “Edit Filter Settings” checkbox on the save dialog. After clicking OK and then confirming that I do really want to save the file with the selected name and format, the “Export Text File” prompt will show up again. My guess is that this is the recommended way to select the delimiter, though I found this to be non-intuitive and clumsy.