With 5.4.0.3 on Windows and Meera regular font, I can confirm the first problem. For example, when a line ends in U+0D26U+0D46 there is extra space. Perhaps when the glyphs are drawn in reverse order, the engine fails to determine where to add spacing.
Apache OpenOffice 4.1.3 does not show extra space. LibreOffice introduced a new layout engine starting in 5.3, as explained at https://wiki.documentfoundation.org/ReleaseNotes/5.3#Text_Layout. One of the main goals was to use a single engine across all platforms, rather than Uniscribe only on Windows.
Kartika, the font I normally use for Malayalam, seems to be worse than Meera. Nirmala UI is somewhat cleaner, but Meera looks better. Anyway, changing to a different font does not fix the problem.
I also tried to reproduce the other issues you described but could not. U+002E, U+002C and U+200D seemed to behave correctly. Perhaps you could edit the question and provide example data. Not just a screenshot, but actual Unicode text. However, the answer is probably still the same.
To solve these problems, use a version before 5.3. Or, reportedly in 5.3 but not 5.4 and newer, it is possible to disable the new engine with the SAL_NO_COMMON_LAYOUT environment variable. As mentioned in comments, it may also help to file a bug.
EDIT:
Your example document clearly demonstrates the problem with U+200D. In recent versions of LibreOffice, the word വാല്ക്കണ്ണാടി (vālkkaṇṇāṭi – U+200D is the fifth character in this word) can be incorrectly split up by end-of-line wrapping.
…വാല്
ക്കണ്ണാടി
The problem does not occur in Apache OpenOffice 4.1.3, so it seems you must use an older version. These are significant problems for anyone who uses Malayalam script, so I recommend that you file a bug report and include this example document.
