Ask LibreOffice - RSS feedhttps://ask.libreoffice.org/en/questions/Questions and answers for LibreOfficeenWed, 14 Apr 2021 15:29:16 +0200high precision, calculate correction valuehttps://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/ hi @all,
a question: '5 minutes' represented as a fraction of a day (represented by '1') are an 'odd' 'uneven' - 'endless fraction' value, already in decimals, and also in doubles (can be calculated as '=time(0;5;0),
thus there is a gap between the value calc can deal with ('=1/24/60*5' or '=1/24/12', or '=1/288', all resulting in '0,00347222222222222') and the ideal precise value of 0,003472222222222222222222222222222... , something like 0,000000000000000002222222222222222... or - as calc has one digit more hidden from users view: 0,000000000000000000222222222222222....
i can! calculate that gap quite precise to 1.9274705288631189921192130137104208E-19 with the 128 bit variant of 'weitz' (www.weitz.de/ieee), doe's anybody have an idea how i can get that value in a calc sheet? wouldn't need 35 digit precision, 15 or 16 digits acc. normal double is sufficient, just 10 digit could be enough.
i know the value cannot be added to the 5 minute value as it's beyond it's ULP, but if one can get such correction values i see some nice possibilities to do correct calculations.
let's say: 1/288 is a fraction which we know, exact, we can check that calc calculates with something like '0,0034722222222222220294, how to calculate the difference 'in the sheet'?
(sorry, i am! somewhat overworked, normally i should know such things by myself ... let me change the question ... what is the best - most elegant - way do get such values ?)Wed, 14 Apr 2021 00:49:46 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/Answer by Hagar Delest for <p>hi @all, </p>
<p>a question: '5 minutes' represented as a fraction of a day (represented by '1') are an 'odd' 'uneven' - 'endless fraction' value, already in decimals, and also in doubles (can be calculated as '=time(0;5;0), </p>
<p>thus there is a gap between the value calc can deal with ('=1/24/60*5' or '=1/24/12', or '=1/288', all resulting in '0,00347222222222222') and the ideal precise value of 0,003472222222222222222222222222222... , something like 0,000000000000000002222222222222222... or - as calc has one digit more hidden from users view: 0,000000000000000000222222222222222.... </p>
<p>i can! calculate that gap quite precise to 1.9274705288631189921192130137104208E-19 with the 128 bit variant of 'weitz' (<a href="http://www.weitz.de/ieee">www.weitz.de/ieee</a>), doe's anybody have an idea how i can get that value in a calc sheet? wouldn't need 35 digit precision, 15 or 16 digits acc. normal double is sufficient, just 10 digit could be enough. </p>
<p>i know the value cannot be added to the 5 minute value as it's beyond it's ULP, but if one can get such correction values i see some nice possibilities to do correct calculations. </p>
<p>let's say: 1/288 is a fraction which we know, exact, we can check that calc calculates with something like '0,0034722222222222220294, how to calculate the difference 'in the sheet'? </p>
<p>(sorry, i am! somewhat overworked, normally i should know such things by myself ... let me change the question ... what is the best - most elegant - way do get such values ?)</p>
https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?answer=304079#post-id-304079Numbers in most spreadsheets are limited to 15 significant digits. If you need better precision, don't think about spreadsheet, that's all.
See: [[Solved] Import 20-digit ICCID numbers from CSV](https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=104466).
To show the community your question has been answered, please [tag the best answer](https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_mark_an_Answer_as_correct.3F) (![](https://ask.libreoffice.org/m/default/media/images/vote-accepted-on.png)), and/or [vote for any helpful answer](https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_vote.3F). Else, please edit your question to [add information](https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_add_to.2Fclarify_my_question) (answers are for solutions only).Wed, 14 Apr 2021 07:33:45 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?answer=304079#post-id-304079Comment by Hagar Delest for <p>Numbers in most spreadsheets are limited to 15 significant digits. If you need better precision, don't think about spreadsheet, that's all.</p>
<p>See: <a href="https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=104466">[Solved] Import 20-digit ICCID numbers from CSV</a>.</p>
<p>To show the community your question has been answered, please <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_mark_an_Answer_as_correct.3F">tag the best answer</a> (<img src="https://ask.libreoffice.org/m/default/media/images/vote-accepted-on.png" alt="">), and/or <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_vote.3F">vote for any helpful answer</a>. Else, please edit your question to <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_add_to.2Fclarify_my_question">add information</a> (answers are for solutions only).</p>
https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304150#post-id-304150No idea.
You can ask the question in the forum also (see link in my profile), there are Calc gurus also.Wed, 14 Apr 2021 15:29:16 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304150#post-id-304150Comment by newbie-02 for <p>Numbers in most spreadsheets are limited to 15 significant digits. If you need better precision, don't think about spreadsheet, that's all.</p>
<p>See: <a href="https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=104466">[Solved] Import 20-digit ICCID numbers from CSV</a>.</p>
<p>To show the community your question has been answered, please <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_mark_an_Answer_as_correct.3F">tag the best answer</a> (<img src="https://ask.libreoffice.org/m/default/media/images/vote-accepted-on.png" alt="">), and/or <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_vote.3F">vote for any helpful answer</a>. Else, please edit your question to <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_add_to.2Fclarify_my_question">add information</a> (answers are for solutions only).</p>
https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304124#post-id-304124@Hagar Delest: i know :-( ... and that's what i'm researching for: which errors are unavoidable, and which can be circumvented,
in nowadays life there is hardly another choice than spreadsheets for plenty tasks, and i'd like them to calculate as precise as possible ... for that: do you have any idea to calculate that small deviation?Wed, 14 Apr 2021 13:51:51 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304124#post-id-304124Comment by Hagar Delest for <p>Numbers in most spreadsheets are limited to 15 significant digits. If you need better precision, don't think about spreadsheet, that's all.</p>
<p>See: <a href="https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=104466">[Solved] Import 20-digit ICCID numbers from CSV</a>.</p>
<p>To show the community your question has been answered, please <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_mark_an_Answer_as_correct.3F">tag the best answer</a> (<img src="https://ask.libreoffice.org/m/default/media/images/vote-accepted-on.png" alt="">), and/or <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_vote.3F">vote for any helpful answer</a>. Else, please edit your question to <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_add_to.2Fclarify_my_question">add information</a> (answers are for solutions only).</p>
https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304099#post-id-304099My point is that spreadsheets are not the best tools if you need such precise calculations. Even if you store a correction value, using them may trigger similar calculation limitations.Wed, 14 Apr 2021 10:45:57 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304099#post-id-304099Comment by newbie-02 for <p>Numbers in most spreadsheets are limited to 15 significant digits. If you need better precision, don't think about spreadsheet, that's all.</p>
<p>See: <a href="https://forum.openoffice.org/en/forum/viewtopic.php?f=9&t=104466">[Solved] Import 20-digit ICCID numbers from CSV</a>.</p>
<p>To show the community your question has been answered, please <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_mark_an_Answer_as_correct.3F">tag the best answer</a> (<img src="https://ask.libreoffice.org/m/default/media/images/vote-accepted-on.png" alt="">), and/or <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_vote.3F">vote for any helpful answer</a>. Else, please edit your question to <a href="https://wiki.documentfoundation.org/Ask/Getting_Started#How_do_I_add_to.2Fclarify_my_question">add information</a> (answers are for solutions only).</p>
https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304090#post-id-304090@Hagar Delest: this limit in precision often accumulates and then affects digits further ahead, which is annoying for some tasks. Against this i need the correction value. Since calc can! store the correction value (calc also can deal with much smaller numbers) and also can work with it i think there might be a trick to get it calculated ...Wed, 14 Apr 2021 09:38:32 +0200https://ask.libreoffice.org/en/question/304044/high-precision-calculate-correction-value/?comment=304090#post-id-304090