<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p>Hi Calvin,</p>
<p><br>
</p>
<p>That is not related to QGIS, but to the way the CPU represents floating points numbers internally.</p>
<p>You can see the same behaviour in the Python console (or any Python interpreter, and surely almost any other programming language that uses doubles): </p>
<p>```<br>
</p>
<p>>>> v = <span>-0.1556640799496235</span><br>
</p>
<p>>>> print(f'{v:.20f}')</p>
<p></p>
<pre>-0.15566407994962350170</pre>
```
<p></p>
<p>The reason those extra digits are not '0' is because doubles are not stored in base 10, and anything beyond their standard ~16 significant digits is approximated. For a more detailed explanation you can take a look at Python documentation, they have a clear
explanation on the topic (<a href="https://docs.python.org/3/tutorial/floatingpoint.html" class="OWAAutoLink" id="LPlnk342318" previewremoved="true">https://docs.python.org/3/tutorial/floatingpoint.html</a>).
<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>If you really, really, really need 20 significant decimal places (that seems an extremely rare situation, see
<a href="https://xkcd.com/2170/" class="OWAAutoLink" id="LPlnk170925" previewremoved="true">
https://xkcd.com/2170/</a> ) you should be looking at your code really carefully, as you need a larger (in number of bits) than usual floating point type; I think that neither QGIS (or standard Python or C++) support it out of the box.</p>
<p><br>
Regards,</p>
<p>Javier<br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>De:</b> QGIS-Developer <qgis-developer-bounces@lists.osgeo.org> en nombre de C Hamilton <adenaculture@gmail.com><br>
<b>Enviado:</b> lunes, 15 de marzo de 2021 15:50:23<br>
<b>Para:</b> qgis-developer<br>
<b>Asunto:</b> [QGIS-Developer] Strange Behavior with QgsProcessingParameterNumber - Bug?</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>In a processing algorithm I have a double QgsProcessingParameterNumber where I have set the number of decimals metadata to be 20.</div>
<div><br>
</div>
<div>param = QgsProcessingParameterNumber(<br>
'NUMBER',<br>
tr('Number'),<br>
QgsProcessingParameterNumber.Double,<br>
defaultValue=0,<br>
optional=False)<br>
param.setMetadata({'widget_wrapper': { 'decimals': 20 }})<br>
self.addParameter(param)<br>
</div>
<div><br>
</div>
<div>In the processing popup, if I enter a number with less than 20 digits then the rest of the digits become some random number. For example if I enter</div>
<div><br>
</div>
<div>-0.1556640799496235 <br>
</div>
<div><br>
</div>
<div>the displayed result is</div>
<div><br>
</div>
<div>-0.15566407994962350170<br>
</div>
<div><br>
</div>
<div>I would think that if it pads with extra digits to fill out to 20, the digits should be 0. Have I exceeded the number of acceptable decimal digits, or is this a bug?</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Calvin</div>
</div>
</div>
</div>
</body>
</html>