## Solved: How to avoid that math problem?

rin67630
Posts: 378
Joined: Fri Mar 04, 2016 10:15 am

### Solved: How to avoid that math problem?

Hi,
I have got a strange behaviour.
Sometimes the math of Python3 returns a weird value, that displays badly in tkinter.

Code: Select all

``````  #!/usr/bin/python3
# dB Meter software for Wensn
# by Mikael Bendiksen (c) 2015
import os, pty, serial
import decimal
import tkinter
import sys
import usb.core
import sched, time
from decimal import *
from tkinter import *

dev = usb.core.find(idVendor=0x16c0, idProduct=0x5dc)
root = Tk()
dB = 0
labelText = StringVar()
labelText.set("0")
getcontext().prec = 2

print ("Starting dB Meter by Mikael Bendiksen")

def update():
ret = dev.ctrl_transfer(0xC0, 4, 0, 0, 200)
global dB
dB = (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30
AK = 50 + (2*dB)
print (dB)
print (AK)
labelText.set(dB);
t = Timer(1.0, update)
t.start();

update()

root.title("dB Metering")
root.geometry('{}x{}'.format(600, 600))

w = root.winfo_screenwidth()
h = root.winfo_screenheight()
rootsize = tuple(int(_) for _ in root.geometry().split('+')[0].split('x'))
x = w/2 - rootsize[0]/2
y = h/2 - rootsize[1]/2
root.geometry("%dx%d+%d+%d" % ((400, 200) + (x, y)))

root.configure(background='black')

Label(root,
text = "dB Level",
bg = "black",
fg = "red",
font = "Helvetica 35 bold").pack()

Output = Label(root,
textvariable = labelText,
fg = "light green",
bg = "black",
font = "Helvetica 72 bold")
Output.pack()

root.mainloop()
``````
returns sometimes these values:
45.3
47.400000000000006
44.5

How can i avoid this and get a clean 47.4 displayed?
I have read that the decima library should return "clean results".
But if I modify
dB = (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30 to
dB =decimal( (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
I get an error:
Traceback (most recent call last):
File "devdB.py", line 35, in <module>
update()
File "devdB.py", line 26, in update
dB = decimal((ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
TypeError: 'module' object is not callable

What does that mean and how to cure it?
Thank you.
Last edited by rin67630 on Sat Mar 23, 2019 8:53 pm, edited 2 times in total.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

### Re: How to avoid that math problem?

Round it to one decimal place during the print.
In C this would be "%.1f", I am very sure there is something comparable in Python.

ericcooper
Posts: 123
Joined: Sat Apr 08, 2017 6:23 pm

### Re: How to avoid that math problem?

It's not the math, but the fact that floating point values are represented in binary. So some values will be different than if they were represented in decimal.

You can control how floating point numbers are printed using Python's format operations.

For example, to display only one decimal place, you can use

Code: Select all

``print(f'{db:.1f}')  # using Python3 'f' strings``
or

Code: Select all

``print('%.1f' % db)  # using printf-style formatting``

rin67630
Posts: 378
Joined: Fri Mar 04, 2016 10:15 am

### Re: How to avoid that math problem?

ericcooper wrote:
Sat Mar 23, 2019 4:46 pm
It's not the math, but the fact that floating point values are represented in binary. So some values will be different than if they were represented in decimal.

You can control how floating point numbers are printed using Python's format operations.

For example, to display only one decimal place, you can use

Code: Select all

``print(f'{db:.1f}')  # using Python3 'f' strings``
or

Code: Select all

``print('%.1f' % db)  # using printf-style formatting``
Thank you, but the print instructions are only for debugging.
The real output should occur in tkinter. I have published the whole code in the initial post.
How could I get a clean display in the tkinter window?

Regards
Laszlo

plugwash
Forum Moderator
Posts: 3433
Joined: Wed Dec 28, 2011 11:45 pm

### Re: How to avoid that math problem?

rin67630 wrote:
Sat Mar 23, 2019 4:22 pm
I have got a strange behaviour.
Sometimes the math of Python3 returns a weird value, that displays badly in tkinter.
Welcome to the world of floating point maths.

Most decimal fractions cannot be expressed exactly as binary floating point numbers, so rounding is inevitable when converting from textual representations of a number to binary floating point.

Binary floating point numbers can be converted exactly to decimal, however this is usually not desirable, it usually just results in many digits of false precision. So languages must choose a policy for rounding numbers on conversion to decimal. Many languages just round to a fixed number of digits, but this tends to break round-tripping, either you end up with numbers that gain false-precision when you convert them from text to binary and back to text or you end up with numbers that lose precision when you convert them from binary to text and back to binary.

So since version 3.2 python defaults to rounding to the shortest representation that will preserve round-trip conversion for both str and repr .

Getting back to your program, 0.1 cannot be represented exactly as a floating point number, so it is rounded to the closest double-precision floating point number ( 0.1000000000000000055511151231257827021181583404541015625 ) then it is multiplied and the result is again rounded to fit in a double precision floating point number. Then 30 is added and again there may be further rounding. Finally the number is converted to text for display.

Much of the time you get lucky, the result you get from your floating point calculation is the same as the result you would get from converting your "expected" result to a floating point number. Frequently these multiple roundings combine to the same effect as a single rounding of the final result would have. So python's "shortest round trip" policy gives you the result you expect, but sometimes things don't work out that way.
rin67630 wrote:
Sat Mar 23, 2019 4:22 pm
How can i avoid this and get a clean 47.4 displayed?
There are a few options.

1. Specify an explicit number of digits when converting from floating point to string for display.

Code: Select all

``````dB = (174) * 0.1 + 30
print ("%0.1f" % dB)
``````
2. Use a type that can represent 0.1 exactly.

Code: Select all

``````from decimal import Decimal
dB = (174) * Decimal("0.1") + 30
print (dB)
``````
3. Use division by 10 instead of multiplication by 0.1 and rearrange the calculation so that the devision is the last step of your calculation meaning things are only rounded once. I don't like this option because it's rather fragile, a seemingly innocuous change to the calculation could break it.

Code: Select all

``````dB = ((174) + 300) / 10
print (dB)
``````

ghp
Posts: 1391
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact: Website

### Re: How to avoid that math problem?

But if I modify
dB = (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30 to
dB =decimal( (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
I get an error:
Traceback (most recent call last):
File "devdB.py", line 35, in <module>
update()
File "devdB.py", line 26, in update
dB = decimal((ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
TypeError: 'module' object is not callable
decimal is used as a module in your program (import decimal). When using this as a function "decimal(...)", you get the Type Error.
I am not familiar with this decimal module, but possibly using decimal.Decimal(...) instead of decimal(...) could avoid the problem.

ericcooper
Posts: 123
Joined: Sat Apr 08, 2017 6:23 pm

### Re: How to avoid that math problem?

You can pass the formatted string to the labelText.set() method instead of the floating point value.

rin67630
Posts: 378
Joined: Fri Mar 04, 2016 10:15 am

### Re: How to avoid that math problem?

ghp wrote:
Sat Mar 23, 2019 5:18 pm
I am not familiar with this decimal module, but possibly using decimal.Decimal(...) instead of decimal(...) could avoid the problem.
That syntax solved the error, but the results got worse. Now 2/3 of all values have plenty of decimals.

Andyroo
Posts: 3770
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

### Re: How to avoid that math problem?

Try the round function in Python.
e.g.

Code: Select all

``round(3.14159,2)``
Need Pi spray - these things are breeding in my house...

rin67630
Posts: 378
Joined: Fri Mar 04, 2016 10:15 am

### Re: Solved: How to avoid that math problem?

plugwash wrote:
Sat Mar 23, 2019 5:13 pm
...
3. Use division by 10 instead of multiplication by 0.1 and rearrange the calculation so that the devision is the last step of your calculation meaning things are only rounded once. I don't like this option because it's rather fragile, a seemingly innocuous change to the calculation could break it.
...
Thank you. That worked flawlessly.

Code: Select all

``````...
dB = (ret[0] + ((ret[1] & 3) * 256)) / 10 + 30
print (dB)
...
``````

plugwash
Forum Moderator
Posts: 3433
Joined: Wed Dec 28, 2011 11:45 pm

### Re: Solved: How to avoid that math problem?

rin67630 wrote:
Sat Mar 23, 2019 4:22 pm
But if I modify
dB = (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30 to
dB =decimal( (ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
I get an error:
Traceback (most recent call last):
File "devdB.py", line 35, in <module>
update()
File "devdB.py", line 26, in update
dB = decimal((ret[0] + ((ret[1] & 3) * 256)) * 0.1 + 30)
TypeError: 'module' object is not callable
There are two problems here.

Firstly "Decimal" (capital D) is a type within the "decimal" (small d) module

So to make your code run without errors you need to either do "from decimal import Decimal" and then do "Decimal(stuff)" or do "import decimal" and then do decimal.Decimal(stuff) . Personally I find the former neater.

Secondly you are bringing "Decimal" into the game too late, when we unpick the nested brackets your code is like.

Code: Select all

``````a = ret[1] & 3 #bitwise and ret[1] with 3 (exact integer maths)
b = a * 256 #multiply by 256 (exact integer maths)
c = b * 0.1 #multiply b by a python float (IEEE double) approximation of 0.1 , round the result to fit in a python float.
d = c + 30 #add 30, round the result to fit in a python float.
dB = Decimal(d) #convert the result to a "Decimal", but the damage was already done.
``````
Whereas what is actually needed to get an exact answer is more like

Code: Select all

``````a = ret[1] & 3 #bitwise and ret[1] with 3 (exact integer maths)
b = a * 256 #multiply by 256 (exact integer maths)
c = b * Decimal("0.1") #multiply by a "Decimal" representing 0.1 . Given the default precision settings for "Decimal" this is an exact operation. Note that it is important that the "Decimal" is created from an exact representation of 0.1, if we had done "Decimal(0.1)" instead we would have ended up with a "Decimal" storing an approximation of 0.1 because 0.1 would be converted to a floating point number before passing it to "Decimal".
dB = c + 30 #add 30, again precise.
``````
Or combining it all back together

Code: Select all

``````dB = (ret[0] + ((ret[1] & 3) * 256)) * Decimal("0.1") + 30
``````
rin67630 wrote:
Sat Mar 23, 2019 6:39 pm
Thank you. That worked flawlessly.

Code: Select all

``````...
dB = (ret[0] + ((ret[1] & 3) * 256)) / 10 + 30
print (dB)
...
``````
You are still running the risk of problems from multiple rounding here. To avoid this rather than adding 30 after division you should add 300 before devision.

rin67630
Posts: 378
Joined: Fri Mar 04, 2016 10:15 am

### Re: Solved: How to avoid that math problem?

plugwash wrote:
Sun Mar 24, 2019 1:32 am
rin67630 wrote:
Sat Mar 23, 2019 6:39 pm
Thank you. That worked flawlessly.

Code: Select all

``````...
dB = (ret[0] + ((ret[1] & 3) * 256)) / 10 + 30
print (dB)
...
``````
You are still running the risk of problems from multiple rounding here. To avoid this rather than adding 30 after division you should add 300 before devision.
OK, thank you, I will change that as well, albeit the current math works without flaw.