Post 1 made on Sunday January 20, 2008 at 19:07 |
chrishug Lurking Member |
Joined: Posts: | December 2007 2 |
|
|
Hi,
I have noticed a stange thing in the floating point addition on the TSU9400. (I don't know if it is unique to the 9400 as I don't have a 9600 to compare with.)
If you create a button and assign the following script to it.
var k = 1.1; k += 0.1; k = k + " "; label = k;
When pressed I expected the button label to change to 1.2 but it changes to 1.2000000000000002
I found this earlier today in a script I was writing to predictively calculate real time frequency changes while using the rotary wheel to tune the FM tuner of my Anthem D2 via the RFX9600 over RS232. To work around this when I want to add 0.1 I now multiply by ten, add 1, and then divide by 10.
Subtracting 0.1 seems to work just fine and does not need a work around.
Is this expected behaivior, or is it just something to be aware of and remember to work around it? Many thanks, Chris.
|
|
Post 2 made on Sunday January 20, 2008 at 21:14 |
Peter Dewildt Loyal Member |
Joined: Posts: | July 2001 6,307 |
|
|
Welcome to floating point arithmetic. 0.1 is not exactly representable in binary. There are lots of ways around it - use a rounding function to get it to 1.2 when converting to a string - use a format function when converting to a string - use an integer to keep track of the number of "0.1"s you have then divide it by 10 and round when setting the label - do what you did (though it is capable of going the same way)
I don't have Flanigan's bible in front of me, so I don't know what rounding and format functions are available.
|
Peter Pronto 1000 (retired), Pronto TSU7000, RFX6000 (retired) Pronto 2xTSU9600, RFX9400 |
|
Post 3 made on Sunday January 20, 2008 at 23:30 |
Lyndel McGee RC Moderator |
Joined: Posts: | August 2001 13,006 |
|
|
There's something about the += and -= in Flanagan's book but not sure if it covers FPA (Floating Point Arithmetic).
|
Lyndel McGee Philips Pronto Addict/Beta Tester
|
|
Please read the following: Unsolicited commercial advertisements are absolutely not permitted on this forum. Other private buy & sell messages should be posted to our
Marketplace. For information on how to advertise your service or product
click here. Remote Central reserves the right to remove or modify any post that is deemed inappropriate.