开发者

Why trace(123.456 - 123) gives me 0.456000000000003 in flash as2

开发者 https://www.devze.com 2023-04-01 22:12 出处:网络
trace(123.456 - 123)//the output is 0.456000000000003!!!!!! Why it gives me this strange value? i need it to 开发者_如何学运维output 0.456

trace(123.456 - 123) //the output is 0.456000000000003!!!!!! Why it gives me this strange value? i need it to 开发者_如何学运维output 0.456 (my aim is to have the fraction)


Just like 1/3rd and 1/7th cannot be expressed well in decimal notation (0.33333... and 0.142857142857... respectively), certain decimal numbers can't be represented well in binary, leading to errors like this. To solve it, try this:

var mynum=(123.456 - 123);
mynum=Math.round(mynum*1000)/1000;
trace(mynum);


Because it's a floating point number and you can't accurately represent all numbers in floating point.

If you need a fraction, make a faction class (maybe one is built into AS already I don't know.)


Because you're dealing with a floating point number. you're going to need to round it up:

http://board.flashkit.com/board/showthread.php?t=778701

exert form the above link (with numbers changed):

var num:Number = 123.456 - 123;
num *= 1000;
num = Math.round(num);
num /= 1000;
trace(num);
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号