-
weird problem with to.Fixed
updated 5 years, 9 months ago
3 Members
·
10
Posts
-
So i have sNum value from a slider with the value of 109.85. I have the following expression in a text field and I want it to round to 1 decimal place and result is 109.8
What am i overlooking? is this a bug?
sNum = thisComp.layer("SLAB 1 CONTROLS").effect("STAT NUMBER")("Slider");
sNum.value.toFixed(1) -
It works for me. What result are you getting?
Dan
-
I get 109.85 being rounded to 109.8 instead of 109.9
-
I’m almost ready to replace the code with Math.round(sNum*10)/10
but I don’t understand why to.Fixed is round .85 down to .8
-
Isn’t that the way rounding is supposed to work? If the digit you’re dropping is 5 and there are no digits to the right of it, you round to the even number. That’s the way I learned it anyway.
Dan
-
If it was consistently like that I would agree. 109.85000001 rounds to 109.9
It seams that to.Fixed is “> 5” for rounding instead of “>= 5”
Thanks for the response anyway. At least I am not crazy.
ps. Math.round is >=
-
>109.85000001 rounds to 109.9
as it should (since there are digits to the right of the 5, so round-to-even doesn’t apply). 🙂
Dan
-
yes, but the original number was 109.85 which was rounded to 109.8 lol.
-
109.85 should round to 109.8
109.850000000000000000001 should round to 109.9As I recall, the analogy was to think of it as a teeter-totter. Anything less than 5 tips it to the left, anything greater than 5 tips it to the right and exactly 5 is perfectly balanced in the middle, so you have to pick a side for it. By convention, you go with whatever makes the previous digit even.
-
That is indeed curious.
.35 and .85 are rounded down
while
.05, .15, .25, .45, .55, .65, .75 and .95 are rounded up.
Log in to reply.
