-
weird problem with to.Fixed
So i have sNum value from a slider with the value of 109.85. I have the following expression in a text field and I want it to round to 1 decimal place and result is 109.8
What am i overlooking? is this a bug?
sNum = thisComp.layer("SLAB 1 CONTROLS").effect("STAT NUMBER")("Slider");
sNum.value.toFixed(1) -
9 Replies
-
It works for me. What result are you getting?
Dan
-
I get 109.85 being rounded to 109.8 instead of 109.9
-
I’m almost ready to replace the code with Math.round(sNum*10)/10
but I don’t understand why to.Fixed is round .85 down to .8
-
Isn’t that the way rounding is supposed to work? If the digit you’re dropping is 5 and there are no digits to the right of it, you round to the even number. That’s the way I learned it anyway.
Dan
-
If it was consistently like that I would agree. 109.85000001 rounds to 109.9
It seams that to.Fixed is “> 5” for rounding instead of “>= 5”
Thanks for the response anyway. At least I am not crazy.
ps. Math.round is >=
-
>109.85000001 rounds to 109.9
as it should (since there are digits to the right of the 5, so round-to-even doesn’t apply). 🙂
Dan
-
yes, but the original number was 109.85 which was rounded to 109.8 lol.
-
109.85 should round to 109.8
109.850000000000000000001 should round to 109.9As I recall, the analogy was to think of it as a teeter-totter. Anything less than 5 tips it to the left, anything greater than 5 tips it to the right and exactly 5 is perfectly balanced in the middle, so you have to pick a side for it. By convention, you go with whatever makes the previous digit even.
-
That is indeed curious.
.35 and .85 are rounded down
while
.05, .15, .25, .45, .55, .65, .75 and .95 are rounded up.
Reply to this Discussion! Login or Sign Up