weird problem with to.Fixed

[Chris Scalzo]

So i have sNum value from a slider with the value of 109.85. I have the following expression in a text field and I want it to round to 1 decimal place and result is 109.8

What am i overlooking? is this a bug?

sNum = thisComp.layer("SLAB 1 CONTROLS").effect("STAT NUMBER")("Slider");
sNum.value.toFixed(1)

[Dan Ebberts]

It works for me. What result are you getting?

Dan

[Chris Scalzo]

I get 109.85 being rounded to 109.8 instead of 109.9

[Chris Scalzo]

I’m almost ready to replace the code with Math.round(sNum*10)/10

but I don’t understand why to.Fixed is round .85 down to .8

[Dan Ebberts]

Isn’t that the way rounding is supposed to work? If the digit you’re dropping is 5 and there are no digits to the right of it, you round to the even number. That’s the way I learned it anyway.

Dan

[Chris Scalzo]

If it was consistently like that I would agree. 109.85000001 rounds to 109.9

It seams that to.Fixed is “> 5” for rounding instead of “>= 5”

Thanks for the response anyway. At least I am not crazy.

ps. Math.round is >=

[Dan Ebberts]

>109.85000001 rounds to 109.9

as it should (since there are digits to the right of the 5, so round-to-even doesn’t apply). 🙂

Dan

[Chris Scalzo]

yes, but the original number was 109.85 which was rounded to 109.8 lol.

[Dan Ebberts]

109.85 should round to 109.8
109.850000000000000000001 should round to 109.9

As I recall, the analogy was to think of it as a teeter-totter. Anything less than 5 tips it to the left, anything greater than 5 tips it to the right and exactly 5 is perfectly balanced in the middle, so you have to pick a side for it. By convention, you go with whatever makes the previous digit even.

[Kalleheikki Kannisto]

That is indeed curious.

.35 and .85 are rounded down

while

.05, .15, .25, .45, .55, .65, .75 and .95 are rounded up.