Rounding errors in DQL query

[rachel6842135]

I am trying to configure a Dashboard tile using DQL. I am able to pull the required data and run various calculations, however for the final stage I am trying to calculate an error rate, using total errors vs total numbers of calls

I can see the calculation is correct, however the output I am seeing is incorrect, due to rounding errors. An example I have is evidenced below, where I am hardcoding a value of 289, but when I divide this my 100 I see an output which is rounded down to the nearest integer (but with 2 decimal places visible)

Is there a setting I am missing which is somehow converting my number to an Integer before re-adding decimal places? I haven't been able to find anything - both fields are configured to show 2 decimal places in settings

Thanks

[t_pawlak · Accepted solution]

Hi.
@rachel6842135Could you paste your DQL?
It's hard to come up with/verify something if you don't know the syntax
It’s integer division. In DQL, if both operands are integers (e.g., 289 / 100 or count()/count()), the result is truncated to an integer first and only then formatted — so you see 2.00
But try this:

| fieldsAdd errorRate2 = 289.0
| fieldsAdd errorRate3 = errorRate2 / 100.0 
| fieldsReplace errorRate3 = round(errorRate3, 2)

 Or For a real error-rate calc (total errors vs total calls), make one side a double:

| summarize total_calls = count(), total_errors = sum(is_error)
| fieldsAdd error_rate =
    if(total_calls == 0, 0.0,
       100.0 * toDouble(total_errors) / toDouble(total_calls))
| fieldsReplace error_rate = round(error_rate, 2)

[rachel6842135]

Thanks so much @t_pawlak - your second solution (converting to double before calculating) has worked perfectly 🙂