"Successes" in Davis Copilot Chat?

[mmevanson]

Why does the CoPilot Adoption Ready-Made dashboard show successes in Chat, when the response is "Sorry I can't do X"

[JeanBlanc]

Hi @mmevanson 🙂

In Dynatrace’s CoPilot Adoption dashboard, I think the “Success” count isn’t a measure of whether the AI actually fulfilled the user’s request, it’s simply a count of how many chat‐API calls returned a valid response payload (HTTP 200). Even a “Sorry, I can’t do X” message is still a successful API call from the platform’s point of view, so it gets tallied under “Success.”

Maybe you could parse the JSON response for error codes or “cannot” keywords and treat those as partial-failures.

Regards,

[mmevanson]

Thanks for the suggestion.  My post was more about the method of calling something successful in the technical manner vs. providing a meaningful answer. 
"Sorry, I can't help you with that"  is not a success in my eyes, or my customers eyes.   This truly is not a success in providing meaningful/applicable responses to your prompt.  
Matt