One of the controls we have to identify that ClaudIA is not able to find enough content to respond to the user is the 'low confidence counter'.
In the Hub, when ClaudIA does not find content above the stipulated confidence (more on that later), a 'Count limit' indicator appears in yellow below ClaudIA's response, as shown below:
After reaching the configured limit, the service is automatically transferred to a human using the message configured in the 'Forward To Human Message' field on the Hub settings screen.
The configuration of the limit counters can also be adjusted on the same screen:
-
'Limit' is the field that defines the maximum number of low-confidence searches. PS: we recommend at least 3, as not every low-confidence content search necessarily generates a bad response. Sometimes ClaudIA clarifies and that resolves it; sometimes the user's message does not require content to be answered (example: "Good morning!")
-
'Low Confidence Threshold' is the minimum expected distance to be considered relevant content (high confidence).
Every content search in the Hub presents a score that you can track directly whenever you see the sessions returned by ClaudIA's search, as in the Conversations audit screen:
Additional functionality
It is possible to configure ClaudIA to ignore the low confidence count on the first response of a ticket, considering that often the user's first interaction is just a greeting. This prevents the counter from being incremented unnecessarily. However, if the first message already contains a question and ClaudIA provides a low-confidence answer, that interaction will not be counted. The other interactions in the ticket follow the normal confidence counting flow.
Currently, this configuration is not available in the Hub. To activate it, you need to request a Clouder to set the option ignoreConfidenceCountOnFirstMessage to true.