What is AI response oversight?

As a true AI company, we have AI doing multiple tasks at AskHandle, including overseeing AI responses to ensure that the information users receive is correct and relevant to their queries. In the Chats sections, below every response provided by AskHandle, you can see two numbers or scores in the bracket. These scores range from 0 to 1.

The first score in the bracket represents the truthfulness of the responses. This score indicates whether the AI used the same words that are in the information imported to the data sources or if it creatively structured the responses using the same data and context. A score of 0 means that none of the words in the response match those in the imported information, while a score of 1 means that the response uses the exact same words and phrases found in the imported information. The numbers between 0 and 1 represent the similarity of the words in the imported information to those in the provided response.

The second score in the bracket represents the relevance of the response. If a user asks a question that cannot be specifically answered because the information is not in the imported data, this score will be zero or close to zero. Conversely, if the user query can be specifically answered with the imported information, this score will be 1 or very close to 1. Scores between 0 and 1 indicate how specifically the user query can be answered given the imported information.

These two scores are powerful indicators for understanding the thoroughness and comprehensiveness of the imported information. If the scores are consistently low for most user queries, it suggests that more specific information should be added to the data sources.