Let’s explain why sometimes our Agent provides wrong or completely false information.
The source of the problem is related to how the LLM ChatGPT works.
In a few words, The AI aims to always give an answer. If it can’t find the right info in the provided dataset, it will invent a response.
This phenomenon is called hallucination.
To prevent this from happening, here are some tips:
First of all, make sure you don’t include more than one source of the same content inside the knowledge hub.
For example:
Fetch the dataset using my website, and then proceed to create a Google Doc or PDF with the same content.
This will overlap and create confusion when the bot is scanning for the right answer.
please note.
( also make sure that the content inside your business description and personality does not crete a conflict with the content created inside your dataset informations )
So, make sure you don’t have any duplicated content.
Secondly, ensure that you provide information accurately.
As we explained before, the bot will make up information when it is not able to find it inside the dataset.
Another problem could arise when a user asks for something that does not exist, and we haven’t instructed the agent on how to handle it.
For example:
3 pricing table structure:
Beginner $10 - Intermediate $20 - Professional $30.
If we ask the chatbot what the price for an Ultra Professional plan above $30 is and we haven’t entered the correct instructions inside the business information about how many plans there are in total, it will likely respond by making up a fourth plan, Ultra Professional, for $40.
In short, it’s all about providing the right information, making sure we don’t have cloned or overlapping data, fine-tuning wrong answers using Q&A, and creating the correct instructions to insert them into the business description.
Recommended settings:
When debugging your chatbot, make sure you have these settings on/off:
- Switch to GPT-4 Turbo if you are on GPT-3.5 Turbo.
- Turn off all quality control options.
- Check for duplicated content (especially if you are importing your dataset using website crawling).
We are working to provide new ways to fine-tune your agent by adding further instruction forms and more complex ways to fine-tune responses. In the meantime, we remind you that you can contact @Simone via personal chat, and he will do his best to check your knowledge hub for mistakes.
Have a great day,
Answerly team