Continuous AI Retraining and Improvement

[Why your AI is never “finished”]

Here's the thing about AI. It's never truly finished. The more conversation your AI handle, the more opportunities you will find to improve it. Today, we are diving into how to continuously retrain and refine your AI assistant so it gets smarter, more accurate, and able to handle even more of your incoming conversations. By the end of this lesson, you will have a clear process to address gaps in your AI knowledge, handle escalations due to low confidence or missing information, spot and fix issues like hallucination. Let's make sure your AI is always improving. When it comes to training your AI, your job isn't done after the initial setup.

To get your AI to handle 80 90% of your conversation on its own, you need assistant to continuously retrain and improve it. Okay. Okay. This lesson is all about how to review escalation caused by low confidence or missing information. Decide whether to add knowledge based content, candidate responses, or create a new intent. Then we will check spot check conversations for issues like hallucination and address content gaps. Let's start with how to handle escalations. Sometimes your AI doesn't have the confidence to

[Handling escalations]

answer a question or doesn't have the right information to handle it. Here's how to fix that. Review escalation. Check the conversation that were escalated due to the low confidence or missing data. Identify what the customer was asking and why the AI couldn't answer it. Decide how to fill that gap. Does the question need a simple knowledgebased article or maybe a cannon responses? So, for example, what are your opening hours? uh you just add a web standard article. Does it need a cannon response? For example, can I change my billing cycle? Add a pre-written or cannon response or does it require a new intent? For example, I would like to upgrade to enterprise.

Then you can create a new intent of his own on a shop floor. Now, let's check some examples over here. First of all, we have a series of conversation with Aura. And if you go under uh customer support, now we have a low confidence escalated issue over here. And as you can see, the customer asked, "Is there a way to use smart contracts with Fincheni?" This thing about smart contract is not there. So that's why she escalated with a low confidence issue and it was right because she didn't feel confident to do it. Okay. And also the predicted message, it seems like there is no specific information available about using smart contract with Pini.

So here, if you know this is something that you cover in a knowledgebased article because maybe smart contracts are something that you actually offer but you never created a dedicated guide to it, you can provide down the response and then you can create a candidate response or an article. So let's write down an actual example on how you should reply to the customer first and then what to do next. Now that I've got my quick response about how to create a smart contract, what I can do is using Aura AI within the inbox is to expand this response. So this is super helpful because it's not only expanding the answer based on the question and what I wrote over here, but it's also searching other points that are connected with your own knowledge base.

So if you're covering in this response something that is already covered in your app center articles then this would be helpful to generate a larger a more practical response based on the different products that the user is asking about. Now if you revise the answer you can see that it's quite perfect in terms of the guidance and everything. So now I can use this in my editor. I can change the link over here. And here we go. I can send this out. Now, what can I do in order to train the AI to answer this? Well, the next time. Now, you can create this as a cannon response, but this might be also helpful for anyone out there searching in your knowledge base.

So probably this would be something that you would provide as a knowledgebased article. Now within Aura AI over here in your inboxes you can create an epenter article draft. So based on the exchange um basically the question and the answer you provided aura will be able to create a draft of your epsenter article. This is so cool because it's saving you plenty of time. Okay. So let's put it down under this collection. Now I can open the editor and this will be ready to be published. Probably you can remove a bit of this but this is fine. You can add some screenshot of course to add more information on how to achieve this goal.

But as you can see this is great because it's helping your team to save plenty of time whenever the answer is not there. Now whenever I'm publishing this they will be immediately trained about using smart contract and you can test it out. So now we are going to ask the very same question again and let's see how Aura will respond to this and that's it. She she just gathered the right information from that article we created with AI after responding with a brief response and you're good to go. So this is how you treat the low confidence escalated uh tickets such as this one. And the next time when a new customer is going to ask me the very same question, it's going to respond automatically in this perfect way.

So it's either a low confidence issue or a missing information. The way you would treat it is the same. You just build either a candidate response or a knowledgebased article the way I showed you today and you will be able to fix this gap for the next time that that question is going to arise. Cool. So this is another scenario that

[Improving Intent recognition]

we are going to cover today. Sometimes people ask things in a completely different way that you plant in your intent. Right? So sometimes aura cannot fully understand that question and try to provide an answer that is more correlated to a knowledgebased article rather than matching an intent. So in this case the user asked what if I finish my invoices for this month which is you know is not proper English or you know it can be tricky to recognize the actual query and so when Aura tried to provide a response then the customer said no I mean I've notable invoices in my infin account anymore which was supposed to trigger the plan upgrade intent that we created before.

Now the way you can fix this is try to provide to the plan upgrade intent that was triggered in the proper way and also a matching intent example with this question in mind. This way your AI will be able to understand this kind of iteration. So this would be the best way to handle those intents that are not fully meshed at the very first time because maybe there was a a slightly different approach in asking that question. Most of the time or I will be able to mesh the intents with very different kind of iterations but sometimes it happens that the customer invents new ways of asking in a completely different way for the very same thing.

So this is how you could fix that kind of gap in your handling. All right. Even the smarter AI can hallucinate

[Spotting and fixing hallucinations]

giving answer that sounds convincing but are incorrect or irrelevant. To spot and fix those issues, you will probably go to your CESAT insight dashboard and filter for responses market as terrible or bad. Open those conversations and review what the AI said and compare it to what the customer was asking and look for gaps or errors such as did the II provide an answer that didn't make sense or was there a lack of accurate context that caused the hallucination and fix the road calls. If the issue was due to missing content, create an article or kind of response to fill that gap. If the issue was intent related, create or refine an intent uh to handle similar case better.

spot checking conversation like this once a week ensure to catch an address hallucination before they become a larger issue. For example, I knew for sure that this question would lead to some sort of hallucination. Why is that? Because by having an article describing API access for custom integrations, which means that basically you can integrate with any software because it's a custom integration just by having access to the API. Right? So Aura over here even though like maybe Finini doesn't have an integration with customly aura provided an answer to integrate finini with customly and you can use the finini AI API for custom integration and here's how to get started it might be right it might be correct but in this case if you don't integrate with customly orderer there is no such a thing you can put it down into a knowledgebased article or a can response we do

for example have a can response response with specific things that are asked for a specific integration. So if custom is not an integration yet, what we would do is we currently we write down a kind of response to say we currently don't integrate with XYZ. You can suggest an integration on our road map for examples or on our idea. So this is how you would do it. We currently don't provide an integration with customerly. Please add it to our road map. Now this is a kind of response of course and uh once you save it and you use for training with AI. Now if we're going to ask the very same question to Aura this would provide the actual correct response.

So when the user is going to ask the very same question right now Aura is able to respond in the proper way. Now, as you can see, Aura provided this time the real answer that we currently don't provide an integration with customly. However, you can add it to our road map here. If you need a custom integration, you can use API to create one. So, now this is a way better response because it's saying that we don't currently have an integration with customly. However, you can either add it to our road map or if you need a custom integration, you can use the API. And if you're going to check the the message over here, of course, it g the information from the very specific kind response.

This is how you would treat an hallucination. All right? And it can especially happens when you have longer articles with a lot of different informations in it and probably sometimes the customer ask a variation of the features that you're offering or something else that it might be getting a to hallucinate. So in this case, you would be creating a new article specifically for that customer request or a kind of response that is going to shift the focus from what or find out and that led to an hallucination and in this way you can fine-tune your model in a way that next time the answer is going to be better.

[Building a review process]

Now to continuously improve your AI, you need a regular review process. And here's a simple plan. Weekly review. Check escalations caused by low confidence or missing information. Review your low sees conversation for patterns and gaps and analyze monthly overall performance trends. Identify recurring issues or intents that needs refining. And after creating new content or intense, test them thorally to ensure they work as intended. By sticking to this review process, you will not only improve your AI's accurrency, but also expand its capabilities over time. Here's what we covered today. If a conversation is escalated due to low confidence or missing information, decide whether to create a knowledgebased article, a canned response, or a new intent. Use a CISAT dashboard to spot low feedback conversation and check for hallucination or content gaps. Third, regularly review your AI's performance weekly for specific issues or monthly for overall trends. By following these steps, you can continuously improve your AI's

[Key takeaways]

performance and get it to end up to 80 90% of incoming conversations. All right, AI isn't just set it and forget it. With a clear process for retraining and improvements, you can keep your assistant sharp, accurate, and capable of handling more complex scenarios every day. If this was helpful, hit like button, subscribe, and let's keep building together a smarter AI systems. I'll see you in the next