Testing AI Agent and the Response to the User does not match the Output shown in the test
Summary:
We are testing the Benefits Analyst Agent and are getting incorrect responses when information is retrieved from a table. Upon investigation, I see that the "answer" ouput in test mode is actually correct, but the response given to the end user asking the question is not correct, and is not even close to the "answer".
Has anyone else observed this in their AI Agents?
Content (please ensure you mask any confidential information):
Test Output:
{ "answer": "According to the provided references, the cost for single dental coverage is as follows: \n\nThe total monthly premium for single dental coverage is $52.29. This is broken down into an employer bi-weekly dental premium of $19.31 and an employee bi-weekly dental premium of $4.83. \n\nPlease note that these rates are subject to change, and it's always best to verify with the benefits provider or the HR department for the most up-to-date information." }