Manage Evaluation and Evaluation set in Oracle AI Agent Studio

Summary:

We are currently using AI Agent Studio Monitoring and Evaluation to validate several AI agents deployed in our Oracle Fusion environment.

We have created evaluation sets and successfully executed evaluations. However, we have observed that the evaluation results appear to be generated using the context of the user who is running the evaluation.

Many of our use cases are dependent on employee-specific data, such as:

• Grade
• Leave entitlement
• Country-specific policies
• Legal Employer
• Business Unit
• Manager hierarchy
• Employment status
• Role-based access

As a result, when the evaluation is executed by a single user, the responses appear to reflect only that user's employee profile and data context.