New Improved Oracle Analytics AI Assistant with option to use third-party LLM

GauravSingh

Many Thanks @Jacques V.-Oracle for the response.

Few more doubts Before using either of them, we would like to know below details around Oracle Default LLM (with 10 OCPU) or registering External (OpenAI) LLM.

1. If we use External LLM, how costing model works? Will it be Natwest to OpenAI agreement or Oracle has any part to play in between? Is there any guidance or document which mention the costing model etc.
2. What are compliance and Guardrails using external LLM for ex. if we have Secret and confidential Data, will the data be exposed or transferred to any LLM? Is there any data leaving the firewall to external world?
3. For your information - we have WAF (Web Application Firewall) as additional OWASP security on Oracle Cloud - will it be challenge using with any LLM?

Many Thanks in advance.

Thanks,

Gaurav

Jacques V.-Oracle

@GauravSingh

1. If we use External LLM, how costing model works? Will it be Natwest to OpenAI agreement or Oracle has any part to play in between? Is there any guidance or document which mention the costing model etc.
   
   A: Oracle is not involved in the agreement at all in the 3rd Party Option. Your company would sign an Agreement with OpenAi (in this example) directly and you would use/put your 'key' in the OAC console. Oracle does not manage usage or do any billing for the 3rd party setup.
   
2. What are compliance and Guardrails using external LLM for ex. if we have Secret and confidential Data, will the data be exposed or transferred to any LLM? Is there any data leaving the firewall to external world?
   
   A: You can find more details on this in the OAC Doc, but to summarize: we send questions and some metadata over to the 3rd party LLM. You can secure/control what is available to the AI Assistant in the dataset/Subject area properties and can 'disable' columns that you might never want to interact with in NL. As for the boundaries of what the 3rd party LLM saves/stores that is an agreement between you and the 3rd party.
   

GauravSingh

Many Thanks @Jacques V.-Oracle .

Bhaskar Konar

Very helpful & useful information.

Thanks for sharing @Benjamin Arnulf-Oracle, @Jacques V.-Oracle!

GauravSingh

@Jacques V.-Oracle - We are on May 2025 version but still i see below message in Assistant Tab in Workbook.

"The Assistant feature is currently disabled as no language model has been assigned. Please contact your service administrator to assign a language model to enable this feature."

Out understanding was from May25 version, we will be able to use Oracle Default LLM in AI Assistant feature.

Thanks

Gaurav

Jacques V.-Oracle

@GauravSingh The Association of your instance to the LLM takes a couple of days after you get updated. Hang tight and you should get it by EOW.

GauravSingh

Thanks @Jacques V.-Oracle

RMJ-kpmg

hi @Jacques V.-Oracle -

I'm trying to add my OpenAI key in the LLM registration screen, but the validation step is failing. I've already confirmed that my API key is valid and can be called using a quick Python script. Using Chrome Dev Tools, it appears that the OAC validation test is returning a '404 Not Found' message when trying to check the API key. Can you recommend how to proceed?

For reference what I'm seeing with Chrome:

Request URL: https://kpmgoacdev-idrzkaavkwkd-ia.analytics.ocp.oraclecloud.com/ui/dv/ui/api/v1/llm/test-registration-details

Request MethodPOST

Payload:

1. {requestStatus: 200, success: true,…}
   1. requestStatus: 200
   2. success: true
   3. validation-response: {status: "FAILURE", errorMessage: "Server returned unexpected status: HTTP/1.1 404 Not Found"}
      1. errorMessage: "Server returned unexpected status: HTTP/1.1 404 Not Found"
      2. status: "FAILURE"

SteveF-Oracle

Hi Rudy, (@RMJ-kpmg) ,

Ensure you are not using a "free-tier" or "trial" type account, they will not work.

[EDIT] Documentation is being updated to reflect this.

RMJ-kpmg

Hi @SteveF-Oracle - Thanks, upgrading to the paid tier did the trick!