Data Flows Monitoring in OAC

CPavan

Hi,

Is any one built any program / code to monitor data flows in OAC instance? We're familiar with Job feature, but we've noticed that it doesn't provide deeper insights into certain aspects we're interested in. Specifically, we're looking for more detailed information regarding the execution duration of the latest job, the length of time it spent in the queue, and, at a high level, the amount of data processed.

Our goal is to gain a clearer view of how long data flows are executing each day, the queue durations, and the processing volume. This would help us optimize our workload and better monitor performance.

Thanks & Regards,

Pavan

Sumanth V -Oracle

@CPavan - Thanks for using Oracle Community!

As of now, there is no such option available within the Oracle Analytics Cloud (OAC) application. We can see that you have already submitted an idea requesting this feature, which is great.

In the meantime, please review the link below to see if it helps with managing notifications:

https://community.oracle.com/products/oracleanalytics/discussion/22864/setting-up-email-notifications-for-data-flow-failures-in-oracle-analytics-cloud-oac

SteveF-Oracle

To add to the previous reply about providing a notification service for dataflow or dataset failures, you can refer to this blog:

https://blogs.oracle.com/analytics/oracle-analytics-data-flow-notification-system

The Log filter task section allows you to configure how you’d like to filter the logs (in this particular example, we’ll be returning only the logs that indicate a data flow failure). To determine how best to fill out the Log filter task section for your use case, examine the JSON code associated with the OAC diagnostic logs. 

You may be able to explore the log filters to see if you can be notified for other dataflow elements.

CPavan

Thank you @Sumanth V -Oracle , let me check this option.

CPavan

Thank you @SteveF-Oracle