Is it possible to extract Data Flow History to be utilised in DV?

philipgodfrey

Hi there,

I was inspecting a completed Data Flow in OAC, and navigating to "History", this gives some basic metrics relevant to the data flow, such as "Start Time", "Status" and "Total Duration".

It would be great to use these data flow metrics (such as total duration) within a DV canvas, displaying overall information about dataflows e.g., number of runs, average time taken etc, if this is possible?

I can see within the History of a data flow run, there is an "Identifier" included, which gives me some hope that metadata may be stored somewhere.

Thanks

Phil

Mike Durran-Oracle

You could try the REST APIs that allow access to a dataflow history - https://docs.oracle.com/en/cloud/paas/analytics-cloud/acapi/op-20210901-catalog-dataflows-dataflowid-history-get.html

Please see Gabrielle's blog: https://medium.com/%40gabrielleprichard/get-started-using-the-oracle-analytics-data-flow-and-sequence-apis-1817600c2174

An example of the output, for this API call: /api/20210901/catalog/dataflows/J21pa2UuZHVycmFuQG9yYWNsZS5jb20nLidGMV9MQVBUSU1FU19MT0NBTF9EQVRBRkxPVyc/history

I get the response:
[ { "jobName": "'mike.durran@oracle.com'.'F1_LAPTIMES_LOCAL_DATAFLOW'", "jobIdentifier": "C04E4954D26FC8F8CC6B78C694FACAC1", "jobStartTimestamp": "2025-07-16T12:39:56.0+0000", "jobExecutionStartTimestamp": "2025-07-16T12:40:19.0+0000", "jobEndTimestamp": "2025-07-16T12:40:32.0+0000", "jobStatus": "SUCCEEDED", "jobRunType": "DATAFLOW", "jobQueuedDuration": "PT36S", "jobRunDuration": "PT13S", "jobOriginOfRequest": "Manual", "jobNumberOfTasks": "1", "jobTasks": [ { "taskName": "DFLW : F1_LAPTIMES_LOCAL_DATAFLOW", "taskStartTimestamp": "2025-07-16T12:40:21.0+0000", "taskEndTimestamp": "2025-07-16T12:40:32.0+0000", "taskStatus": "SUCCEEDED", "taskDuration": "PT11S" } ] }, { "jobName": "'mike.durran@oracle.com'.'F1_LAPTIMES_LOCAL_DATAFLOW'", "jobIdentifier": "EBE6737052C3E4587FBD31549FE1F43", "jobStartTimestamp": "2025-07-09T12:45:12.0+0000", "jobExecutionStartTimestamp": "2025-07-09T12:46:02.0+0000", "jobEndTimestamp": "2025-07-09T12:46:15.0+0000", "jobStatus": "SUCCEEDED", "jobRunType": "DATAFLOW", "jobQueuedDuration": "PT1M3S", "jobRunDuration": "PT13S", "jobOriginOfRequest": "Manual", "jobNumberOfTasks": "1", "jobTasks": [ { "taskName": "DFLW : F1_LAPTIMES_LOCAL_DATAFLOW", "taskStartTimestamp": "2025-07-09T12:46:04.0+0000", "taskEndTimestamp": "2025-07-09T12:46:15.0+0000", "taskStatus": "SUCCEEDED", "taskDuration": "PT11S" } ] }]

Benjamin Arnulf-Oracle

Adding @GabriellePrichard-Oracle and @Luis E. Rivas -Oracle to this question.

philipgodfrey

Excellent, thanks very much @Mike Durran-Oracle, that looks like it should provide the information I'm looking for. I'll give that try.

Thanks

Phil