New admin tool version to provide data lineage
Organization Name
HSBC Plc
Description
Hi, it is great that Oracle are looking at revamping the BI Admin tool. Our customer has asked if any new version of the admin tool could provide not only entity-relationship (physical / logical) diagrams, but also data lineage so we can see how data flows through the presentation --> business model --> physical layers.
The current Admin Tool allows you to select an object and view any related objects, which is useful, but is not ideal for data lineage/dictionary requirements.
To go one step further, it would be AMAZING if the data lineage would provide flows of data beyond the presentation layer and show the flow of objects into analyses, dashboards and also data viz projects and data sets. So we could select a physical column, for example, and see exactly which reports/dashboards use that column.
Thanks
Use Case and Business Need
This would help with impact analysis and help with data dictionary/lineage requirements that customer have, especially in a cloud environment where customers need to be 100% sure of what data flows where, and any logic applied en route.
Many customers have asked for this type of capability over the years I believe.
Original Idea Number: fd914cfc59
Comments
-
That's a very good idea Antony.
1 -
Definitelly this will provide great value in terms of trustfulness of BI content (data governance must feature) and something , which should be incorporated long time ago (for OBIEE).
0 -
I see 2 ideas here.
One is within the data modeling tool, a need for a lineage view showing data mapping between presentation, business model and physical layer.
The second idea is the need for lineage from analyses, dashboard and data viz, dashboards to the data model and to the data source. These features primarily needed for impact analysis and also as an end to end data mapping documentation.
The first one is planned in the new modeling tool. The scope of the second one is beyond the modeling tool. This can be a data lineage dictionary or a top-down lineage from presentation services (data viz, analysis, answers) and dashboards or a bottom-up lineage from the data model physical layer to presentation services.
0 -
Hi Pravin, thanks for the note. Great that there will be a lineage view in the new admin tool.
However, I think for the 2nd point you are thinking from an Oracle or technical perspective and not the customer's perspective. The customer does not consider lineage and/or impact analysis just within the modelling tool, they consider it throughout the entire platform and this includes the additional layers for reports, dashboards (and now projects).
So whilst any lineage feature within in the modelling component would be greatly appreciated and very helpful to developers, what we are saying here is that customers would still require a separate lineage tool if they want to do full-blown lineage.
0 -
Hi Antony,
Appreciate the idea and feedback. I absolutely understand the need for an end to end data lineage across the platform. I just wanted to let you know that the first part of the idea is planned in the new Data Modeler.
0 -
Hi Pravin,
great to hear this announcement. I know, that Oracle is not commiting to any timeframes, but anyway, I am courious when we can expect this new modelling tool (new version of Bi Administration Tool) ? I am especially courious about availability for on-premise OAS sw.
Thanks and kind regards
Michal
0 -
Thanks Pravin, I appreciate your feedback on the suggestion and understand it is one step at a time. Really positive either way and looking forward to seeing the web based version of the tool.
0 -
Yes, it's extremely difficult to explain to upper management that an enterprise BI platform doesn't have this capability built in to perform impact analysis to see which front-end Analyses/Dashboards/DV would be impacted if a change is planned for a column(s) in any layer of the Admin tool.
Very highly desirable feature from customer's perspective.
0 -
Agree that this is very much needed! If I'm not mistaken, didn't Oracle Enterprise Metadata Management have some of these features? Haven't looked at it in a while. I also remember it being ridiculously expensive.
0 -
In addition to the semantic modeler project, we are talking to the data catalog team to explore more advanced lineage reporting. Changing the status, will update once we have detailed plans.
0 -
The first idea here -- a lineage view showing data mapping between presentation, business model and physical layer -- can be seen in the Semantic Modeler preview. Try this in the May 2022 release in a dev or test environment.
0 -
voting for this. It has always been a problem to trace usage and see metadata for end-users in a more open format that would allow analysts to easily share the metadata/column description along with data exports.
0
