Ability to refresh a Dataset with deleted/renamed columns in DV

Jahnavi-Oracle

Description (Required):

After we upload a dataset into DV, if we have to reload or refresh the data, currently we need to make sure that all columns in the uploaded dataset are present in the new dataset as well. If any column is deleted from the new dataset then DV doesn't do the refresh an shows error instead. There should be a way to be able to upload the new dataset with deleted columns with a warning.

Use Case and Business Need (Required):

If we already have a dataset D1 uploaded to DV (with 10 columns) and the new dataset D2 ( with 9 old and 2 new columns) is used to refresh the data. Then the refresh process would compare D2 with D1 and it would show a warning for the missing columns if the user still wants to go ahead and refresh data with D2.

Also if any column names are renamed in D2 it would be good to have an option to map the D1 columns with D2.

Enhancement Request

Luis Rivas -Oracle

A few follow up questions: Are you referring to datasets based on files or from a DB - and are you referring to a manual "Reload" of the data in order to update the columns, or the automated Manual or Scheduled reload from a DB Table? Each of these have their own restrictions. DB Table metadata changes are more restrictive and will need manual intervention in the Edit Definition Page. If files, these also require going into the Dataset Edit Definition page and changes in columns including deleted and added columns is doable. Please provide additional information and we can follow up with this Idea. Thank you.

Luis Rivas -Oracle

Also note that renamed columns also pose a challenge, but that would be an enhancement to "map" the renamed columns to their existing transforms.

Bret Grinslade - Oracle Analytics-Oracle

You can accomplish this on the "Edit Definition" screen when editing the dataset. This is how you "change the columns being input into a dataset."

If we mis-understand the scenario, please provide another example.

Rajakumar Burra

@Bret Grinslade - Oracle Analytics-Oracle Currently if reload data , it just reloads if all columns are matches.

In Edit definition, we can upload with different columns. It works fine.

Expectation is that it should show , new columns are adding and old columns deleting warning. So that user will know what is missing and adding.

Bret Grinslade - Oracle Analytics-Oracle

Understood. This in an improvement to the edit definition experience to highlight changes so a user understands if they want to proceed or if they should cancel and fix the source/input file first.