Problem with Data Flows Losing Column Type Meta Data in DVD 4.0

TimVlamis

Summary

Problem with Data Flows Losing Column Type Meta Data in DVD 4.0

Content

There is a problem with the loss of column type meta data when the new Data Flow interface is used. Data sets which have previously had columns designated as numeric with an aggregation rule lose this information after a data flow is run. The below screen shot shows the Sample Project included with DVD. Note that several columns are of data type Numeric.

Here I created the simplest possible Data Flow. This data flow makes no transformations. It simply inputs and outputs the Sample Order Lines data set used in the Sample Project. The data set is now saved as “Sample Order Lines Data Flow Ex”.

I created a new project using the Sample Order Lines Data Flow Ex” data set. As you can see, all the measures have been converted into Attribute columns.

When we look at the prepare tab, we can see that DVD knows that some of the columns are numeric because they show up in orange in the Data Tiles view.

Below I added a column in a new Data Flow and see if we can get DVD to know that the new column is a measure. Note that when I typed “order” in the definition field, DVD knows which columns with the word “order” in the name are Attribute columns and which columns are Measure columns. We can see the "#" sign next to the measure columns and the "A" next to the attribute columns. However, this information is not being passed on to the output. The new column “Cast Quantity Ordered” which has been created using the “cast as integer” function is also an Attribute in the resulting data set.

Obviously, there are times when we will upload a data set and create a project before we create a data flow. It’s important that we be able to retain the work in fixing automated column types, setting aggregation rules, and creating new columns when creating a new data flow. The column types should be read and retained in the Data Flow without having to redefine them. Anyone have any insights?