FDI - Data Augmentation Completed but no data Inserted into the table

Question

I have created an Oracle CPQ connection in Fusion Data Intelligence (FDI).

After that, I have created Data Augmentation with 13 columns (1 Primary Key, 1 Incremental Key and 11 normal Attribute columns). The Data Augmentation run completed, but no data was inserted into the table.

I checked the Warehouse Refresh statistics, and the Record Rejection code is "schema columns and data columns mismatch, schemaColumns:13, dataColumn:15".

I checked the Data Augmentation table description and system generating W$_INSERT_DT and W$_UPDATE_DT columns. Is this because of these two columns?

Can you please help me on this?

RichardChan · Answer

Right, you've run into a classic snag with Oracle Fusion Data Intelligence (FDI) Data Augmentations, and your hunch about W$_INSERT_DT and W$_UPDATE_DT is spot-on! The Nuisance: Schema Columns and Data Columns Mismatch The error message, "schema columns and data columns mismatch, schemaColumns:13, dataColumn:15," is as clear as a bell. It means FDI expects 13 columns based on your Data Augmentation setup, but it's getting 15 columns from your source or during the loading process. Why W$_INSERT_DT and W$_UPDATE_DT are the Culprits Spot on! You've likely found the cause. When FDI creates the target table for your Data Augmentation in the Autonomous Data Warehouse (ADW), it automatically chucks in two system-generated columns: * W$_INSERT_DT: This is a timestamp showing when the record was first put into the FDI warehouse table. * W$_UPDATE_DT: And this one's a timestamp for when the record was changed in the FDI warehouse table. These two columns are handled by FDI's own internal wizardry (the ETL process) and aren't counted among the 13 columns you defined for your Data Augmentation. So, if your source is somehow spitting out 15 columns, or if FDI's internal mapping is getting its wires crossed about your 13 chosen columns, you end up with this mismatch. Possible Scenarios and Solutions: * Your Source is Pushing Extra Columns (This is the usual suspect!):* Check your Oracle CPQ source query/view: Give the underlying query or view that your Oracle CPQ connection is using for the Data Augmentation a thorough once-over. Make sure it's only selecting the 13 columns you want to augment. Sometimes, a SELECT * or a view might expose extra columns you're not explicitly mapping in FDI. * Explicitly pick columns in FDI: When you're setting up the Data Augmentation in FDI, be absolutely sure you're explicitly selecting just the 13 columns you need from the Oracle CPQ source. Don't rely on any "select all" or default behaviour. * Dodgy Column Mapping in Data Augmentation:* Pop back into your Data Augmentation definition in the FDI console. * Carefully go over the column mapping to confirm that each of your 13 source columns is correctly mapped to a target column, and that there aren't any accidental additions or duplicates that would make FDI expect more columns than you've defined. * Double-check that your Primary Key and Incremental Key are correctly identified from your 13 columns. * Metadata Refresh Woes:* After making any tweaks to your Oracle CPQ source (like adjusting a view or query), it's absolutely vital to refresh the metadata for your Oracle CPQ connection in FDI. This ensures FDI has the most current understanding of your source table's structure. * Sometimes, even if you haven't changed the source, a metadata refresh can sort out inconsistencies. * Cache Gremlins (Less common, but worth a punt if all else fails):* While less likely to be the main reason for a schema mismatch, sometimes browser cache or internal FDI cache can cause odd behaviour. * Try clearing your browser's cache and then giving the Data Augmentation run another go. * In some rare cases, for big changes, you might need to try resetting data augmentation-related cache in FDI (have a gander at Oracle documentation or speak to support for specific steps if this becomes necessary). Steps to Troubleshoot and Fix: * Examine your Oracle CPQ Source:* Log into Oracle CPQ or wherever your underlying data source (view or table) lives. * Run a query directly against that source to see precisely how many columns it's returning and what their names are. This will tell you if the source itself is providing 15 columns. * Review your FDI Data Augmentation Definition:* Head to your Data Augmentation in the FDI console. * Go to the step where you select and map your columns. * Count the columns you've explicitly selected. Make sure it's exactly 13. * Verify that the Primary Key and Incremental Key are correctly assigned from these 13. * Refresh Connection Metadata:* In the FDI Data Configuration, go to your Oracle CPQ connection. * Look for an option like "Refresh Metadata" and give it a click. * Run the Data Augmentation Again:* Once you've checked and hopefully corrected your source query/view and FDI Data Augmentation definition, try running the Data Augmentation pipeline once more. If you're still scratching your head after all these steps, it would be a good idea to lodge a Service Request (SR) with Oracle Support. Be sure to give them: * The exact error message (including the "Record Rejection code"). * Details of your Oracle CPQ source (e.g., if it's a view, what's the SQL definition). * Screenshots of your Data Augmentation definition in FDI, especially the column selection/mapping. * The fact that W$_INSERT_DT and W$_UPDATE_DT are being generated, as this points to an internal FDI process.

Sumanth V -Oracle · Answer

@Mayuresh Chopade - Trinamix- Welcome to the Oracle Analytics Community!

Thank you for posting your query here.

The error appears to be due to a mismatch between the schema columns and the actual data columns, as you've correctly identified. Since CPQ is still a Limited Availability (LA) feature, I would recommend that you raise a Service Request (SR) with Oracle Support for deeper analysis and resolution.

Please continue to post your questions here — we're always happy to help and assist wherever possible!