Discussions
Cleaning up Unmapped CDO records
With GDPR fast approaching, many customers will be wanting to tidy up orphaned CDO records in order to be as compliant as possible.
This guide aims to demonstrate how Oracle Eloqua can accomplish this for you.
To avoid the potential deletion of incorrect data, a backup of your CDO is recommended before any records are actually deleted.
To do this you will need 5 assets.
- A program builder
- A program canvas
- An update rule
- A shared Filter
- your chosen CDO*
* this solution works 'per CDO' so if you have 10 CDO's you'll need 10 of the above (with the exception of the update rule & the shared filter).
Here is an overview of the solution:
What this demonstrates is that you add an extra field to your CDO titled Unmapped Record,
You then pass ALL records through a Program Builder program which updates any records NOT mapped to a contact in your shared Filter. (your update rule will set that field to 'true'.)
After that the CDO record services evaluates all updated records & sends matching ones to the program Canvas.
The Program canvas deletes any unmapped records passed into it.
The Shared Filter:
Very simply your shared Filter just contains All contacts in your contact table.
Most simply this is achieved by filtering on contacts whose Email Address contains '@', however if your instance is enabled for blank email addresses then also you should include all records whose email address field is blank.
The Update Rule:
Create an update Rule (Audience > Tools > Data tools) of entity Type Custom Object, which will be configured to set the 'Unmapped Record?' field in your Custom data Object to equal the value ‘true’
The Program Builder Program
Now you create a program Builder Program that takes Entity Type: CDO Records (selecting your chosen CDO) & the flow needs to be as follows:
The decision step verifies if the record has a mapped contact within the specified filter (Which should include all contacts.)
If the Record has no mapped contact then the Update Rule is run in Step 100, setting that record's value in the Unmapped Record field to 'true'
The CDO service will detect that the record was modified and because the condition is now met, the record will be pushed into your Program canvas.
The Program canvas.
Very simply, this receives the record passed into it, verifies if the record has a mapped contact (it should not have) & then deletes the record accordingly.
If any records happen to sneak in that still have mapped contacts, they are excluded (I added a week wait step just for testing purposes)
The CDO Record Services
| To configure your CDO record services within your CDO: |
|---|
Choose Custom Object > Custom Object Record Services Choose Modified Data Choose ‘Add Processing step’ From the Add single Processing step option choose ‘Add to step in Program’ (this will route to your Program Canvas) On the setup screen name your service. Choose Custom Object records from Entity Type In ‘This Processing Step Gets Executed’ set it to conditional: only when the field Unmapped Record = “true” Once this is set up you need to remember to Enable the service. |
Once this is all tied together, you should go to your Program builder & in the start step (Step 000), Add Members to step & add ALL CDO records.
To do this choose source: your CDO & do not restrict the members.
Due to the performance limitations of CDO services, if large volumes of records are involved. pushing these through during prime business hours could have a performance effect on your instance.
For very large volumes of records it is recommended to pass them through the program Builder in smaller batches.
Good Luck!
Comments
-
Great post, I like it how detailed you made it with the screenshots!
-
Great post, I like it how detailed you made it with the screenshots!
Hi Jan-Pieter,
Glad you liked it. - I thought a less ambiguous approach was wise here as it is on the subject of record deletion (which could go horribly wrong).I'm also hoping to do similar posts on other topics in the future. I hope they're all as well received as this one.
- Andy
-
Super useful. Thank you, Andy, for taking the time to document it.
-
Thank you.
-
Would you be able to assess the impact on overall pod performance given every client implemented this to regularly clean up all of their instance's CDO?
Also, with CDO Record Services processing 5000 records per 15 mins, this does not scale very well.
I believe, Oracle needs to come up with a better idea to comply with GDPR in this area.
-
Would you be able to assess the impact on overall pod performance given every client implemented this to regularly clean up all of their instance's CDO?
Also, with CDO Record Services processing 5000 records per 15 mins, this does not scale very well.
I believe, Oracle needs to come up with a better idea to comply with GDPR in this area.
Hi Claudia, Thank you for your feedback, - I do agree that pushing through large volumes of records may have a performance effect, therefore there is an advisory notice at the end of my post highlighting that these processes should be run out of hours.
Additionally it highlights that passing through in batches is recommended.This solution is designed as a starting point for Eloqua users to get rid of any historical unmapped records.
Because this is a partial manual process (involving the user adding all records into the program builder step 000), it is not designed to be an automated solution moving forward.
Once that unmapped record count is at a starting point of 0 then to automate CDO record deletion it would be necessary to set up some other mechanism that would allow you to clear out unwanted CDO records in advance of contact deletion, so that this situation wasn't repeated.
Clearly referencing CDO records mapped to existing contacts is far easier than trying to batch delete all existing unmapped records. But in situations where unmapped records already exist, this solution should act as a suitable way to get rid of them.Thanks,
Andy -
Hi Claudia, Thank you for your feedback, - I do agree that pushing through large volumes of records may have a performance effect, therefore there is an advisory notice at the end of my post highlighting that these processes should be run out of hours.
Additionally it highlights that passing through in batches is recommended.This solution is designed as a starting point for Eloqua users to get rid of any historical unmapped records.
Because this is a partial manual process (involving the user adding all records into the program builder step 000), it is not designed to be an automated solution moving forward.
Once that unmapped record count is at a starting point of 0 then to automate CDO record deletion it would be necessary to set up some other mechanism that would allow you to clear out unwanted CDO records in advance of contact deletion, so that this situation wasn't repeated.
Clearly referencing CDO records mapped to existing contacts is far easier than trying to batch delete all existing unmapped records. But in situations where unmapped records already exist, this solution should act as a suitable way to get rid of them.Thanks,
AndyThanks for the feedback, Andy.
I'm dealing with integrated systems mostly, and with automated Contact deletion being scarcely supported by Eloqua (E9 dedup and match rules), I certainly would want the integration to physically rather than logically delete a Contact.
I would also like to lessen the utilization of Program Builder over time, as well as avoid a frail scheduled import/export setup for Contact deletion post CDO deletion.
Hence, I'm continuously left with a significant number of unmapped CDO records .
I wonder, why Eloqua would not simply delete mapped CDO records when the “parent” Contact is deleted - similar to a lot of other systems dealing with related records?
-
Well explained... Thank you!
Regards,
Vashant
-
Thanks for the feedback, Andy.
I'm dealing with integrated systems mostly, and with automated Contact deletion being scarcely supported by Eloqua (E9 dedup and match rules), I certainly would want the integration to physically rather than logically delete a Contact.
I would also like to lessen the utilization of Program Builder over time, as well as avoid a frail scheduled import/export setup for Contact deletion post CDO deletion.
Hence, I'm continuously left with a significant number of unmapped CDO records .
I wonder, why Eloqua would not simply delete mapped CDO records when the “parent” Contact is deleted - similar to a lot of other systems dealing with related records?
i see, that makes sense.
I think the reason that Eloqua doesn't auto-delete CDO records is because there are some use cases where persistent data might need to be stored, even after a contact is deleted. Such an example of this would be under GDPR, where as I understand it, even after a contact has asked to be forgotten and you delete them from your marketing database, you are legally allowed to continue store any information essential to proving your compliance with the law (providing access is restricted to only your DPO, or other essential staff)
Therefore if you are audited you can provide logs that X user subscribed on date Y, where they opted in, - then opted out on Z date etc.
Obviously I am not a lawyer or solicitor, so anyone would need to verify this with their own legal counsel, but that is just one example of a persistent CDO that might need to be retained after a contact's deletion. If all CDOs were erased, then use cases like this would not be possible. -
Hey Andy! Awesome documentation here, a real time saver for us.
However I was told by a support rep that the "Delete CDO Records" program step has been deprecated; is this true?
"the Delete CDO Record step was part of a feature that was deprecated when we switched from E9 to E10 due to overwhelming negative feedback from our client base at the time. Unfortunately, the step hasn't been removed from the program, but that's why it's not functioning anymore."
-
Hey Andy! Awesome documentation here, a real time saver for us.
However I was told by a support rep that the "Delete CDO Records" program step has been deprecated; is this true?
"the Delete CDO Record step was part of a feature that was deprecated when we switched from E9 to E10 due to overwhelming negative feedback from our client base at the time. Unfortunately, the step hasn't been removed from the program, but that's why it's not functioning anymore."
Hi Tommy, I'm glad it was useful.
As regards the step deprecation - as the reference here is to the E9-E10 release which was in 2011, I don't think it's the same thing. my assumption is that refers to a Program Builder step that was deprecated?
The Program Canvas I am using in this solution was only released 18 months-2 years ago and as far as I'm aware it still functions exactly as intended.
I just checked and the step is still available in the Actions section of a CDO Program Canvas.
-
Hi Tommy, I'm glad it was useful.
As regards the step deprecation - as the reference here is to the E9-E10 release which was in 2011, I don't think it's the same thing. my assumption is that refers to a Program Builder step that was deprecated?
The Program Canvas I am using in this solution was only released 18 months-2 years ago and as far as I'm aware it still functions exactly as intended.
I just checked and the step is still available in the Actions section of a CDO Program Canvas.
Thanks for clarifying that. I have to assume they were referring to Program Builder as well.
-
Just an update for those who might be interested t hear that from release 19B there will be Controlled Availability feature that would add a checkbox to each CDO highlighting if records in that CDO should be deleted along with any contact deletion. this would simplify CDO maintenance going forward.
More information on this upcoming feature can be found in the 19B Product overview video on the Release centre: (Jump to time 15m 24s for the exact bit)
-
Just an update for those who might be interested t hear that from release 19B there will be Controlled Availability feature that would add a checkbox to each CDO highlighting if records in that CDO should be deleted along with any contact deletion. this would simplify CDO maintenance going forward.
More information on this upcoming feature can be found in the 19B Product overview video on the Release centre: (Jump to time 15m 24s for the exact bit)
Sweet, thanks Andy.
-
Thanks @Andy BaII-Oracle , I rememberd about it and found the post, just what i was looking for. Cheers !
