Removing a customer item cross reference at the customer level

One of our business users inadvertently setup a customer item cross reference at the customer level instead of the address level.  We have inactivated it however when we try to setup the cross reference at the address level we receive an error indicating the cross reference already exists.  Is there something that can be done to clean up at the customer level and re-establish at the address level?