Customer Contact relationship record error - You can't update column SUBJECT_ID/OBJECT_ID

Summary:

We are live on Sales cloud and already have customer & contact party in system and linked to each other. However, we are import AR customer accounts and linking contacts to their sites. For few customers, we get import error in relationship records - You can't update column {SUBJECT_ID/OBJECT_ID}.
- When we check HZ_RELATIONSHIP table, we see that the record with RELATIONSHIP_CODE = CONTACT_OF is marked as DIRECTIONAL_FLAG = B. For other records which import fine, it is set to F.
- When we check HZ_CUST_ACCOUNT_ROLES table, we see that CONTACT_PERSON_ID holds party id of customer and not contact.