is there a way not to see ERP contact in Sales/CRM?

Summary:

currently we are having an issue wherein our CRM users see ERP contacts in workspace, but ERP users can't see the CRM Contacts,

is there away we can avoid our CRM users to see ERP Contacts?

our last option is to use access group, but just wondering if there is any other way to restrict our CRM users to see ERP contacts? like is there a specific job role or job role config that we need to change on our part? the reason why i asked this is that because ERP users are restricted to see CRM contacts.

else this will be my last option