Segment Identity Manager access between HCM and ERP

Summary

We need separate and segmented off access to Identity Manager for HCM group and ERP group

Content

Is there a way to have an administrator within HCM and an administrator in ERP who both have access and full use of Identity Manager but to limit the ERP administrator from being able to add or remove HCM related roles, such as Payroll.  The idea here is have an admin withint ERP who can't access or have the ability to add/remove payroll roles or any other HCM related role for that matter.  

Is this an issue anyone else has had to deal with?  If so could you provide your solution or work-around?