Question re: Value Set permission grants

We just upgraded to EBS HCM 12.2.11.

I have granted permission to Value Sets to a specific responsibility/user. But now it has come to light that other business group responsibilities/users have access to value sets they should not.

I've been asked to take the access to value sets down to the business group access. I've tested in a lower instance and can't seem to figure it out.

What I am missing? Or can the permission be granted at the business group level?

Thanks for any insight, Laura