What are the cons of setting up a custom responsibility for Item Categories KFF Values?

I have a request for a limited set of end users to have access to add new category segment values to the existing value sets assigned to the Item Category KFF.  Today, we only have Admin users setting up values to existing KFFs. 

If we were able to grant users access only to a particular value set (or sets) in order to add to the LOV manually, would there be a negative impact?

Any lessons learned for others attempting this?