Flexfield Value Set Security

Currently all users can insert/update all value sets if they have access to the menu option.  The seeded Grant was implemented (Flexfield Value Set Security:  All privileges).   We have designed our menus to exclude access to the Flexfield menus to those users that should not have access.  However, we have exceptions and  I want to give a few individuals access to a handful of value sets.  Can I modify the seeded Grant  to exclude these individuals (or responsibility) and then give those individuals (or responsibility) access to the specific values sets?