Restrict Document Types to only specific users

Summary:

I want to restrict a few specific document types so that they are only visible by specific team members. I know about the Document Type Security Profile, but there are a lot of customized roles, and I'm not sure exactly which ones have access to document records of others. Plus, even then, I'd have to go to each role and change their Document Type Security? Is there a way to change it globally, and then 'add back in' the ones only those team members should see. I hope I'm asking the question correct. I swore there was previously restriction possible