Is it possible to restrict access by department yet manager hierarchy still works?

Summary:

We have hundreds of departments and about 5-7 of those departments we consider to be restricted access departments. A person who is in one of those departments should not have access to another person within the same department or one of the other restricted departments. We still want the manager hierarchy to overrule the department restriction.

Example - Human Resources Director is in Department A which is a restricted department Talent Acquisition Director is in Department B(also restricted). The HR Director should not see the TA Director and vice versa. The HR Director has a direct report in the job of Human Resources Supervisor. The HR Supervisor is in Deparment C(also restricted). Although departments A & C are restricted we want HR Director to still have access no matter the department their direct/indirect reports are in.