Apply dept security to Company Directory

We have created a company directory for the whole organisation.

However, we do not want to allow everyone to see the entire organisational structure, due to sheer volumn of employees in the tree (6000+)

Is there a way to restrict people to view only from a certain node within the tree? 

Or is there a way to restrict based on their row level security? (Such as only employees within the user's row level security are visible to the user)

Thanks