remove organization chart icon in advanced search

Hello,

we are configuring the system so that employees and managers are not able to view the company organization chart. However, as you can see in the photo attached, we are unable to remove it from the advanced search bar within the directory. 

Do you know how we could handle it?

Thank you,

Giulia